Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/optimizer opt_joinpath.mx, , 1.21, 1.22
On 2008-02-07 18:45, Romulo Goncalves wrote:
Update of /cvsroot/monetdb/MonetDB5/src/optimizer In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv22937
Modified Files: opt_joinpath.mx Log Message: Revert Martin code because it does not work. The code will remain, but commented for future improvements as Martin suggested...
A better way to comment larger sections of code is to use #if 0/#endif. That nests.
Index: opt_joinpath.mx
RCS file: /cvsroot/monetdb/MonetDB5/src/optimizer/opt_joinpath.mx,v retrieving revision 1.21 retrieving revision 1.22 diff -u -d -r1.21 -r1.22 --- opt_joinpath.mx 7 Feb 2008 16:57:18 -0000 1.21 +++ opt_joinpath.mx 7 Feb 2008 17:44:59 -0000 1.22 @@ -231,22 +231,30 @@ The join path optimizer takes a join sequence and attempts to minimize the intermediate result. The choice depends on a good estimate of intermediate -results.SAMPLE_THRESHOLD_lOG +results. For the time being, we use a simplistic model, based on the assumption that most joins are foreign key joins anyway.
We use a sample based approach for sizeable tables. -The model is derived from the select statement. +The model is derived from the select statement. However, we did not succeed. +The code is now commented for future improvement. +Final conclusion from this exercise is: +The difference between the join input size and the join output size is not +the correct (or unique) metric which should be used to decide which order +should be used in the joinPath.
@c -#define SAMPLE_THRESHOLD_lOG 17 +/*#define SAMPLE_THRESHOLD_lOG 17*/ size_t ALGjoinCost(BAT *l, BAT *r) {
- size_t lc, rc, lsize,rsize;
- BAT *lsample, *rsample, *j;
- size_t lc, rc;
+/* size_t lsize,rsize;
BAT *lsample, *rsample, *j; */
lc = BATcount(l); rc = BATcount(r);
- if( lc > 100000 || rc > 100000){
+/* if( lc > 100000 || rc > 100000){ lsize= MIN(lc/100, (1<<SAMPLE_THRESHOLD_lOG)/3); lsample= BATsample(l,lsize); BBPreclaim(lsample); @@ -257,7 +265,7 @@ lsize= BATcount(j); BBPreclaim(j); return lsize;
- }
- }*/ if( l->ttype== TYPE_oid || r->htype== TYPE_oid ) return MIN(lc,rc); return lc * rc;
This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ Monetdb-checkins mailing list Monetdb-checkins@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
Sjoerd Mullender wrote:
On 2008-02-07 18:45, Romulo Goncalves wrote:
Update of /cvsroot/monetdb/MonetDB5/src/optimizer In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv22937
Modified Files: opt_joinpath.mx Log Message: Revert Martin code because it does not work. The code will remain, but commented for future improvements as Martin suggested...
A better way to comment larger sections of code is to use #if 0/#endif.
#ifdef 0
#endif
does not work: mizer_la-opt_joinpath.o /ufs/goncalve/MonetDB/MonetDB5/src/optimizer/opt_joinpath.mx:257:8: error: macro names must be identifiers make[5]: *** [lib_optimizer_la-opt_joinpath.lo] Error 1 make[5]: Leaving directory `/export/scratch1/goncalve/MonetDB/debug/MonetDB5/build/src/optimizer' make[4]: *** [all] Error 2 make[4]: Leaving directory `/export/scratch1/goncalve/MonetDB/debug/MonetDB5/build/src/optimizer' make[3]: *** [all-recursive] Error 1 make[3]: Leaving directory `/export/scratch1/goncalve/MonetDB/debug/MonetDB5/build/src' make[2]: *** [all] Error 2 make[2]: Leaving directory `/export/scratch1/goncalve/MonetDB/debug/MonetDB5/build/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/export/scratch1/goncalve/MonetDB/debug/MonetDB5/build' make: *** [all] Error 2 [goncalve@amelia build]$ vim ~/MonetDB/MonetDB5/src/optimizer/opt_joinpath.mx
Romulo
That nests.
Index: opt_joinpath.mx
RCS file: /cvsroot/monetdb/MonetDB5/src/optimizer/opt_joinpath.mx,v retrieving revision 1.21 retrieving revision 1.22 diff -u -d -r1.21 -r1.22 --- opt_joinpath.mx 7 Feb 2008 16:57:18 -0000 1.21 +++ opt_joinpath.mx 7 Feb 2008 17:44:59 -0000 1.22 @@ -231,22 +231,30 @@ The join path optimizer takes a join sequence and attempts to minimize the intermediate result. The choice depends on a good estimate of intermediate -results.SAMPLE_THRESHOLD_lOG +results. For the time being, we use a simplistic model, based on the assumption that most joins are foreign key joins anyway.
We use a sample based approach for sizeable tables. -The model is derived from the select statement. +The model is derived from the select statement. However, we did not succeed. +The code is now commented for future improvement. +Final conclusion from this exercise is: +The difference between the join input size and the join output size is not +the correct (or unique) metric which should be used to decide which order +should be used in the joinPath.
@c -#define SAMPLE_THRESHOLD_lOG 17 +/*#define SAMPLE_THRESHOLD_lOG 17*/ size_t ALGjoinCost(BAT *l, BAT *r) {
- size_t lc, rc, lsize,rsize;
- BAT *lsample, *rsample, *j;
- size_t lc, rc;
+/* size_t lsize,rsize;
BAT *lsample, *rsample, *j; */
lc = BATcount(l); rc = BATcount(r);
- if( lc > 100000 || rc > 100000){
+/* if( lc > 100000 || rc > 100000){ lsize= MIN(lc/100, (1<<SAMPLE_THRESHOLD_lOG)/3); lsample= BATsample(l,lsize); BBPreclaim(lsample); @@ -257,7 +265,7 @@ lsize= BATcount(j); BBPreclaim(j); return lsize;
- }
- }*/ if( l->ttype== TYPE_oid || r->htype== TYPE_oid ) return MIN(lc,rc); return lc * rc;
This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ Monetdb-checkins mailing list Monetdb-checkins@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
On Thu, Feb 07, 2008 at 08:00:24PM +0100, Romulo Goncalves wrote:
Sjoerd Mullender wrote:
On 2008-02-07 18:45, Romulo Goncalves wrote:
Update of /cvsroot/monetdb/MonetDB5/src/optimizer In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv22937
Modified Files: opt_joinpath.mx Log Message: Revert Martin code because it does not work. The code will remain, but commented for future improvements as Martin suggested...
A better way to comment larger sections of code is to use #if 0/#endif.
^^^^^
#ifdef 0
^^^!!!^^
no need not to say more, I gues ...
Stefan
#endif
does not work: mizer_la-opt_joinpath.o /ufs/goncalve/MonetDB/MonetDB5/src/optimizer/opt_joinpath.mx:257:8: error: macro names must be identifiers make[5]: *** [lib_optimizer_la-opt_joinpath.lo] Error 1 make[5]: Leaving directory `/export/scratch1/goncalve/MonetDB/debug/MonetDB5/build/src/optimizer' make[4]: *** [all] Error 2 make[4]: Leaving directory `/export/scratch1/goncalve/MonetDB/debug/MonetDB5/build/src/optimizer' make[3]: *** [all-recursive] Error 1 make[3]: Leaving directory `/export/scratch1/goncalve/MonetDB/debug/MonetDB5/build/src' make[2]: *** [all] Error 2 make[2]: Leaving directory `/export/scratch1/goncalve/MonetDB/debug/MonetDB5/build/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/export/scratch1/goncalve/MonetDB/debug/MonetDB5/build' make: *** [all] Error 2 [goncalve@amelia build]$ vim ~/MonetDB/MonetDB5/src/optimizer/opt_joinpath.mx
Romulo
That nests.
Index: opt_joinpath.mx
RCS file: /cvsroot/monetdb/MonetDB5/src/optimizer/opt_joinpath.mx,v retrieving revision 1.21 retrieving revision 1.22 diff -u -d -r1.21 -r1.22 --- opt_joinpath.mx 7 Feb 2008 16:57:18 -0000 1.21 +++ opt_joinpath.mx 7 Feb 2008 17:44:59 -0000 1.22 @@ -231,22 +231,30 @@ The join path optimizer takes a join sequence and attempts to minimize the intermediate result. The choice depends on a good estimate of intermediate -results.SAMPLE_THRESHOLD_lOG +results. For the time being, we use a simplistic model, based on the assumption that most joins are foreign key joins anyway.
We use a sample based approach for sizeable tables. -The model is derived from the select statement. +The model is derived from the select statement. However, we did not succeed. +The code is now commented for future improvement. +Final conclusion from this exercise is: +The difference between the join input size and the join output size is not +the correct (or unique) metric which should be used to decide which order +should be used in the joinPath.
@c -#define SAMPLE_THRESHOLD_lOG 17 +/*#define SAMPLE_THRESHOLD_lOG 17*/ size_t ALGjoinCost(BAT *l, BAT *r) {
- size_t lc, rc, lsize,rsize;
- BAT *lsample, *rsample, *j;
- size_t lc, rc;
+/* size_t lsize,rsize;
BAT *lsample, *rsample, *j; */
lc = BATcount(l); rc = BATcount(r);
- if( lc > 100000 || rc > 100000){
+/* if( lc > 100000 || rc > 100000){ lsize= MIN(lc/100, (1<<SAMPLE_THRESHOLD_lOG)/3); lsample= BATsample(l,lsize); BBPreclaim(lsample); @@ -257,7 +265,7 @@ lsize= BATcount(j); BBPreclaim(j); return lsize;
- }
- }*/ if( l->ttype== TYPE_oid || r->htype== TYPE_oid ) return MIN(lc,rc); return lc * rc;
This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ Monetdb-checkins mailing list Monetdb-checkins@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ Monetdb-developers mailing list Monetdb-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-developers
Romulo Goncalves wrote:
Sjoerd Mullender wrote:
On 2008-02-07 18:45, Romulo Goncalves wrote:
Update of /cvsroot/monetdb/MonetDB5/src/optimizer In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv22937
Modified Files: opt_joinpath.mx Log Message: Revert Martin code because it does not work. The code will remain, but commented for future improvements as Martin suggested...
A better way to comment larger sections of code is to use #if 0/#endif.
#ifdef 0
Is it ok if I use #ifdef SAMPLE_THRESHOLD_lOG
#endif
Because
#define SAMPLE_THRESHOLD_lOG is commented?
Or is there a better way to do it?
Romulo
#endif
does not work: mizer_la-opt_joinpath.o /ufs/goncalve/MonetDB/MonetDB5/src/optimizer/opt_joinpath.mx:257:8: error: macro names must be identifiers make[5]: *** [lib_optimizer_la-opt_joinpath.lo] Error 1 make[5]: Leaving directory `/export/scratch1/goncalve/MonetDB/debug/MonetDB5/build/src/optimizer' make[4]: *** [all] Error 2 make[4]: Leaving directory `/export/scratch1/goncalve/MonetDB/debug/MonetDB5/build/src/optimizer' make[3]: *** [all-recursive] Error 1 make[3]: Leaving directory `/export/scratch1/goncalve/MonetDB/debug/MonetDB5/build/src' make[2]: *** [all] Error 2 make[2]: Leaving directory `/export/scratch1/goncalve/MonetDB/debug/MonetDB5/build/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/export/scratch1/goncalve/MonetDB/debug/MonetDB5/build' make: *** [all] Error 2 [goncalve@amelia build]$ vim ~/MonetDB/MonetDB5/src/optimizer/opt_joinpath.mx
Romulo
That nests.
Index: opt_joinpath.mx
RCS file: /cvsroot/monetdb/MonetDB5/src/optimizer/opt_joinpath.mx,v retrieving revision 1.21 retrieving revision 1.22 diff -u -d -r1.21 -r1.22 --- opt_joinpath.mx 7 Feb 2008 16:57:18 -0000 1.21 +++ opt_joinpath.mx 7 Feb 2008 17:44:59 -0000 1.22 @@ -231,22 +231,30 @@ The join path optimizer takes a join sequence and attempts to minimize the intermediate result. The choice depends on a good estimate of intermediate -results.SAMPLE_THRESHOLD_lOG +results. For the time being, we use a simplistic model, based on the assumption that most joins are foreign key joins anyway.
We use a sample based approach for sizeable tables. -The model is derived from the select statement. +The model is derived from the select statement. However, we did not succeed. +The code is now commented for future improvement. +Final conclusion from this exercise is: +The difference between the join input size and the join output size is not +the correct (or unique) metric which should be used to decide which order +should be used in the joinPath.
@c -#define SAMPLE_THRESHOLD_lOG 17 +/*#define SAMPLE_THRESHOLD_lOG 17*/ size_t ALGjoinCost(BAT *l, BAT *r) {
- size_t lc, rc, lsize,rsize;
- BAT *lsample, *rsample, *j;
- size_t lc, rc;
+/* size_t lsize,rsize;
BAT *lsample, *rsample, *j; */
lc = BATcount(l); rc = BATcount(r);
- if( lc > 100000 || rc > 100000){
+/* if( lc > 100000 || rc > 100000){ lsize= MIN(lc/100, (1<<SAMPLE_THRESHOLD_lOG)/3); lsample= BATsample(l,lsize); BBPreclaim(lsample); @@ -257,7 +265,7 @@ lsize= BATcount(j); BBPreclaim(j); return lsize;
- }
- }*/ if( l->ttype== TYPE_oid || r->htype== TYPE_oid ) return MIN(lc,rc); return lc * rc;
This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ Monetdb-checkins mailing list Monetdb-checkins@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ Monetdb-developers mailing list Monetdb-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-developers
On 07-02-2008 20:04:42 +0100, Romulo Goncalves wrote:
Romulo Goncalves wrote:
Sjoerd Mullender wrote:
On 2008-02-07 18:45, Romulo Goncalves wrote:
Update of /cvsroot/monetdb/MonetDB5/src/optimizer In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv22937
Modified Files: opt_joinpath.mx Log Message: Revert Martin code because it does not work. The code will remain, but commented for future improvements as Martin suggested...
A better way to comment larger sections of code is to use #if 0/#endif.
#ifdef 0
Is it ok if I use #ifdef SAMPLE_THRESHOLD_lOG
#endif
Because
#define SAMPLE_THRESHOLD_lOG is commented?
Or is there a better way to do it?
Watch Sjoerd's suggestion closely:
#if 0
:)
Fabian Groffen wrote:
On 07-02-2008 20:04:42 +0100, Romulo Goncalves wrote:
Romulo Goncalves wrote:
Sjoerd Mullender wrote:
On 2008-02-07 18:45, Romulo Goncalves wrote:
Update of /cvsroot/monetdb/MonetDB5/src/optimizer In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv22937
Modified Files: opt_joinpath.mx Log Message: Revert Martin code because it does not work. The code will remain, but commented for future improvements as Martin suggested...
A better way to comment larger sections of code is to use #if 0/#endif.
#ifdef 0
Is it ok if I use #ifdef SAMPLE_THRESHOLD_lOG
#endif
Because
#define SAMPLE_THRESHOLD_lOG is commented?
Or is there a better way to do it?
Watch Sjoerd's suggestion closely:
#if 0
:)
Xiiii that was stupid one, time for a break....
Thanks...
Romulo
PS: Thanks to all that made me aware of such mistake :D
This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ Monetdb-developers mailing list Monetdb-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-developers
On 2008-02-07 20:00, Romulo Goncalves wrote:
Sjoerd Mullender wrote:
On 2008-02-07 18:45, Romulo Goncalves wrote:
Update of /cvsroot/monetdb/MonetDB5/src/optimizer In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv22937
Modified Files: opt_joinpath.mx Log Message: Revert Martin code because it does not work. The code will remain, but commented for future improvements as Martin suggested...
A better way to comment larger sections of code is to use #if 0/#endif.
#ifdef 0
#if 0
Read what I write! ;-)
#endif
does not work: mizer_la-opt_joinpath.o /ufs/goncalve/MonetDB/MonetDB5/src/optimizer/opt_joinpath.mx:257:8: error: macro names must be identifiers make[5]: *** [lib_optimizer_la-opt_joinpath.lo] Error 1 make[5]: Leaving directory `/export/scratch1/goncalve/MonetDB/debug/MonetDB5/build/src/optimizer' make[4]: *** [all] Error 2 make[4]: Leaving directory `/export/scratch1/goncalve/MonetDB/debug/MonetDB5/build/src/optimizer' make[3]: *** [all-recursive] Error 1 make[3]: Leaving directory `/export/scratch1/goncalve/MonetDB/debug/MonetDB5/build/src' make[2]: *** [all] Error 2 make[2]: Leaving directory `/export/scratch1/goncalve/MonetDB/debug/MonetDB5/build/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/export/scratch1/goncalve/MonetDB/debug/MonetDB5/build' make: *** [all] Error 2 [goncalve@amelia build]$ vim ~/MonetDB/MonetDB5/src/optimizer/opt_joinpath.mx
Romulo
That nests.
Index: opt_joinpath.mx
RCS file: /cvsroot/monetdb/MonetDB5/src/optimizer/opt_joinpath.mx,v retrieving revision 1.21 retrieving revision 1.22 diff -u -d -r1.21 -r1.22 --- opt_joinpath.mx 7 Feb 2008 16:57:18 -0000 1.21 +++ opt_joinpath.mx 7 Feb 2008 17:44:59 -0000 1.22 @@ -231,22 +231,30 @@ The join path optimizer takes a join sequence and attempts to minimize the intermediate result. The choice depends on a good estimate of intermediate -results.SAMPLE_THRESHOLD_lOG +results. For the time being, we use a simplistic model, based on the assumption that most joins are foreign key joins anyway.
We use a sample based approach for sizeable tables. -The model is derived from the select statement. +The model is derived from the select statement. However, we did not succeed. +The code is now commented for future improvement. +Final conclusion from this exercise is: +The difference between the join input size and the join output size is not +the correct (or unique) metric which should be used to decide which order +should be used in the joinPath.
@c -#define SAMPLE_THRESHOLD_lOG 17 +/*#define SAMPLE_THRESHOLD_lOG 17*/ size_t ALGjoinCost(BAT *l, BAT *r) {
- size_t lc, rc, lsize,rsize;
- BAT *lsample, *rsample, *j;
- size_t lc, rc;
+/* size_t lsize,rsize;
BAT *lsample, *rsample, *j; */
lc = BATcount(l); rc = BATcount(r);
- if( lc > 100000 || rc > 100000){
+/* if( lc > 100000 || rc > 100000){ lsize= MIN(lc/100, (1<<SAMPLE_THRESHOLD_lOG)/3); lsample= BATsample(l,lsize); BBPreclaim(lsample); @@ -257,7 +265,7 @@ lsize= BATcount(j); BBPreclaim(j); return lsize;
- }
- }*/ if( l->ttype== TYPE_oid || r->htype== TYPE_oid ) return MIN(lc,rc); return lc * rc;
This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ Monetdb-checkins mailing list Monetdb-checkins@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ Monetdb-developers mailing list Monetdb-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-developers
participants (4)
-
Fabian Groffen -
Romulo Goncalves -
Sjoerd Mullender -
Stefan Manegold