developers-list

Download

developers-list@monetdb.org

February 2012

6 participants
11 discussions

Re: [Monetdb-developers] MonetDB: default - More advice on the optimizer template.
by Stefan Manegold 11 Feb '12

11 Feb '12

On Wed, Feb 08, 2012 at 10:27:11AM +0100, Martin Kersten wrote: > Changeset: 67c12a700166 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=67c12a700166 > Modified Files: > monetdb5/extras/mal_optimizer_template/opt_sql_append.mx > Branch: default > Log Message: > > More advice on the optimizer template. > > > diffs (140 lines): > > diff --git a/monetdb5/extras/mal_optimizer_template/opt_sql_append.mx b/monetdb5/extras/mal_optimizer_template/opt_sql_append.mx > --- a/monetdb5/extras/mal_optimizer_template/opt_sql_append.mx > +++ b/monetdb5/extras/mal_optimizer_template/opt_sql_append.mx [...] > @@ -39,6 +39,8 @@ All Rights Reserved. > * i.e., an sql.append() statement that is eventually followed by some other > * statement later on in the MAL program that uses the same v0 BAT as > * argument as the sql.append() statement does, > + * Do you assume a single re-use of the variable v0? No. Why? > + * Do you assume a non-nested MAL block ? Not necessarily. I must admit, that I do not know how the oprimizer framework handles nested MAL blocks, and what an optimizer needs to do to be aware of nested MAL blocks and to handle them correctly. In the sample optimizer, for now, I'd be fine if there are no false-positives, i.e., the optimizer triggers in case it should not trigger or in cases it cannot handle correctly. I can accept false-negatives, i.e., not triggering in all case it could handle correctly. > * > * and transform them into > * > @@ -52,6 +54,7 @@ All Rights Reserved. > * > * i.e., handing a BAT view v2 of BAT v0 as argument to the sql.append() > * statement, rather than the original BAT v0. > + * My advice, always use new variable names, it may capture some easy to make errors. I/my optimizer does use new variables for all new statements/results. I/my optimizer re-use variable names only for identical results. > * > * As a refinement, patterns like > * [...] > @@ -181,13 +195,17 @@ OPTsql_appendImplementation(Client cntxt > pushInstruction(mb, q); > q1 = q; > i++; > - actions++; > + actions++; /* to keep track if anything has been done */ > } > } > > - /* look for > + /* look for > * v5 := ... v0 ...; > */ > + /* an expensive loop, better would be to remember that v0 has a different role. > + * A typical method is to keep a map from variable -> instruction where it was > + * detected. The you can check each assignment for use of v0 > + */ This is general support functionality. Is this already available in the optimizer framework? If so, where is it and how can I use it? If not, where/how could we add it? > for (j = i+1; !found && j < limit; j++) > for (k = old[j]->retc; !found && k < old[j]->argc; k++) > found = (getArg(old[j], k) == getArg(p, 5)); > @@ -202,6 +220,8 @@ OPTsql_appendImplementation(Client cntxt > > /* push new v1 := aggr.count( v0 ); unless already available */ > if (q1 == NULL) { > + /* use mal_buil.mx primitives q1 = newStmt(mb, aggrRef,countRef); setArgType(mb,q1,TYPE_wrd) */ > + /* it will be added to the block and even my re-use MAL instructions */ Is this (supposed to be) documentation of the existing code below, or rather advice how to implement the below functionality differently? > q1 = newInstruction(mb,ASSIGNsymbol); > getArg(q1,0) = newTmpVariable(mb, TYPE_wrd); > setModuleId(q1, aggrRef); > @@ -211,6 +231,7 @@ OPTsql_appendImplementation(Client cntxt > } > > /* push new v2 := algebra.slice( v0, 0, v1 ); */ > + /* use mal_buil.mx primitives q1 = newStmt(mb, algebraRef,sliceRef); */ Is this (supposed to be) documentation of the existing code below, or rather advice how to implement the below functionality differently? > q2 = newInstruction(mb,ASSIGNsymbol); > getArg(q2,0) = newTmpVariable(mb, TYPE_any); > setModuleId(q2, algebraRef); > @@ -240,6 +261,7 @@ OPTsql_appendImplementation(Client cntxt > for(i++; i<limit; i++) > if (old[i]) > pushInstruction(mb, old[i]); > + /* any remaining MAL instruction records are removed */ > for(; i<slimit; i++) > if (old[i]) > freeInstruction(old[i]); > @@ -253,6 +275,9 @@ OPTsql_appendImplementation(Client cntxt > return actions; > } > > +/* optimizers have to be registered in the optcatalog in opt_support.c. Why? If at all possible, I'd prefer to be able to add a new optimizer without the need to change existing code ... > + * you have to path the file accordingly. ^^^^ parse? What does this mean? What am I supposed to do in detail? > + */ > @include ../../optimizer/optimizerWrapper.mx > @c > #include "opt_statistics.h" > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > Thanks! Stefan -- | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) | | http://CWI.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |

1 0