February 2009 - developers-list

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/common sql_types.mx, Feb2009, 1.121.2.1, 1.121.2.2
by Stefan Manegold 27 Feb '09

27 Feb '09

FYI: This checkin seems to change the DB schema, making existing databases incompatible with the code after this change: !FATAL: SQL schema missing or incompatible, rebuild from archive Stefan On Thu, Feb 26, 2009 at 04:56:32PM +0000, Roberto Cornacchia wrote: > Update of /cvsroot/monetdb/sql/src/common > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv29270/common > > Modified Files: > Tag: Feb2009 > sql_types.mx > Log Message: > added prod() aggregation function in SQL > > Not part of standard SQL, but useful and available in most SQL DBs > > > > U sql_types.mx > Index: sql_types.mx > =================================================================== > RCS file: /cvsroot/monetdb/sql/src/common/sql_types.mx,v > retrieving revision 1.121.2.1 > retrieving revision 1.121.2.2 > diff -u -d -r1.121.2.1 -r1.121.2.2 > --- sql_types.mx 11 Feb 2009 19:35:44 -0000 1.121.2.1 > +++ sql_types.mx 26 Feb 2009 16:56:29 -0000 1.121.2.2 > @@ -1361,6 +1361,35 @@ > t++; /* LNG */ > sql_create_aggr("sum", "aggr", "sum", *(t), *(t)); > > + /* prod for numerical and decimals */ > + sql_create_aggr("prod", "aggr", "product", BTE, SHT); > + sql_create_aggr("prod", "aggr", "product", BTE, INT); > + sql_create_aggr("prod", "aggr", "product", BTE, WRD); > + sql_create_aggr("prod", "aggr", "product", BTE, LNG); > + > + sql_create_aggr("prod", "aggr", "product", SHT, INT); > + sql_create_aggr("prod", "aggr", "product", SHT, WRD); > + sql_create_aggr("prod", "aggr", "product", SHT, LNG); > + > + sql_create_aggr("prod", "aggr", "product", INT, WRD); > + sql_create_aggr("prod", "aggr", "product", INT, LNG); > + > + sql_create_aggr("prod", "aggr", "product", WRD, WRD); > + sql_create_aggr("prod", "aggr", "product", WRD, LNG); > + sql_create_aggr("prod", "aggr", "product", LNG, LNG); > + > + t = decimals; /* BTE */ > + sql_create_aggr("prod", "aggr", "product", *(t), *(t+1)); > + sql_create_aggr("prod", "aggr", "product", *(t), *(t+2)); > + sql_create_aggr("prod", "aggr", "product", *(t), *(t+3)); > + t++; /* SHT */ > + sql_create_aggr("prod", "aggr", "product", *(t), *(t+1)); > + sql_create_aggr("prod", "aggr", "product", *(t), *(t+2)); > + t++; /* INT */ > + sql_create_aggr("prod", "aggr", "product", *(t), *(t+1)); > + t++; /* LNG */ > + sql_create_aggr("prod", "aggr", "product", *(t), *(t)); > + > for (t = numerical; t < floats; t++) { > sql_create_aggr("avg", "aggr", "avg", *(t), DBL); > sql_create_func("mod", "calc", "%", *t, *t, *t, SCALE_FIX); > @@ -1368,6 +1397,7 @@ > > for (t = floats; t < dates; t++) { > sql_create_aggr("sum", "aggr", "sum", *t, *t); > + sql_create_aggr("prod", "aggr", "product", *t, *t); > sql_create_aggr("avg", "aggr", "avg", *t, DBL); > > sql_create_func("mod", "calc", "fmod", *t, *t, *t, SCALE_FIX); > > > ------------------------------------------------------------------------------ > Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA > -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise > -Strategies to boost innovation and cut costs with open source participation > -Receive a $600 discount off the registration fee with the source code: SFAD > http://p.sf.net/sfu/XcvMzF8H > _______________________________________________ > Monetdb-sql-checkins mailing list > Monetdb-sql-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-sql-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/common sql_types.mx, Feb2009, 1.121.2.2, 1.121.2.3
by Stefan Manegold 27 Feb '09

27 Feb '09

On Thu, Feb 26, 2009 at 11:47:57PM +0000, Roberto Cornacchia wrote: > Update of /cvsroot/monetdb/sql/src/common > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv3519/common > > Modified Files: > Tag: Feb2009 > sql_types.mx > Log Message: > reverting addition of prob() aggregation, as it would break db compatibility Thanks! (In particular for properly doing it using CVS commands, only!) I prevented propagation of this "undo" to the developlment trunk; the "do" had already been propagated yesterday; hence, the prod() aggregation is now avilable in the development trunk. Stefan > > > U sql_types.mx > Index: sql_types.mx > =================================================================== > RCS file: /cvsroot/monetdb/sql/src/common/sql_types.mx,v > retrieving revision 1.121.2.2 > retrieving revision 1.121.2.3 > diff -u -d -r1.121.2.2 -r1.121.2.3 > --- sql_types.mx 26 Feb 2009 16:56:29 -0000 1.121.2.2 > +++ sql_types.mx 26 Feb 2009 23:47:55 -0000 1.121.2.3 > @@ -1361,35 +1361,6 @@ > t++; /* LNG */ > sql_create_aggr("sum", "aggr", "sum", *(t), *(t)); > > - /* prod for numerical and decimals */ > - sql_create_aggr("prod", "aggr", "product", BTE, SHT); > - sql_create_aggr("prod", "aggr", "product", BTE, INT); > - sql_create_aggr("prod", "aggr", "product", BTE, WRD); > - sql_create_aggr("prod", "aggr", "product", BTE, LNG); > - > - sql_create_aggr("prod", "aggr", "product", SHT, INT); > - sql_create_aggr("prod", "aggr", "product", SHT, WRD); > - sql_create_aggr("prod", "aggr", "product", SHT, LNG); > - > - sql_create_aggr("prod", "aggr", "product", INT, WRD); > - sql_create_aggr("prod", "aggr", "product", INT, LNG); > - > - sql_create_aggr("prod", "aggr", "product", WRD, WRD); > - sql_create_aggr("prod", "aggr", "product", WRD, LNG); > - sql_create_aggr("prod", "aggr", "product", LNG, LNG); > - > - t = decimals; /* BTE */ > - sql_create_aggr("prod", "aggr", "product", *(t), *(t+1)); > - sql_create_aggr("prod", "aggr", "product", *(t), *(t+2)); > - sql_create_aggr("prod", "aggr", "product", *(t), *(t+3)); > - t++; /* SHT */ > - sql_create_aggr("prod", "aggr", "product", *(t), *(t+1)); > - sql_create_aggr("prod", "aggr", "product", *(t), *(t+2)); > - t++; /* INT */ > - sql_create_aggr("prod", "aggr", "product", *(t), *(t+1)); > - t++; /* LNG */ > - sql_create_aggr("prod", "aggr", "product", *(t), *(t)); > - > for (t = numerical; t < floats; t++) { > sql_create_aggr("avg", "aggr", "avg", *(t), DBL); > sql_create_func("mod", "calc", "%", *t, *t, *t, SCALE_FIX); > @@ -1397,7 +1368,6 @@ > > for (t = floats; t < dates; t++) { > sql_create_aggr("sum", "aggr", "sum", *t, *t); > - sql_create_aggr("prod", "aggr", "product", *t, *t); > sql_create_aggr("avg", "aggr", "avg", *t, DBL); > > sql_create_func("mod", "calc", "fmod", *t, *t, *t, SCALE_FIX); > > > ------------------------------------------------------------------------------ > Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA > -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise > -Strategies to boost innovation and cut costs with open source participation > -Receive a $600 discount off the registration fee with the source code: SFAD > http://p.sf.net/sfu/XcvMzF8H > _______________________________________________ > Monetdb-sql-checkins mailing list > Monetdb-sql-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-sql-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 2

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/optimizer opt_accumulators.mx, Feb2009, 1.32, 1.32.2.1 opt_aliases.mx, Feb2009, 1.24, 1.24.2.1 opt_coercion.mx, Feb2009, 1.14, 1.14.2.1 opt_commonTerms.mx, Feb2009, 1.49, 1.49.2.1 opt_costModel.mx, Feb2009, 1.39, 1.39.2.1 opt_dataflow.mx, Feb2009, 1.21, 1.21.2.1 opt_deadcode.mx, Feb2009, 1.40, 1.40.2.1 opt_emptySet.mx, Feb2009, 1.54, 1.54.2.1 opt_evaluate.mx, Feb2009, 1.22, 1.22.2.1 opt_garbageCollector.mx, Feb2009, 1.45.2.2, 1.45.2.3 opt_history.mx,
by Stefan Manegold 25 Feb '09

25 Feb '09

On Wed, Feb 25, 2009 at 02:19:44PM +0000, Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/optimizer > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv25174 > > Modified Files: > Tag: Feb2009 > opt_accumulators.mx opt_aliases.mx opt_coercion.mx > opt_commonTerms.mx opt_costModel.mx opt_dataflow.mx > opt_deadcode.mx opt_emptySet.mx opt_evaluate.mx > opt_garbageCollector.mx opt_history.mx opt_inline.mx > opt_joinpath.mx opt_mergetable.mx opt_mitosis.mx > opt_multiplex.mx opt_prelude.mx opt_reduce.mx opt_remap.mx > optimizer.mx > Log Message: > Give a summary of the optimizer decisions and effects when --optimizers is > used. > [...] > > U opt_commonTerms.mx > Index: opt_commonTerms.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/optimizer/opt_commonTerms.mx,v > retrieving revision 1.49 > retrieving revision 1.49.2.1 > diff -u -d -r1.49 -r1.49.2.1 > --- opt_commonTerms.mx 27 Jan 2009 22:29:48 -0000 1.49 > +++ opt_commonTerms.mx 25 Feb 2009 14:19:41 -0000 1.49.2.1 > @@ -244,6 +244,9 @@ > if( old[i]) > freeInstruction(old[i]); > GDKfree(old); > + PROPDEBUG ^^^^ shouldn't this one read "OPTDEBUG" ? Stefan > + if (actions) > + stream_printf(cntxt->fdout,"#opt_commonTerms: %d statemtents catched\n",actions); > return actions; > } > [...] -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/optimizer opt_garbageCollector.mx, Feb2009, 1.45, 1.45.2.1
by Stefan Manegold 25 Feb '09

25 Feb '09

I could isolate this checkin as being the cuase that made most SQL tests segfault during testin glast night. Stefan On Tue, Feb 24, 2009 at 11:17:19PM +0000, Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/optimizer > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv25714 > > Modified Files: > Tag: Feb2009 > opt_garbageCollector.mx > Log Message: > Keep the optimizer statements. > > > U opt_garbageCollector.mx > Index: opt_garbageCollector.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/optimizer/opt_garbageCollector.mx,v > retrieving revision 1.45 > retrieving revision 1.45.2.1 > diff -u -d -r1.45 -r1.45.2.1 > --- opt_garbageCollector.mx 7 Jan 2009 14:15:13 -0000 1.45 > +++ opt_garbageCollector.mx 24 Feb 2009 23:17:16 -0000 1.45.2.1 > @@ -218,8 +218,11 @@ > } > #endif > for(; i<slimit; i++) > + if( old[i] == pci) > + freeInstruction(pci); > + else > if( old[i]) > - freeInstruction(old[i]); > + pushInstruction(mb,old[i]); > GDKfree(old); > GDKfree(span); > return actions; > > > ------------------------------------------------------------------------------ > Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA > -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise > -Strategies to boost innovation and cut costs with open source participation > -Receive a $600 discount off the registration fee with the source code: SFAD > http://p.sf.net/sfu/XcvMzF8H > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

[Monetdb-developers] monetdb-client for python doesn't seem to work for booleans.
by Vikram Bhandoh 23 Feb '09

23 Feb '09

Hi I've been working on a django backend for monetdb. For people who are interested in it have a look at it http://github.com/vikram/monetdb/tree/master I found a couple of issues with the python monetdb-client which are important for python. It doesn't seem to support boolean fields and doesn't give the lastrowid. I have a patch here for both. I'm sure about the lastrowid. As I haven't tested it in anger, could anyone tell me if this is alright or have I got things completely wrong. Thanks Vikram diff MonetSQLdb/converters.py ./converters.py 120,125d119 < def Boolean2Str(x, d): < if x: < return 'TRUE' < else: < return 'FALSE' < 144d137 < types.BooleanType: Boolean2Str, diff MonetSQLdb/cursors.py ./cursors.py 38d37 < self.lastrowid = 0 82d80 < 132,134d129 < if self._result.lastid()>0: < self.lastrowid = self._result.lastid() <

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime shredder.mx, Feb2009, 1.153.2.5, 1.153.2.6
by Stefan Manegold 20 Feb '09

20 Feb '09

On Fri, Feb 20, 2009 at 02:18:38PM +0000, Jan Flokstra wrote: > Update of /cvsroot/monetdb/pathfinder/runtime > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv16245 > > Modified Files: > Tag: Feb2009 > shredder.mx > Log Message: > - fixed shred_characters() bug, sorry :-( Testing suggests that this indeed fixes the previously reported errors / failing tests, again. Thanks! We would *VERY MUCH APPRECIATE*, though, if you could run Mtest yourself --- and check the results to ensure that your changes do not break previously working code & tests --- *before* checking in, at least when bugfixing on the stable branch just before a planned release... Running Mtest costs (each of) you just 12 key strokes (`Mtest.py -r[Enter]`) and (even for pathfinder) takes much less time than it takes us to sort out the problems of many checkins by different people, and Sjoerd to re-build the release candidates more often than necessary ... (Once again) thank you very much in advance for you cooperation! Stefan > > > U shredder.mx > Index: shredder.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/shredder.mx,v > retrieving revision 1.153.2.5 > retrieving revision 1.153.2.6 > diff -u -d -r1.153.2.5 -r1.153.2.6 > --- shredder.mx 20 Feb 2009 13:06:50 -0000 1.153.2.5 > +++ shredder.mx 20 Feb 2009 14:18:35 -0000 1.153.2.6 > @@ -1105,9 +1105,20 @@ > const xmlChar *cs, > int n) > { > +#ifdef OLD_SHRED_CHARACTERS > shredCtxStruct *shredCtx = (shredCtxStruct*) xmlCtx; > + int l = MIN(PFSHRED_STRLEN_MAX -(int) shredCtx->content, n); > > - if ( (shredCtx->content + n + 1) < shredCtx->content_max) { > + memcpy(&(shredCtx->content_buf[shredCtx->content]), cs, l); > + shredCtx->content += l; > + > + if (l < n) > + stream_printf(GDKout, "!WARNING: shred_characters truncated text node > %d characters(starts with `%.16s...')\n", > + (int) PFSHRED_STRLEN_MAX, cs); > +#else > + shredCtxStruct *shredCtx = (shredCtxStruct*) xmlCtx; > + > + if ( (shredCtx->content + n + 1) > shredCtx->content_max) { > shredCtx->content_max = shredCtx->content + n + 1; > shredCtx->content_buf = GDKrealloc(shredCtx->content_buf, shredCtx->content_max); > if (shredCtx->content_buf == NULL) { > @@ -1117,6 +1128,7 @@ > } > memcpy(&(shredCtx->content_buf[shredCtx->content]), cs, n); > shredCtx->content += n; > +#endif > } > > /** > > > ------------------------------------------------------------------------------ > Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA > -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise > -Strategies to boost innovation and cut costs with open source participation > -Receive a $600 discount off the registration fee with the source code: SFAD > http://p.sf.net/sfu/XcvMzF8H > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime shredder.mx, Feb2009, 1.153.2.3, 1.153.2.4
by Stefan Manegold 20 Feb '09

20 Feb '09

Jan, unfortunately, this checkin breaks more than 30 pathfinder tests: benchmarks/XBench/TC/MD/shred_docs.milS benchmarks/XBench/TC/MD/load_docs.xq benchmarks/XBench/TC/MD/count_nodes.xq benchmarks/XBench/TC/MD/q01.xq ... benchmarks/XBench/TC/MD/q19.xq benchmarks/XMach-1/load_docs.xq benchmarks/XMach-1/count_nodes.xq benchmarks/XMach-1/q1.xq ... benchmarks/XMach-1/q8.xq tests/BugTracker/immune_for_updates.SF-1766259.xq tests/BugTracker/immune_for_updates.SF-1981852.xq Stefan On Fri, Feb 20, 2009 at 09:06:47AM +0000, Jan Flokstra wrote: > Update of /cvsroot/monetdb/pathfinder/runtime > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv26141 > > Modified Files: > Tag: Feb2009 > shredder.mx > Log Message: > - fix [ monetdb-Bugs-2586088 ] XQ: large text nodes > > > > U shredder.mx > Index: shredder.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/shredder.mx,v > retrieving revision 1.153.2.3 > retrieving revision 1.153.2.4 > diff -u -d -r1.153.2.3 -r1.153.2.4 > --- shredder.mx 6 Feb 2009 22:52:30 -0000 1.153.2.3 > +++ shredder.mx 20 Feb 2009 09:06:45 -0000 1.153.2.4 > @@ -362,6 +362,7 @@ > int sp; /* the stackpointer */ > /* buffer administration */ > int content; /* content ptr in char buff */ > + int content_max; /* max size of content buffer */ > char *content_buf; /* character buffer */ > /* */ > size_t fileSize; /* size of the input file */ > @@ -1105,14 +1106,13 @@ > int n) > { > shredCtxStruct *shredCtx = (shredCtxStruct*) xmlCtx; > - int l = MIN(PFSHRED_STRLEN_MAX -(int) shredCtx->content, n); > - > - memcpy(&(shredCtx->content_buf[shredCtx->content]), cs, l); > - shredCtx->content += l; > > - if (l < n) > - stream_printf(GDKout, "!WARNING: shred_characters truncated text node > %d characters(starts with `%.16s...')\n", > - (int) PFSHRED_STRLEN_MAX, cs); > + if ( (shredCtx->content + n + 1) < shredCtx->content_max) { > + shredCtx->content_max = shredCtx->content + n + 1; > + shredCtx->content_buf = GDKrealloc(shredCtx->content_buf, shredCtx->content_max); > + } > + memcpy(&(shredCtx->content_buf[shredCtx->content]), cs, n); > + shredCtx->content += n; > } > > /** > @@ -1831,7 +1831,8 @@ > /* allocate some buffers used during parse */ > shredCtx->val = (char*) GDKmalloc(PFSHRED_BUFLEN+1); > shredCtx->lifo = (node_t *) GDKmalloc((XML_DEPTH_MAX+1) * sizeof(node_t)); > - shredCtx->content_buf = (char*) GDKmalloc(PFSHRED_STRLEN_MAX); > + shredCtx->content_max = PFSHRED_STRLEN_MAX; > + shredCtx->content_buf = (char*) GDKmalloc(shredCtx->content_max); > if (shredCtx->val == NULL || shredCtx->content_buf == NULL || shredCtx->lifo == NULL) { > return shredder_free(shredCtx,-shredCtx->incremental); > } > > > ------------------------------------------------------------------------------ > Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA > -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise > -Strategies to boost innovation and cut costs with open source participation > -Receive a $600 discount off the registration fee with the source code: SFAD > http://p.sf.net/sfu/XcvMzF8H > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/optimizer opt_joinpath.mx, Feb2009, 1.36.2.2, 1.36.2.3
by Sjoerd Mullender 20 Feb '09

20 Feb '09

Sjoerd Mullender wrote: > Update of /cvsroot/monetdb/MonetDB5/src/optimizer > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv2163 > > Modified Files: > Tag: Feb2009 > opt_joinpath.mx > Log Message: > Undo Martin's checkin per his request: > "The joinpath optimizer handles the documented front-end compression scheme > essential to reduce the number of projection joins in the SQL code." > before creating a new candidate release. The code has not been tested > in any form so will have to wait until later. Ok, I overstated thing just a tad: it apparently was tested on one platform, but not on any others. -- Sjoerd Mullender

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/gdk gdk_heap.mx, Feb2009, 1.107, 1.107.2.1
by Stefan Manegold 20 Feb '09

20 Feb '09

On Tue, Feb 10, 2009 at 09:13:21AM +0000, Stefan Manegold wrote: > Update of /cvsroot/monetdb/MonetDB/src/gdk > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv18180/MonetDB/src/gdk > > Modified Files: > Tag: Feb2009 > gdk_heap.mx > Log Message: > > [on behalf of Peter] re-doing > > 2009/02/05 - boncz: MonetDB/src/gdk/gdk_heap.mx,1.108 > - make large allocs fall back again on mmap Peter, (how) can we make the limit (more) dependend on / adaptive to the actual memory size? Shouldn't this have an effect only with blocks > gdk_vm_minsize == 128 GB ? I have sample scenarios (on Linux) where, e.g., queries produce intermediate results of ~ 5GB --- on my 64-bit 8GB machine those queries run in 40 secs before this change (i.e., with h->storage = STORE_MMAP;), but take 200 secs (i.e., 5 times as long) after this change (i.e., with h->newstorage = STORE_MMAP;) ... Jan R. has seen similar effects with XMark on his Mac ... Thanks, Stefan > now only the actually intended changes > and as bug fix, i.e., in the Feb2009 release branch > > > Index: gdk_heap.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_heap.mx,v > retrieving revision 1.107 > retrieving revision 1.107.2.1 > diff -u -d -r1.107 -r1.107.2.1 > --- gdk_heap.mx 22 Jan 2009 14:47:53 -0000 1.107 > +++ gdk_heap.mx 10 Feb 2009 09:13:19 -0000 1.107.2.1 > @@ -115,7 +115,7 @@ > fp = GDKfilelocate(nme, "wb", ext); > if (fp != NULL) { > fclose(fp); > - h->storage = STORE_MMAP; > + h->newstorage = STORE_MMAP; > HEAPload(h, nme, ext, FALSE); > } > GDKfree(of); > > > ------------------------------------------------------------------------------ > Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM) > software. With Adobe AIR, Ajax developers can use existing skills and code to > build responsive, highly engaging applications that combine the power of local > resources and data with the reach of the web. Download the Adobe AIR SDK and > Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 4

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/mal mal_interpreter.mx, , 1.286, 1.287
by Stefan Manegold 16 Feb '09

16 Feb '09

Milena, do you consider this a bug fix or a new feature? if the former, it will not be in the upcoming release, since it has not (yet) been checked in to the Feb2009 release branch... in case you consider an exceptional back-port of this checkin to the Feb2009 release branch, please make sure you od this using the approriate CVS commands to ensure that the identical changes are applied --- see, e.g., page 14 of http://homepages.cwi.nl/~manegold/Presentations/MonetDBdevTTT.pdf for details --- don't hesitate to ask inc ase you need help ... Stefan On Mon, Feb 16, 2009 at 01:56:58PM +0000, Milena Ivanova wrote: > Update of /cvsroot/monetdb/MonetDB5/src/mal > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv13891 > > Modified Files: > mal_interpreter.mx > Log Message: > Call recycler only for instructions with positive value of the recycle field. > > > U mal_interpreter.mx > Index: mal_interpreter.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/mal/mal_interpreter.mx,v > retrieving revision 1.286 > retrieving revision 1.287 > diff -u -d -r1.286 -r1.287 > --- mal_interpreter.mx 15 Feb 2009 23:15:30 -0000 1.286 > +++ mal_interpreter.mx 16 Feb 2009 13:56:55 -0000 1.287 > @@ -1993,14 +1993,14 @@ > The recycler works for any variable and relies on policy functions > registered. > @= MALrecycleStart > - if ( pci->recycle >= 0) > + if ( pci->recycle > 0) > @1->clk = GDKusec(); > if( !RECYCLEentry(cntxt, mb,stk,pci) ) > @= MALrecycleExit > - if ( pci->recycle >= 0) > + if( pci->recycle > 0 ){ > tcs = GDKusec() - @1->clk; > - if( pci->recycle > 0 ) > RECYCLEexit(cntxt, mb,stk,pci,tcs); > + } > @} > @- > @node Garbage Collection, Stack Management, Exception Handling, The MAL Interpreter > > > ------------------------------------------------------------------------------ > Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA > -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise > -Strategies to boost innovation and cut costs with open source participation > -Receive a $600 discount off the registration fee with the source code: SFAD > http://p.sf.net/sfu/XcvMzF8H > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1