August 2009 - developers-list

[Monetdb-developers] monentdb autocommit
by 颜勋 07 Sep '09

07 Sep '09

Hi all, I use BenchmarkFactory and ODBC to test the monentdb performance. But it seems the default option of monentdb is autocommit, so some test scripts seems not work. How should i change the autocommit option (not only use "set autocommit 0" and just change it in the range of one transaction )? Does anyone know the details(that'are tools, scripts, monetdb enviroment) of TPC-H test in monentdb website? Many thanks~ Regards Yan

3 2

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/mal replicator.mx, , 1.14, 1.15
by Stefan de Konink 29 Aug '09

29 Aug '09

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 ../../../../src/modules/mal/replicator.mx: In function ‘MASTERreplay’: ../../../../src/modules/mal/replicator.mx:551: error: ‘msg’ may be used uninitialized in this function make[8]: *** [lib_replicator_la-replicator.lo] Error 1 I get the following nag from the compiler. Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEAREKAAYFAkqZWRYACgkQYH1+F2Rqwn2GYACcCyJVoQQzXV72lthLqcuzzCGx ef0AniCCaJy8qHd0IZYjeccYOFwvRCgo =xUjB -----END PGP SIGNATURE-----

2 1

[Monetdb-developers] MonetDB/XQuery: reading XML files from TAR archives
by Wouter Alink 27 Aug '09

27 Aug '09

Hello devs, Roberto and I yesterday discussed that it would be useful to be able to load (compressed) XML collections directly into MonetDB/XQuery. The attached diff provides a new feature for loading multiple XML docs directly from tar files. Usage: "mclient -lxq -C <colname>" and pass a tarfile via stdin, see example below. My question: is this useful enough to make it into MonetDB? And if so, is the current syntax appropriate. Comments are appreciated. Greetings, Wouter $ mkdir xmlfiles $ echo "<abc/>" > xmlfiles/aap.xml $ echo "<abc/>" > xmlfiles/beer.xml $ tar cf xmlfiles.tar xmlfiles $ mclient -lxq -C xmlfiles < xmlfiles.tar Copying TAR file into collection: 'xmlfiles' Name: xmlfiles/beer.xml Length: 7 Name: xmlfiles/aap.xml Length: 7 $ echo 'pf:documents("xmlfiles")' | mclient -lxq <document updatable="false" url="xmlfiles/aap.xml" collection="xmlfiles">xmlfiles/aap.xml</document>, <document updatable="false" url="xmlfiles/beer.xml" collection="xmlfiles">xmlfiles/beer.xml</document> $

2 2

[Monetdb-developers] Announcement: New Aug2009 feature release of MonetDB suite
by Stefan Manegold 20 Aug '09

20 Aug '09

Dear all, The MonetDB team at CWI/MonetDB BV are pleased to announce the new "Aug2009" feature release of the MonetDB suite of programs. This feature release is backward compatible with the previous releases (Nov2008 through May2009-SP2). Caveat: When upgrading from a release older than May2009 (i.e., Nov2008 through Feb2009-SP2), you need to move your write-ahead log following the "recommended" instructions for SQL (<http://monetdb.cwi.nl/Development/Releases/May2009/#SQL>) and/or XQuery (<http://monetdb.cwi.nl/Development/Releases/May2009/#XQuery>). More information on the new release is available at <http://monetdb.cwi.nl/Development/Releases/Aug2009/>. The sources and binaries are available at <http://monetdb.cwi.nl/Download/>. Enjoy! The MonetDB team at CWI/MonetDB BV. -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/mal mal_interpreter.mx, Aug2009, 1.313.2.2, 1.313.2.3
by Stefan Manegold 18 Aug '09

18 Aug '09

Martin, just for info: while chasing other property checking errors, I happened to notice that this checkin makes tests MonetDB5/src/mal/Tests/tst911.mal MonetDB5/src/mal/Tests/tst912.mal fail with !MALException:mal.propertyCheck:Object not found Stefan On Tue, Aug 18, 2009 at 06:10:37AM +0000, Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/mal > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv1398 > > Modified Files: > Tag: Aug2009 > mal_interpreter.mx > Log Message: > The accidental loss of a BAT was ignored, while it should lead to an > exception. All other GDKerrors are already converted in a MALexception > with catchKernelException before the interpreter loop terminates. > > > U mal_interpreter.mx > Index: mal_interpreter.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/mal/mal_interpreter.mx,v > retrieving revision 1.313.2.2 > retrieving revision 1.313.2.3 > diff -u -d -r1.313.2.2 -r1.313.2.3 > --- mal_interpreter.mx 17 Aug 2009 13:53:47 -0000 1.313.2.2 > +++ mal_interpreter.mx 18 Aug 2009 06:10:35 -0000 1.313.2.3 > @@ -1615,8 +1615,10 @@ > if( stk->stk[getArg(pci,i)].vtype == TYPE_bat && > stk->stk[getArg(pci,i)].val.bval ){ > b= BATdescriptor(stk->stk[getArg(pci,i)].val.bval); > - if( b == NULL) > + if( b == NULL){ > + ret = createException(MAL, "mal.propertyCheck", RUNTIME_OBJECT_MISSING); > continue; > + } > if( b->batStamp <= stamp){ > if( GDKdebug & 8) > @:batpropcheck(b)@ > @@ -1624,9 +1626,6 @@ > if( GDKdebug & 2) > @:batpropcheck(b)@ > BBPunfix(b->batCacheid); > - if( cntxt->errbuf && cntxt->errbuf[0]) { > - ret = createException(MAL, "mal.propertyCheck", RUNTIME_OBJECT_MISSING); > - } > } > } > #endif > > > ------------------------------------------------------------------------------ > Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day > trial. Simplify your report design, integration and deployment - and focus on > what you do best, core application coding. Discover what's new with > Crystal Reports now. http://p.sf.net/sfu/bobj-july > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

[Monetdb-developers] Module dependencies in Aug2009 release branch/candidate
by Stefan Manegold 17 Aug '09

17 Aug '09

Sjoerd, fellow developers, I just realize the following: (at least) due to my recent changes, (at least) MonetDB4 & MonetDB5 in the Aug2009 release branch (most probably) do no longer compile with an earlier version of MonetDB common (GDK), mainly because I added (for convenience) the assert_shift_width() C macro to gdk.mx/gdk.h . It could well be, that binary build of the latest versions of MonetDB4 & MonetDB5 still work with binary builds of earlier versions of MonetDB common (GDK); at least I'm not aware of any interface changes. Hence, unless we remove the assert_shift_width() C macro, again (and implement its functionlity differently), we'd need to make the upcoming MonetDB-4.32.0 and MonetDB-5.14.0 requiring MonetDB-1.32.0. However, this also changes the dependencies for the binaries. Well, given our regular release cycle, and Windows installers and super source balls that contain "everything", anyway, I don't expect too many users that would like to upgrade only M4 or M5 or sql, while keeping the "old" GDK ... Hence, to be on the save side, I'd propose to update all dependecies in the Aug2009 branch to require part from that very branch (and possibly introduce this as a standard procedure, unless/until we indeed do partial releases). Any thoughs, comments, objections? Thanks! Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

3 2

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/optimizer opt_joinpath.mx, Aug2009, 1.50, 1.50.2.1
by Stefan de Konink 16 Aug '09

16 Aug '09

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Martin Kersten schreef: > For key-valued columns the max join result size is known. > /* foreign key joins are common */ > if( l->ttype== TYPE_oid || r->htype == TYPE_oid || r->htype== TYPE_void) > return MAX(lc,rc); > + if (l->tkey || r->hkey) > + return MAX(lc,rc); Is there a reason not to merge this in the previous if? Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEAREKAAYFAkqIBOQACgkQYH1+F2Rqwn1GvwCfQVokOe6Z1nw9fzn1gALoxKN1 nl8An38ArAJcdAtlQgJWXOdg2XBOMx4V =SZZ9 -----END PGP SIGNATURE-----

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/mal tablet.mx, , 1.226, 1.227
by Sjoerd Mullender 14 Aug '09

14 Aug '09

Is this correct? What about the case where you have a field which ends in \\, such as \\,more data which is two fields separated by a comma. On 2009-08-14 08:17, Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/modules/mal > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv11440 > > Modified Files: > tablet.mx > Log Message: > Only start looking at the escape if you really have to. > > > U tablet.mx > Index: tablet.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/mal/tablet.mx,v > retrieving revision 1.226 > retrieving revision 1.227 > diff -u -d -r1.226 -r1.227 > --- tablet.mx 14 Aug 2009 05:50:11 -0000 1.226 > +++ tablet.mx 14 Aug 2009 06:17:10 -0000 1.227 > @@ -2278,21 +2278,21 @@ > /* we split based on simple character separators for speed */ > if ( rseplen == 1 ){ > for (; *e; e++) > - if ( *e == '\\') > - e++; > - else > - if ( *e == *rsep) > + if ( *e == *rsep) { > + if ( e> s&& *(e-1) == '\\') > + continue; > break; > + } > if (*e == 0) > e = 0; > } else > do { > for ( ; *e ; e++) > - if ( *e == '\\') > - e++; > - else > - if ( *e == *rsep&& strncmp(e,rsep,rseplen) == 0 ) > + if ( *e == *rsep&& strncmp(e,rsep,rseplen) == 0 ) { > + if ( e> s&& *(e-1) == '\\') > + continue; > break; > + } > > if ( *e ) break; > e = 0; > > > ------------------------------------------------------------------------------ > Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day > trial. Simplify your report design, integration and deployment - and focus on > what you do best, core application coding. Discover what's new with > Crystal Reports now. http://p.sf.net/sfu/bobj-july > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Sjoerd Mullender

3 3

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/gdk gdk_heap.mx, Aug2009, 1.125, 1.125.2.1
by Stefan Manegold 13 Aug '09

13 Aug '09

Sjoerd, this checkin seems to fix the following tests, while not harming any others (on my 64-bit Fedora 10 desktop): MonetDB4/src/gdk/malloc_mmap_priv.milS MonetDB4/src/modules/plain/bat.py sql/src/test/BugTracker-2009/union_leak.SF-2540169.SQL.sh Stefan On Thu, Aug 13, 2009 at 02:41:08PM +0000, Sjoerd Mullender wrote: > Update of /cvsroot/monetdb/MonetDB/src/gdk > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv31891 > > Modified Files: > Tag: Aug2009 > gdk_heap.mx > Log Message: > Experimental checkin. > > Memory maps come in two flavors: normal and private > (a.k.a. copy-on-write). The former are more efficient when saving > since less data needs to be copied, but they can be a pain when > dealing with pre-existing heaps, since then a backup needs to be made > before starting to update the data. > > When switching from malloced to memory mapped allocation in a heap, we > now see whether the heap file already existed, and if so, we use a > private memory map (as before), but if the file did not exist, we now > use a normal memory map. We don't need to make a backup first, > because there is no pre-existing data to backup, so this should work. > > The result of this change is that loading data into fresh bats should > be quicker. Since the new heaps are now "normal" memory maps, less > copying takes place when the data is saved. A quick test suggests > that this is indeed the case: the loading part of the 15M Periscope > test took longer than the timeout (1800 seconds) allowed without this > change, but was finished in about 1000 seconds with the change. > > Tonight's testing will hopefully show whether this idea does indeed > work. > > > U gdk_heap.mx > Index: gdk_heap.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_heap.mx,v > retrieving revision 1.125 > retrieving revision 1.125.2.1 > diff -u -d -r1.125 -r1.125.2.1 > --- gdk_heap.mx 25 Jul 2009 21:51:40 -0000 1.125 > +++ gdk_heap.mx 13 Aug 2009 14:41:05 -0000 1.125.2.1 > @@ -206,13 +206,26 @@ > } > /* too big: convert it to a disk-based temporary heap */ > if (can_mmap) { > - FILE *fp = GDKfilelocate(nme, "wb", ext); > + FILE *fp; > char *of = h->filename; > + int existing = 0; > + > + /* if the heap file already exists, we want to > + switch to STORE_PRIV (copy-on-write memory > + mapped files), but if the heap file doesn't > + exist yet, the BAT is new and we can use > + STORE_MMAP */ > + fp = GDKfilelocate(nme, "rb", ext); > + if (fp != NULL) { > + existing = 1; > + fclose(fp); > + } > h->filename = NULL; > + fp = GDKfilelocate(nme, "wb", ext); > if (fp != NULL) { > fclose(fp); > if (h->storage == STORE_MEM) { > - int newmode = (h->newstorage != STORE_MMAP)?h->newstorage:STORE_PRIV; > + int newmode = h->newstorage == STORE_MMAP && existing ? STORE_PRIV : h->newstorage; > /* make sure we realy MMAP */ > if (must_mmap && h->newstorage == STORE_MEM) > newmode = STORE_MMAP; > > > ------------------------------------------------------------------------------ > Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day > trial. Simplify your report design, integration and deployment - and focus on > what you do best, core application coding. Discover what's new with > Crystal Reports now. http://p.sf.net/sfu/bobj-july > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > > > > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-checkins] clients/src/examples/C sample0.c, Aug2009, 1.9, 1.9.2.1 sample1.c, Aug2009, 1.9, 1.9.2.1 sample2.c, Aug2009, 1.9, 1.9.2.1 sample3.c, Aug2009, 1.11, 1.11.2.1 sample4.c, Aug2009, 1.10
by Stefan de Konink 13 Aug '09

13 Aug '09

Could this also be fixed with the same construction? Stefan On Thu, 13 Aug 2009, Sjoerd Mullender wrote: > Update of /cvsroot/monetdb/clients/src/examples/C > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv30784/src/examples/C > > Modified Files: > Tag: Aug2009 > sample0.c sample1.c sample2.c sample3.c sample4.c smack00.c > smack01.c > Log Message: > We don't need clients_config.h for HAVE_LONG_LONG/HAVE___INT64 anymore, but we do need to define LLFMT. > > U sample0.c > Index: sample0.c > =================================================================== > RCS file: /cvsroot/monetdb/clients/src/examples/C/sample0.c,v > retrieving revision 1.9 > retrieving revision 1.9.2.1 > diff -u -d -r1.9 -r1.9.2.1 > --- sample0.c 26 Jul 2009 07:47:10 -0000 1.9 > +++ sample0.c 13 Aug 2009 14:28:00 -0000 1.9.2.1 > @@ -17,7 +17,6 @@ > * All Rights Reserved. > */ > > -#include <clients_config.h> > #include <stdlib.h> > #include <stdio.h> > #include <string.h> > > U smack00.c > Index: smack00.c > =================================================================== > RCS file: /cvsroot/monetdb/clients/src/examples/C/smack00.c,v > retrieving revision 1.12 > retrieving revision 1.12.2.1 > diff -u -d -r1.12 -r1.12.2.1 > --- smack00.c 28 Jul 2009 09:08:16 -0000 1.12 > +++ smack00.c 13 Aug 2009 14:28:00 -0000 1.12.2.1 > @@ -17,7 +17,6 @@ > * All Rights Reserved. > */ > > -#include <clients_config.h> > #include <stdlib.h> > #include <stdio.h> > #include <string.h> > > U smack01.c > Index: smack01.c > =================================================================== > RCS file: /cvsroot/monetdb/clients/src/examples/C/smack01.c,v > retrieving revision 1.11 > retrieving revision 1.11.2.1 > diff -u -d -r1.11 -r1.11.2.1 > --- smack01.c 28 Jul 2009 08:20:06 -0000 1.11 > +++ smack01.c 13 Aug 2009 14:28:00 -0000 1.11.2.1 > @@ -17,7 +17,6 @@ > * All Rights Reserved. > */ > > -#include <clients_config.h> > #include <stdlib.h> > #include <stdio.h> > #include <string.h> > > U sample4.c > Index: sample4.c > =================================================================== > RCS file: /cvsroot/monetdb/clients/src/examples/C/sample4.c,v > retrieving revision 1.10 > retrieving revision 1.10.2.1 > diff -u -d -r1.10 -r1.10.2.1 > --- sample4.c 29 Jul 2009 08:03:37 -0000 1.10 > +++ sample4.c 13 Aug 2009 14:28:00 -0000 1.10.2.1 > @@ -17,13 +17,15 @@ > * All Rights Reserved. > */ > > -#include <clients_config.h> > #include <stdlib.h> > #include <stdio.h> > #include <string.h> > #include <mapilib/Mapi.h> > #ifdef _MSC_VER > #define snprintf _snprintf > +#define LLFMT "%I64d" > +#else > +#define LLFMT "%lld" > #endif > > #define die(dbh,hdl) do { \ > > U sample1.c > Index: sample1.c > =================================================================== > RCS file: /cvsroot/monetdb/clients/src/examples/C/sample1.c,v > retrieving revision 1.9 > retrieving revision 1.9.2.1 > diff -u -d -r1.9 -r1.9.2.1 > --- sample1.c 26 Jul 2009 07:47:10 -0000 1.9 > +++ sample1.c 13 Aug 2009 14:28:00 -0000 1.9.2.1 > @@ -17,7 +17,6 @@ > * All Rights Reserved. > */ > > -#include <clients_config.h> > #include <stdlib.h> > #include <stdio.h> > #include <string.h> > > U sample2.c > Index: sample2.c > =================================================================== > RCS file: /cvsroot/monetdb/clients/src/examples/C/sample2.c,v > retrieving revision 1.9 > retrieving revision 1.9.2.1 > diff -u -d -r1.9 -r1.9.2.1 > --- sample2.c 26 Jul 2009 07:47:10 -0000 1.9 > +++ sample2.c 13 Aug 2009 14:28:00 -0000 1.9.2.1 > @@ -17,7 +17,6 @@ > * All Rights Reserved. > */ > > -#include <clients_config.h> > #include <stdlib.h> > #include <stdio.h> > #include <string.h> > > U sample3.c > Index: sample3.c > =================================================================== > RCS file: /cvsroot/monetdb/clients/src/examples/C/sample3.c,v > retrieving revision 1.11 > retrieving revision 1.11.2.1 > diff -u -d -r1.11 -r1.11.2.1 > --- sample3.c 29 Jul 2009 10:40:17 -0000 1.11 > +++ sample3.c 13 Aug 2009 14:28:00 -0000 1.11.2.1 > @@ -17,13 +17,15 @@ > * All Rights Reserved. > */ > > -#include <clients_config.h> > #include <stdlib.h> > #include <stdio.h> > #include <string.h> > #include <mapilib/Mapi.h> > #ifdef _MSC_VER > #define snprintf _snprintf > +#define LLFMT "%I64d" > +#else > +#define LLFMT "%lld" > #endif > > #define die(dbh,hdl) do { \ > > > ------------------------------------------------------------------------------ > Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day > trial. Simplify your report design, integration and deployment - and focus on > what you do best, core application coding. Discover what's new with > Crystal Reports now. http://p.sf.net/sfu/bobj-july > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins >

2 3