September 2009 - developers-list

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/backends/monet5/merovingian ChangeLog.Aug2009, Aug2009, NONE, 1.1.2.1 ChangeLog, Aug2009, 1.18.2.2, NONE
by Sjoerd Mullender 30 Sep '09

30 Sep '09

Fabian wrote: > Update of /cvsroot/monetdb/sql/src/backends/monet5/merovingian > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv13466 > > Added Files: > Tag: Aug2009 > ChangeLog.Aug2009 > Removed Files: > Tag: Aug2009 > ChangeLog > Log Message: > switch to branch specific ChangeLog (please don't remove ChangeLog in HEAD) I took care of the not removing part. -- Sjoerd Mullender

1 0

[Monetdb-developers] Important Announcement: New release branch
by Sjoerd Mullender 29 Sep '09

29 Sep '09

I have created a new branch which is to be used for the upcoming Nov2009 release. This branch is called "Nov2009". There are now two "stable" branches, the Aug2009 and Nov2009 branch. The Aug2009 branch will be used for the Aug2009-SP2 release which is scheduled for October. The Nov2009 branch will be used for the Nov2009 feature release in November (and the Nov2009-SP1 and Nov2009-SP2 releases in December and January). The reason for making the Nov2009 branch now, and not waiting until after the Aug2009-SP2 release is that this way there is more time to stabilize the code for the release. There are a number of new features that are planned for the November release. Some of these features are not completed yet, and all of these features have to be properly tested and fixed if they're broken. The not-yet-completed features *must* be completed before the end of October, or else they *will* be removed from the release. New features for the November 2009 release are: - new SQL import from CSV files - new code for large joins - monetdb/merovingian distribution - database replication - native PHP client - others? Features that will definitely not be part of the November 2009 release include: - Stratos's slicing and histogram stuff - more? These will be removed from the Nov2009 branch but of course kept in the development trunk. All developers *must* now decide where each change they want to make to the code belongs. If a bug occurs in the Aug2009 branch, the bug *must* be fixed in that branch. If a bug does not occur in the Aug2009 branch, but it does occur in the Nov2009 branch, it *must* be fixed in the Nov2009 branch (this includes fixes to complete the above-mentioned features). All other changes *must* go to the development trunk. Propagation will be done from the Aug2009 branch to the Nov2009 branch, and from the Nov2009 branch to the development trunk. -- Sjoerd Mullender

3 2

[Monetdb-developers] Announcement: New Aug2009-SP1 bugfix release of MonetDB suite
by Sjoerd Mullender 28 Sep '09

28 Sep '09

The MonetDB team at CWI/MonetDB BV is pleased to announce the Aug2009-SP1 bug fix release of the MonetDB suite of programs. This is the first bug fix release for the Aug2009 release. More information on this release can be found at <http://monetdb.cwi.nl/projects/monetdb//Development/Releases/Aug2009/#Aug20…>. The sources and binaries are available at <http://monetdb.cwi.nl/projects/monetdb/Download/>. -- Sjoerd Mullender

1 0

[Monetdb-developers] Remote Merovingian control
by Fabian Groffen 25 Sep '09

25 Sep '09

This message only applies to the current development branch of MonetDB, and is informational only, mainly addressed to others working with the current development branch. monetdb[1] goes abroad After quite some hacking to move code back and forth between components, we now got the first version of merovingian[2] and monetdb[1] that can do remote control. I will give some small examples here so everyone that is interested can get up to speed fast, and/or get an idea on the possibilities. Basically, all functionality that used to be implemented in monetdb, is now implemented in merovingian. That means, for instance creation of a database is performed by merovingian. A positive side-effect of this is that the discovery aspect of databases is now real-time, e.g.: % monetdb release dumptest taken database out of maintenance mode: dumptest 2009-09-25 11:54:10 MSG control[8590]: (local): released database 'dumptest' 2009-09-25 11:54:10 MSG discovery[8590]: new database mapi:monetdb://pegasus.ins.cwi.nl:50000/dumptest (ttl=660s) We can see that when database is released again using monetdb release, it is immediately announced in the network, making it available for use. Locking a database has the oposite effect, of course. Next, the "(local)" string indicates that this control command was received over the local UNIX socket, as was the case in previous releases of merovingian. In addition to this local access, monetdb now also can connect over a TCP socket, which needs not to be remote, e.g.: % monetdb -hjarl.ins.cwi.nl -Pxxxxxx status name state uptime health last crash testdbatjenniesmachine stopped 100%, 4m - 2009-09-25 11:56:19 MSG control[11449]: pegasus.ins.cwi.nl:49652: served status list Here we can see that the merovingian at jarl reports it served a status list to pegasus, which is the host I performed the monetdb status at. Note the -Pxxxxxxxx in there, which is the passphrase of the remote merovingian. This passphrase is by default a random string, but can be set to something else by editing the .merovingian_pass file in the dbfarm and restarting merovingian. The way to find other merovingians in the vincinity is by using monetdb discover. Its output now includes how to reach each merovingian, if remote control is enabled for them. Remote control over TCP can be disabled in monetdb5.conf by unsetting mero_controlport. Control entries can be recognised in discover's output by the trailing asterisk, which one can also glob for: % monetdb discover "*\*" location pegasus.ins.cwi.nl:50001* volund.ins.cwi.nl:50001* jarl.ins.cwi.nl:64001* This command just returned all merovingians that have remote control enabled and reports their hostname and port to connect to. The remote control protocol is very simple, and I'm planning to make at least a Java library available to perform these commands, such that monetdb[1] no longer needs to be the only application to perform commands with merovingian. [1] http://homepages.cwi.nl/~fabian/MonetDB/Man%20Pages.html/monetdb.html [2] http://homepages.cwi.nl/~fabian/MonetDB/Man%20Pages.html/merovingian.html

1 0

[Monetdb-developers] Embedded MonetDb under Windows
by Andzej Tatul 22 Sep '09

22 Sep '09

Hello I have installed MonedDB Sql (32 bit) under Windows (xp), but cannot run it as embedded server. In MonetDB documentation it is mentioned that embedded monetDb engine is available. How can I get embedded server installation for windows? Best regards Andzej _________________________________________________________________ Daugiau nei pranešimai – išbandykite likusias „Windows Live™“ dalis. http://www.microsoft.com/windows/windowslive/

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/mal/crackers crackers.mx, , 1.16, 1.17 crackers_sideways.mx, , 1.21, 1.22
by Stefan Manegold 16 Sep '09

16 Sep '09

Stratos (and others ;-)), please find below some comments, hints and questions. Stefan On Wed, Sep 16, 2009 at 09:53:56PM +0000, stratos wrote: > Update of /cvsroot/monetdb/MonetDB5/src/modules/mal/crackers > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv11656/src/modules/mal/crackers > > Modified Files: > crackers.mx crackers_sideways.mx > Log Message: > position-based tuple reconstruction to use in sideways plans with crack joins > > > U crackers.mx > Index: crackers.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/mal/crackers/crackers.mx,v > retrieving revision 1.16 > retrieving revision 1.17 > diff -u -d -r1.16 -r1.17 > --- crackers.mx 30 Apr 2009 18:43:40 -0000 1.16 > +++ crackers.mx 16 Sep 2009 21:53:53 -0000 1.17 > @@ -504,6 +504,10 @@ > address CRKdProject_@1_@2 > comment "Sync the cracking pair and project the tail. Use for disjunctive queries that require a larger bit vector"; > > +command positionproject( b:bat[:oid,:@1],bp:bat[:oid,:@2],pivot:bat[:oid,:oid]):bat[:oid,:@2] > +address CRKdPositionProject_@1_@2 > +comment "Sync the cracking pair and project the tail. Use position based pivot"; > + > command projectH( b:bat[:oid,:@1],bp:bat[:oid,:@2],l:@1,h:@1,li:bit,hi:bit):bat[:oid,:@1] > address CRKProjectH_@1_@2 > comment "Sync the cracking pair and project the head"; > > U crackers_sideways.mx > Index: crackers_sideways.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/mal/crackers/crackers_sideways.mx,v > retrieving revision 1.21 > retrieving revision 1.22 > diff -u -d -r1.21 -r1.22 > --- crackers_sideways.mx 30 Apr 2009 18:43:40 -0000 1.21 > +++ crackers_sideways.mx 16 Sep 2009 21:53:53 -0000 1.22 > @@ -153,6 +153,7 @@ > crackers_export str CRKMarkedProject_@1_@2(int *res, int *bid, int *pbid, int *pivot, @1 *low, @1 *hgh, bit *inclusiveLow, bit *inclusiveHgh); > crackers_export str CRKPositionProject_@1_@2(int *res, int *bid, int *pbid, int *PositionPivot, @1 *low, @1 *hgh, bit *inclusiveLow, bit *inclusiveHgh); > crackers_export str CRKdProject_@1_@2(int *res, int *bid, int *pbid, int *pivot); > +crackers_export str CRKdPositionProject_@1_@2(int *res, int *bid, int *pbid, int *pivot); > @ > > @{ > @@ -1025,6 +1026,63 @@ > > return MAL_SUCCEED; > } > + > +str > +CRKdPositionProject_@1_@2(int *res, int *bid, int *pbid,int *pivot){ > + BAT *b, *p, *r, *c; > + @2 *pairt, *rest; > + oid *pivott; > + oid size; > + int m; > + oid j; > + > + /* If the necessary map does not exits, then create it */ > + m = existsCrackerIndexSideways(*bid, *pbid); > + if (m==-1){ > + @:CreateCrackerMap(@1,@2)@ > + BBPkeepref(b->batCacheid); > + BBPkeepref(c->batCacheid); > + } > + > + /* Align the map based on the cracker tape of this set */ > + Sync_@1_@2(m,existsCrackerIndexSideways(*bid, -1)); > + > + /* Get the map */ > + if ((b = BATdescriptor(CrackerIndex[m].cbid)) == NULL) > + throw(MAL, "crackers.CRKProject", "Cannot access descriptor"); > + > + if ((p = BATdescriptor(*pivot)) == NULL) > + throw(MAL, "crackers.CRKProject", "Cannot access descriptor"); > + > + size = BATcount(p); > + r = BATnew(TYPE_void,b->ttype,size); This might fail. Error handling? > + > + pairt = (@2 *)Tloc(b, BUNfirst(b)); > + pivott = (oid*)Tloc(p, BUNfirst(p)); > + rest = (@2 *)Tloc(r, BUNfirst(r)); > + > + for (j = 0; j<size; j++){ > + *rest = *(pairt + *pivott); > + rest ++; > + pivott ++; What about replacing the above three lines by rest[j] = pairt[pivott[j]]; ? > + } > + > + BATmark(BATmirror(p),0); ^^^^^^^^^^^^^^^^^^!^^^^^ What's is the purpose/intention of this statement? It IMHO represents a side-effect, since it marks one of the input operands ... Stefan > + > + BATsetcount(r, size); > + > + r->hsorted = FALSE; > + r->tsorted = FALSE; > + r->hdense = FALSE; > + r->tdense = FALSE; > + > + *res = r->batCacheid; > + BBPkeepref(r->batCacheid); > + BBPunfix(p->batCacheid); > + BBPunfix(b->batCacheid); > + > + return MAL_SUCCEED; > +} > @ > > @- Local support functions and macros > > > ------------------------------------------------------------------------------ > Come build with us! The BlackBerry® Developer Conference in SF, CA > is the only developer event you need to attend this year. Jumpstart your > developing skills, take BlackBerry mobile applications to market and stay > ahead of the curve. Join us from November 9-12, 2009. Register now! > http://p.sf.net/sfu/devconf > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

[Monetdb-developers] monentdb autocommit
by 颜勋 07 Sep '09

07 Sep '09

Hi all, I use BenchmarkFactory and ODBC to test the monentdb performance. But it seems the default option of monentdb is autocommit, so some test scripts seems not work. How should i change the autocommit option (not only use "set autocommit 0" and just change it in the range of one transaction )? Does anyone know the details(that'are tools, scripts, monetdb enviroment) of TPC-H test in monentdb website? Many thanks~ Regards Yan

3 2

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/mal/Tests pithole.mal, Aug2009, NONE, 1.1.2.1 All, Aug2009, 1.113, 1.113.2.1
by Stefan Manegold 06 Sep '09

06 Sep '09

Martin, this checkin seems to contain only the new test. Did you by accident forget to checkin the related code changes? Stefan On Sun, Sep 06, 2009 at 07:40:28AM +0000, Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/mal/Tests > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv29612 > > Modified Files: > Tag: Aug2009 > All > Added Files: > Tag: Aug2009 > pithole.mal > Log Message: > Defense against recursion segfaults. > > > --- NEW FILE: pithole.mal --- > #challenge the runtime stack protection > user.main(); > > U All > Index: All > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/mal/Tests/All,v > retrieving revision 1.113 > retrieving revision 1.113.2.1 > diff -u -d -r1.113 -r1.113.2.1 > --- All 28 Jun 2009 20:53:50 -0000 1.113 > +++ All 6 Sep 2009 07:40:23 -0000 1.113.2.1 > @@ -250,3 +250,4 @@ > flowchk > > dynamicload > +pithole.mal > > > ------------------------------------------------------------------------------ > Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day > trial. Simplify your report design, integration and deployment - and focus on > what you do best, core application coding. Discover what's new with > Crystal Reports now. http://p.sf.net/sfu/bobj-july > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

[Monetdb-developers] Think someone forgot something again...
by Stefan de Konink 04 Sep '09

04 Sep '09

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 !SyntaxException:slave.close[0]:RETURN missing Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEAREKAAYFAkqgR5MACgkQYH1+F2Rqwn2RSQCeKz0nIOwSoT51GTAA3w6nWHz1 WKgAn3uV7MmDc2vwQz0YALzwDf3AOYly =HqhO -----END PGP SIGNATURE-----

2 2

[Monetdb-developers] New login intentional?
by Stefan de Konink 04 Sep '09

04 Sep '09

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 It seems to be that there is a new mechanism that basically doesn't allow mclient to connect directly anymore. Is this behavior intentional? The workaround seems to be -umonetdb with monetdb as password. Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEAREKAAYFAkqgSCkACgkQYH1+F2Rqwn3bGgCeIIj438dKmOp1lasT7sGbgdgE Ow0An1BBzIVLAr7cuLRhsOcexCP287GY =qRmk -----END PGP SIGNATURE-----

2 1