January 2007 - developers-list

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/modules/plain/Tests bat.py, NONE, 1.1.2.1 bat2.py.src, NONE, 1.1.2.1
by Ying Zhang 18 Jan '07

18 Jan '07

Hello, The log message gives me a hint I must set gdk_mem_pagebits to run XRPC server, otherwise I will get an error: $ Mserver --dbinit="module(xrpc_server);" # Monet Database Server V4.14.0 # ... !ERROR: Set gdk_mem_pagebits=16; please restart Mserver. But I don't need to set gdk_mem_pagebits to start Mserver as said in the log message: $ Mserver # Monet Database Server V4.14.0 # ... MonetDB> And I also don't need to set gdk_mem_pagebits to be able to run Mtest: MSERVER = "/net/andorea/export/scratch1/zhang/monet/dev/G.64.64.d.debug/bin/Mserver" : Mserver "--config=/net/andorea/export/scratch1/zhang/monet/dev/G.64.64.d.debug/etc/MonetDB.conf" --debug=10 --set "monet_mod_path=/net/andorea/export/scratch1/zhang/monet/dev/G.64.64.d.debug/lib/MonetDB:/net/andorea/export/scratch1/zhang/monet/dev/G.64.64.d.debug/lib/MonetDB/lib:/net/andorea/export/scratch1/zhang/monet/dev/G.64.64.d.debug/lib/MonetDB/bin" --set "gdk_dbfarm=/net/andorea/export/scratch1/zhang/monet/dev/G.64.64.d.debug/var/MonetDB/dbfarm" --set "sql_logdir=/net/andorea/export/scratch1/zhang/monet/dev/G.64.64.d.debug/var/MonetDB/sql_logs" --set "xquery_logdir=/net/andorea/export/scratch1/zhang/monet/dev/G.64.64.d.debug/var/MonetDB/xquery_logs" --set mapi_port=38869 --set monet_prompt= --trace Does anyone have an idea how this thing works? Jennie On Tue, Jan 16, 2007 at 10:18:18AM +0000, Stefan Manegold wrote: > Update of /cvsroot/monetdb/MonetDB/src/modules/plain/Tests > In directory sc8-pr-cvs7.sourceforge.net:/tmp/cvs-serv2519 > > Added Files: > Tag: MonetDB_4-14 > bat.py bat2.py.src > Log Message: > > tests bat.milM and bat2.milM (now) require that Mserver is called with "--set gdk_mem_pagebits=16" > > > --- NEW FILE: bat.py --- > import os > MSERVER = os.environ['MSERVER'] > TSTDB = os.environ['TSTDB'] > TST = os.environ['TST'] > os.system("%s --dbname=%s --set gdk_mem_pagebits=16 < %s.milM" % (MSERVER, TSTDB, TST)) > > --- NEW FILE: bat2.py.src --- > $TSTSRCDIR/bat.py > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys - and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins

3 3

[Monetdb-developers] strange Mtest + Mapi problem
by Jan Flokstra 18 Jan '07

18 Jan '07

Hi All, We are experiencing module problems during XQUERY Mapi execution with Mtest which we do not know how to solve. The problem is the Mserver does not know any command from the pftijah module which is already loaded en executing commands. Executing the same query through the mil interface of Mtest works correct. Execution through the MapiClient also works correct but execution through the Mapi interface with Mtest fails. DETAIL: Our XQUERY query containing pftijah functions is translated by 'milprint_summer'to a MIL script containing a call to run_tijah_query() which is in module(pftijah). This script is executed and the run_tijah_query() function is executed. This function in its turn generates a new MIL script where the first line is 'module(pftijah);'. When we execute this script this script does not kown any command from module(pftijah). It does however know functions from module(pathfinder). And this all happens in the Mtest/Mapi setting. In all other settings it works OK but here it fails! Anybody any ideas?? JanF

3 6

[Monetdb-developers] M4 compilation
by Romulo Goncalves 17 Jan '07

17 Jan '07

I updated all MonetDB background (buildtools, MonetDB, clients) I just checked out M4 and during the compilation I got the error: gcc -m64 -DHAVE_CONFIG_H -I. -I/ufs/goncalve/MonetDB/MonetDB4/src/tools -I../.. -I/ufs/goncalve/MonetDB/MonetDB4/src/tools -I/ufs/goncalve/scratch/MonetDB/MonetDB/include/MonetDB -I/ufs/goncalve/scratch/MonetDB/MonetDB/include/MonetDB/common -I/ufs/goncalve/scratch/MonetDB/MonetDB/include/MonetDB/gdk -I/ufs/goncalve/scratch/MonetDB/clients/include/MonetDB -I/ufs/goncalve/scratch/MonetDB/clients/include/MonetDB/mapilib -I../monet -I/ufs/goncalve/MonetDB/MonetDB4/src/tools/../monet -I../modules/plain -I/ufs/goncalve/MonetDB/MonetDB4/src/tools/../modules/plain -Wall -W -std=c99 -g -Werror-implicit-function-declaration -Werror -Wno-unused-function -Wno-unused-label -D_REENTRANT -c Mserver.c /ufs/goncalve/MonetDB/MonetDB4/src/tools/Mserver.mx: In function ‘Monet_init’: /ufs/goncalve/MonetDB/MonetDB4/src/tools/Mserver.mx:173: error: ‘GDK_mem_pagebits’ undeclared (first use in this function) /ufs/goncalve/MonetDB/MonetDB4/src/tools/Mserver.mx:173: error: (Each undeclared identifier is reported only once /ufs/goncalve/MonetDB/MonetDB4/src/tools/Mserver.mx:173: error: for each function it appears in.) make[5]: *** [Mserver.o] Error 1 make[5]: Leaving directory `/net/gio/export/scratch0/goncalve/MonetDB/MonetDB4/build/src/tools' make[4]: *** [all] Error 2 make[4]: Leaving directory `/net/gio/export/scratch0/goncalve/MonetDB/MonetDB4/build/src/tools' make[3]: *** [all-recursive] Error 1 make[3]: Leaving directory `/net/gio/export/scratch0/goncalve/MonetDB/MonetDB4/build/src' make[2]: *** [all] Error 2 make[2]: Leaving directory `/net/gio/export/scratch0/goncalve/MonetDB/MonetDB4/build/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/net/gio/export/scratch0/goncalve/MonetDB/MonetDB4/build' make: *** [all] Error 2 [goncalve@gio ~]$ In the Path I have buildtools_prefix/bin MonetDB_prefix/bin and clients_prefix/bin Is any trick that I should do to resolve this problem? Note: The prefix of M4 is different from MonetDB. Regards, Romulo

2 3

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/mal mal_client.mx, 1.139, 1.140 mal_session.mx, 1.129, 1.130
by Stefan Manegold 17 Jan '07

17 Jan '07

On Wed, Jan 17, 2007 at 07:13:52AM +0000, Niels Nes wrote: > Update of /cvsroot/monetdb/MonetDB5/src/mal > In directory sc8-pr-cvs7.sourceforge.net:/tmp/cvs-serv9203/src/mal > > Modified Files: > mal_client.mx mal_session.mx > Log Message: > increased number of client slots from 32 to 64 > don't crash on running out ot client slots for info: M4 uses/allows even 512 clients: ./MonetDB4/src/monet/monet_client.mx:#define MAXCLIENTS 512 Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

[Monetdb-developers] M5 conf file....
by Romulo Goncalves 17 Jan '07

17 Jan '07

Dear developers, The M5 is now in source forge with the name MonetDB5. It would be nice to have MonetDB5.conf instead of monet5.conf Regards, ROmulo

4 8

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/modules/pftijah pftijah.mx, 1.85, 1.86
by Stefan Manegold 17 Jan '07

17 Jan '07

On Wed, Jan 17, 2007 at 12:34:18PM +0000, Jan Flokstra wrote: > Update of /cvsroot/monetdb/pathfinder/modules/pftijah > In directory sc8-pr-cvs7.sourceforge.net:/tmp/cvs-serv5537 > > Modified Files: > pftijah.mx > Log Message: > - repair "jan was right" bug. On 64 bit machines BUNtloc() is different from > BUNtail() on void tail'ed BAT's. To avoid the spreading of incomplete/incorrect facts/knowledge: BUNhloc/BUNtloc are significantly different from BUNhead/BUNtail on ALL platforms, i.e., in ALL cases: BUNhloc/BUNtloc are for fixed-sized atoms (e.g., chr, bte, sht, int, wrd, lng, flt, dbl, oid, etc.) that are stored *loc*ally in the BUNheap, and hence alloc direct/*loc*al access. BUNhvar/BUNtvar are for *var*iable-sized atoms (e.g., str, void, etc.) that are stored in the extra head-/tail-heaps, or not stored ("materialized") at all (void). BUNhead/BUNtail are wrappers, that check whether head/tail is fixed- or var-sized and use either of the above respectively. Hence, BUNhead/BUNtail work in all cases, but at the expense of an extra if (branch) that can considereably contribute to overall costs in case it has to be evaluated millions of times in inner loops. Hence, to gain speed, BUNhvar/BUNtvar and in particular BUNhloc/BUNtloc allow to avoid this extra if (branch => costs) in cases where we know that either only var-sized or only fixed-sized atoms (i.e., columns) occur, e.g., by moving this loop-invariant if out of the loop. see also MonetDB/src/gdk/gdk.mx for the definition and description of these (and other) macros. Stefan > - repair no-result bug which haunted the testweb last night. > > Hopefully the pftijah testweb is OK again tonight. > > > > Index: pftijah.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/modules/pftijah/pftijah.mx,v > retrieving revision 1.85 > retrieving revision 1.86 > diff -u -d -r1.85 -r1.86 > --- pftijah.mx 16 Jan 2007 13:37:23 -0000 1.85 > +++ pftijah.mx 17 Jan 2007 12:34:13 -0000 1.86 > @@ -321,7 +321,7 @@ > bat("tj_" + collName + "_param").insert("preExpansion","4"); > bat("tj_" + collName + "_param").insert("status","building"); > bat("tj_" + collName + "_param").insert("_last_tijahPre","1"); > - bat("tj_" + collName + "_param").insert("_last_finalizedPre","1"); > + bat("tj_" + collName + "_param").insert("_last_finalizedPre","0"); > # > commit(); > } > @@ -1365,7 +1365,7 @@ > > # evaluate doc/term (anc/desc) relationship > var t2 := time(); > - var elem_tid := _containing_desc(left.mark(0@0), pre_tid, pre_size); > + var elem_tid := _containing_desc3(left.mark(0@0), pre_tid, pre_size); > var t3 := time(); > pre_tid := nil; > if (elem_tid.count() = 0) {return new(oid,dbl);} > @@ -3187,8 +3187,9 @@ > > > BATloopFast(elem_tid, p, q, xx) { > - oid *h = (oid*) BUNhloc(elem_tid, p); > - oid *t = (oid*) BUNtloc(elem_tid, p); > + oid *h = (oid*) BUNhead(elem_tid, p); > + oid *t = (oid*) BUNtail(elem_tid, p); > + > for(i = 0; i < term_cnt; i++) > if (terms[i] == *t) { > if (0) BUNappend(bats[i], h, 0); > @@ -3208,8 +3209,9 @@ > } > break; > } > - if (i == term_cnt) > - GDKerror("%s: jan was right.\n",name); > + if (i == term_cnt) { > + GDKerror("WARNING:%s: cannot find term " OIDFMT ".\n",name,*t); > + } > } > > t = BUNfirst(tid_cnt); > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys - and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

[Monetdb-developers] Proposed changes to build setup
by Jens Teubner 17 Jan '07

17 Jan '07

Hi all, recently, Jan and I have looked at some aspects of the Pathfinder/MonetDB build process. And we had the impression that there is some inconsistency in the current setup. The makefiles of all MonetDB-related projects currently include $MONETDB_CONFDIR/rules.mk. This file basically contains instructions on how to use the tools from the buildtools package to compile all the projects from CVS sources. To compile a project from distributed tarballs, none of these rules should be necessary. Thus, I think a much better place to host the rules.mk file would be in the buildtools package. Thus I would suggest to do the following changes and ask for your comments whether these changes make sense: -- rules.mk goes to the buildtools package. This is where it belongs to. -- The buildtools package gets its `buildtools-config' bash script in order to allow remaining packages to actually find rules.mk. -- Instead of include $(MONETDB_CONFDIR)/*.mk the bootstrap script inserts if HAVE_BUILDTOOLS include $(BUILDTOOLS_CONFDIR)/rules.mk endif to make the inclusion of the rules dependent on the outcome of the ./configure run. (Otherwise, we would depend on the buildtools in the distribution tarballs, which we don't want.) -- The tests invoked by ./configure finally need to take care of setting HAVE_BUILDTOOLS appropriately (using buildtools-config). If nobody objects, I would do the necessary changes as soon as we have forked off the new stable branch. Jens -- Jens Teubner Technische Universitaet Muenchen, Department of Informatics D-85748 Garching, Germany Tel: +49 89 289-17259 Fax: +49 89 289-17263 Statistics show that most people are in the majority, while a few are in the minority. -- Nitin Borwankar

2 1

[Monetdb-developers] incorrectly marked keyed
by Henning Rode 17 Jan '07

17 Jan '07

I don't really understand the monetdb warning below, which I get when running test cases for pf/tijah. it's actually nice that monetdb performs all these BATpropchecks when running the test cases, but what's actually the problem here? I printed the BAT and its info() now to check myself, but can't find any problem. A void-headed BAT will always be keyed, or not? best -henning # Shredded 1 XML document (dilbert.xml), total time after commit=0.007s !WARNING: BATpropcheck: BAT tj_testcoll1_size1(-79) was incorrectly marked keyed! bat("tj_testcoll1_size1").info().print(); #-------------------------------------------------# # h t # name # str str # type #-------------------------------------------------# [ "version", "25105" ] [ "batId", "tj_testcoll1_size1" ] [ "batCacheid", "79" ] [ "batParentid", "0" ] [ "batSharecnt", "0" ] [ "head", "void" ] [ "tail", "int" ] [ "batPersistence", "persistent" ] [ "batRestricted", "read-only" ] [ "batRefcnt", "1" ] [ "batLRefcnt", "2" ] [ "batDirty", "clean" ] [ "batSet", "0" ] [ "void_tid", "1" ] [ "void_cnt", "1" ] [ "hsorted", "65" ] [ "hident", "h" ] [ "hdense", "1" ] [ "hseqbase", "1@0" ] [ "hkey", "1" ] [ "hloc", "0" ] [ "hvarsized", "1" ] [ "halign", "1013524" ] [ "hnosorted", "0" ] [ "hnosorted_rev", "0" ] [ "hnodense", "0" ] [ "hnokey[0]", "0" ] [ "hnokey[1]", "0" ] [ "tident", "t" ] [ "tdense", "0" ] [ "tseqbase", "0@0" ] [ "tsorted", "0" ] [ "tkey", "0" ] [ "tloc", "0" ] [ "tvarsized", "0" ] [ "talign", "1013523" ] [ "tnosorted", "1" ] [ "tnosorted_rev", "0" ] [ "tnodense", "0" ] [ "tnokey[0]", "3" ] [ "tnokey[1]", "6" ] [ "batInserted", "44" ] [ "batDeleted", "0" ] [ "batFirst", "0" ] [ "top", "44" ] [ "batStamp", "-784" ] [ "lastUsed", "621603" ] [ "curStamp", "1210" ] [ "batCopiedtodisk", "1" ] [ "batDirtydesc", "clean" ] [ "batDirtybuns", "clean" ] [ "batBuns.free", "176" ] [ "batBuns.size", "4096" ] [ "batBuns.maxsize", "4096" ] [ "batBuns.storage", "malloced" ] [ "batBuns.filename", "01/117.buns" ] [ "hheapdirty", "clean" ] [ "theapdirty", "clean" ] bat("tj_testcoll1_size1").print(); #-----------------# # h t # name # void int # type #-----------------# [ 1@0, 43 ] [ 2@0, 11 ] [ 3@0, 1 ] [ 4@0, 0 ] [ 5@0, 8 ] [ 6@0, 3 ] [ 7@0, 0 ] [ 8@0, 0 ] [ 9@0, 0 ] [ 10@0, 3 ] [ 11@0, 0 ] [ 12@0, 0 ] [ 13@0, 0 ] [ 14@0, 12 ] [ 15@0, 1 ] [ 16@0, 0 ] [ 17@0, 9 ] [ 18@0, 1 ] [ 19@0, 0 ] [ 20@0, 6 ] [ 21@0, 0 ] [ 22@0, 0 ] [ 23@0, 0 ] [ 24@0, 0 ] [ 25@0, 0 ] [ 26@0, 0 ] [ 27@0, 17 ] [ 28@0, 1 ] [ 29@0, 0 ] [ 30@0, 6 ] [ 31@0, 3 ] [ 32@0, 0 ] [ 33@0, 0 ] [ 34@0, 0 ] [ 35@0, 1 ] [ 36@0, 0 ] [ 37@0, 7 ] [ 38@0, 1 ] [ 39@0, 0 ] [ 40@0, 4 ] [ 41@0, 0 ] [ 42@0, 0 ] [ 43@0, 0 ] [ 44@0, 0 ]

2 2

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/compiler/algebra/map intro_proxy.c, 1.19, 1.20
by Stefan Manegold 17 Jan '07

17 Jan '07

On Tue, Jan 16, 2007 at 01:23:11PM +0000, Jan Rittinger wrote: > Update of /cvsroot/monetdb/pathfinder/compiler/algebra/map > In directory sc8-pr-cvs7.sourceforge.net:/tmp/cvs-serv16649/algebra/map > > Modified Files: > intro_proxy.c > Log Message: > -- Perhaps this change keeps icc quiet... It does. Thank you very much! Stefan > Index: intro_proxy.c > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/compiler/algebra/map/intro_proxy.c,v > retrieving revision 1.19 > retrieving revision 1.20 > diff -u -d -r1.19 -r1.20 > --- intro_proxy.c 4 Jan 2007 15:26:18 -0000 1.19 > +++ intro_proxy.c 16 Jan 2007 13:23:07 -0000 1.20 > @@ -2193,7 +2193,7 @@ > switch (p->kind) { > case la_attach: > collect_mappings_worker (p->sem.attach.attname, > - PFalg_attlist (), > + PFalg_attlist_ (0, NULL), > req_count, > req_col_names, > new_col_names); -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

[Monetdb-developers] MonetDB is at SourceForge
by Sjoerd Mullender 16 Jan '07

16 Jan '07

The transition of monet5 to SourceForge is complete. To get it do one of (each command is one line): cvs -d :ext:<name>@monetdb.cvs.sourceforge.net:/cvsroot/monetdb co MonetDB5 cvs -d :pserver:anonymous@monetdb.cvs.sourceforge.net:/cvsroot/monetdb co MonetDB5 The former requires that you have developer access to the MonetDB project at SourceForge and you need to fill in your SourceForge account. The latter is an anonymous, read-only checkout. If you have changes in your old CWI-based copy, do the following: cd .../monet5 # (go to your old copy from CWI) cvs update cvs diff -c > $HOME/m5-diffs cd .../MonetDB5 # (go to the freshly checked out copy from SF) patch -p0 < $HOME/m5-diffs After this you can look at what you changed, correct any conflicts, and commit. Enjoy! -- Sjoerd Mullender

1 0