January 2009 - developers-list

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/compiler/mil milgen.brg, , 1.201, 1.202
by Sjoerd Mullender 30 Jan '09

30 Jan '09

Is this a fix that also applies to the Feb2009 branch? If so, please apply it there as well. In your copy of the Feb2009 stable branch do: cvs update -j1.201 -j1.202 milgen.brg Jan Flokstra wrote: > Update of /cvsroot/monetdb/pathfinder/compiler/mil > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv10667/mil > > Modified Files: > milgen.brg > Log Message: > * reduce memory footprint by throwing away global score db explicitly. > > > > U milgen.brg > Index: milgen.brg > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/compiler/mil/milgen.brg,v > retrieving revision 1.201 > retrieving revision 1.202 > diff -u -d -r1.201 -r1.202 > --- milgen.brg 26 Jan 2009 14:50:15 -0000 1.201 > +++ milgen.brg 30 Jan 2009 08:43:40 -0000 1.202 > @@ -3825,8 +3825,12 @@ > error (var (err->name)), > nop ())); > } > - > unpin (err, 1); > +#ifdef HAVE_PFTIJAH > + execute( > + assgn (var (PF_MIL_TIJAH_SCORE_DB), unused() ) > + ); > +#endif > } break; /* fold) */ > > /* Query: serialize (Side, empty_tbl) */ > @@ -3932,6 +3936,11 @@ > nop ())); > > unpin (err, 1); > +#ifdef HAVE_PFTIJAH > + execute( > + assgn (var (PF_MIL_TIJAH_SCORE_DB), unused() ) > + ); > +#endif > } break; /* fold) */ > > /* Rel: lit_tbl */ > > > ------------------------------------------------------------------------------ > This SF.net email is sponsored by: > SourcForge Community > SourceForge wants to tell your story. > http://p.sf.net/sfu/sf-spreadtheword > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins -- Sjoerd Mullender

1 0

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/modules/pftijah/tjc tjc_abssyn.c, , 1.4, 1.5 tjc_abssyn.h, , 1.6, 1.7 tjc_main.c, , 1.10, 1.11 tjc_parse.y, , 1.5, 1.6 tjc_phys_optimize.c, , 1.3, 1.4 tjc_phys_optimize.h, , 1.1, 1.2
by Sjoerd Mullender 30 Jan '09

30 Jan '09

Is this a fix that also applies to the Feb2009 branch? Henning Rode wrote: > Update of /cvsroot/monetdb/pathfinder/modules/pftijah/tjc > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv7540 > > Modified Files: > tjc_abssyn.c tjc_abssyn.h tjc_main.c tjc_parse.y > tjc_phys_optimize.c tjc_phys_optimize.h > Log Message: > forgot to free some allocated memory > > > > U tjc_main.c > Index: tjc_main.c > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/modules/pftijah/tjc/tjc_main.c,v > retrieving revision 1.10 > retrieving revision 1.11 > diff -u -d -r1.10 -r1.11 > --- tjc_main.c 28 Jan 2009 22:31:01 -0000 1.10 > +++ tjc_main.c 29 Jan 2009 22:33:16 -0000 1.11 > @@ -121,20 +121,6 @@ > } > } > > -void free_tree(TJptree_t *ptree) > -{ > - int c; > - TJpnode_t *n; > - for (c = 0; c < ptree->length; c++) { > - n = &ptree->node[c]; > - if (n->kind == p_query) > - TJCfree (n->sem.qnode); > - if (n->kind == p_tag && strcmp (n->sem.str, "*")) > - TJCfree (n->sem.str); > - } > - TJCfree (ptree); > -} > - > /* THE NEW STUFF */ > > tjc_config* tjc_c_GLOBAL; > @@ -334,7 +320,8 @@ > if (DEBUG) stream_printf(GDKout,"#!tjc start of mil:\n%s",milres); > if (DEBUG) stream_printf(GDKout,"#!tjc end of mil\n"); > > - free_tree(ptree); > + free_atree (atree); > + tjcp_freetree (ptree); > } > else { > *errBUFF = GDKstrdup(tjc_c->errBUFF); > > U tjc_phys_optimize.h > Index: tjc_phys_optimize.h > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/modules/pftijah/tjc/tjc_phys_optimize.h,v > retrieving revision 1.1 > retrieving revision 1.2 > diff -u -d -r1.1 -r1.2 > --- tjc_phys_optimize.h 28 Jan 2009 22:31:01 -0000 1.1 > +++ tjc_phys_optimize.h 29 Jan 2009 22:33:16 -0000 1.2 > @@ -86,6 +86,9 @@ > phys_optimize(TJptree_t *ptree, TJpnode_t *proot, BAT* rtagbat); > > extern void > +free_atree(TJatree_t *atree); > + > +extern void > printTJatree(tjc_config *tjc_c, TJatree_t *atree); > > #endif /* PYHS_OPTIMIZE_H */ > > U tjc_phys_optimize.c > Index: tjc_phys_optimize.c > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/modules/pftijah/tjc/tjc_phys_optimize.c,v > retrieving revision 1.3 > retrieving revision 1.4 > diff -u -d -r1.3 -r1.4 > --- tjc_phys_optimize.c 29 Jan 2009 20:30:05 -0000 1.3 > +++ tjc_phys_optimize.c 29 Jan 2009 22:33:16 -0000 1.4 > @@ -123,6 +123,11 @@ > return atree; > } > > +void free_atree (TJatree_t *atree) { > + TJCfree (atree->nodes); > + TJCfree (atree); > +} > + > TJanode_t* pnode2anode (TJatree_t *atree, TJpnode_t *s, TJpnode_t *pnode, short *nt) > { > TJanode_t *anode; > > U tjc_parse.y > Index: tjc_parse.y > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/modules/pftijah/tjc/tjc_parse.y,v > retrieving revision 1.5 > retrieving revision 1.6 > diff -u -d -r1.5 -r1.6 > --- tjc_parse.y 28 Jan 2009 22:31:01 -0000 1.5 > +++ tjc_parse.y 29 Jan 2009 22:33:16 -0000 1.6 > @@ -145,7 +145,7 @@ > { $$ = $1; $$->root = tjcp_wire2 (tjc_tree, p_pred, $1->root, $2); } > | Path PathNoPred PredicateList > { $$ = $1; *($2->fixme) = $1->root; > - $$->root = tjcp_wire2 (tjc_tree, p_pred, $2->root, $3); } > + $$->root = tjcp_wire2 (tjc_tree, p_pred, $2->root, $3); TJCfree($2); } > ; > OptPathNoPred : /* empty */ > { $$ = NULL; } > > U tjc_abssyn.c > Index: tjc_abssyn.c > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/modules/pftijah/tjc/tjc_abssyn.c,v > retrieving revision 1.4 > retrieving revision 1.5 > diff -u -d -r1.4 -r1.5 > --- tjc_abssyn.c 28 Jan 2009 22:31:01 -0000 1.4 > +++ tjc_abssyn.c 29 Jan 2009 22:33:16 -0000 1.5 > @@ -48,7 +48,6 @@ > TJpnode_t *n = NULL; > int c; > > - //n = (TJpnode_t *) TJCmalloc (sizeof (TJpnode_t)); > if (t->length < t->capacity) > n = &(t->node[t->length++]); > else > @@ -170,6 +169,20 @@ > return ptree; > } > > +void tjcp_freetree(TJptree_t *ptree) > +{ > + int c; > + TJpnode_t *n; > + for (c = 0; c < ptree->length; c++) { > + n = &ptree->node[c]; > + if (n->kind == p_query) > + TJCfree (n->sem.qnode); > + if (n->kind == p_tag && strcmp (n->sem.str, "*")) > + TJCfree (n->sem.str); > + } > + TJCfree (ptree); > +} > + > /** > * Allocates enough memory to hold a copy of @a str > * and return a pointer to this copy > > U tjc_abssyn.h > Index: tjc_abssyn.h > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/modules/pftijah/tjc/tjc_abssyn.h,v > retrieving revision 1.6 > retrieving revision 1.7 > diff -u -d -r1.6 -r1.7 > --- tjc_abssyn.h 28 Jan 2009 22:31:01 -0000 1.6 > +++ tjc_abssyn.h 29 Jan 2009 22:33:16 -0000 1.7 > @@ -160,6 +160,9 @@ > extern struct TJptree_t * > tjcp_inittree (void); > > +extern void > +tjcp_freetree (TJptree_t *ptree); > + > extern char * > TJstrndup (const char *str); > > > > ------------------------------------------------------------------------------ > This SF.net email is sponsored by: > SourcForge Community > SourceForge wants to tell your story. > http://p.sf.net/sfu/sf-spreadtheword > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins -- Sjoerd Mullender

1 0

[Monetdb-developers] New stable branch created
by Sjoerd Mullender 29 Jan '09

29 Jan '09

I have created new stable branches for the upcoming release. From now on *all* bug fixes and *only* bug fixes *must* go to this new stable branch. I reserve the right to undo any changes that do not comply with this rule. All packages use the same name for the branch: Feb2009. So in each of your stable checkout directories you can do cvs update -rFeb2009 to update to the new branch. I have created branches for the following packages: MonetDB MonetDB4 MonetDB5 TestTools buildtools clients geom java pathfinder sql template testing. Note that datacell is missing from this list. It is not yet released and will not be released in this round. Also note that two packages now have stable branches that didn't use to have them: java and testing. Testing is a new package (split off from MonetDB) and java is now also incorporated in the release cycle. -- Sjoerd Mullender

1 0

[Monetdb-developers] char *mapi_get_type(MapiHdl hdl, int fnr)
by Stefan de Konink 25 Jan '09

25 Jan '09

What is the reason that $subj is not a const char *? Stefan

2 1

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/server rel_semantic.mx, , 1.35, 1.36 sql_parser.mx, , 1.299, 1.300 sql_psm.mx, , 1.65, 1.66 sql_semantic.mx, , 1.194, 1.195 sql_updates.mx, , 1.161, 1.162
by Sjoerd Mullender 23 Jan '09

23 Jan '09

Niels Nes wrote: > Update of /cvsroot/monetdb/sql/src/server > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv13513/src/server > > Modified Files: > rel_semantic.mx sql_parser.mx sql_psm.mx sql_semantic.mx > sql_updates.mx > Log Message: > added binary import support (using new BATattach) > > > Index: sql_parser.mx > =================================================================== > RCS file: /cvsroot/monetdb/sql/src/server/sql_parser.mx,v > retrieving revision 1.299 > retrieving revision 1.300 > diff -u -d -r1.299 -r1.300 > --- sql_parser.mx 17 Jan 2009 18:17:01 -0000 1.299 > +++ sql_parser.mx 22 Jan 2009 21:58:06 -0000 1.300 > @@ -2400,6 +2401,12 @@ > append_list(l, $2); > append_string(l, $8); > $$ = _symbol_create_list( SQL_COPYFROM, l ); } > +/* binary copy from */ > + | COPY INTO qname FROM '(' string_commalist ')' > + { dlist *l = L(); > + append_list(l, $3); > + append_list(l, $6); > + $$ = _symbol_create_list( SQL_BINCOPYFROM, l ); } > | COPY select_no_parens_orderby INTO string opt_seps opt_null_string > { dlist *l = L(); > append_symbol(l, $2); This appears to break the syntax copy into a from '/tmp/i' using delimiters '\t'; (!syntax error, unexpected STRING, expecting '(' in: "explain copy into a from '/tmp/i'"). It can be salvaged by adding opt_nr like the two COPY statements before this, but then you'd have to check "by hand" that opt_nr returns NULL. -- Sjoerd Mullender

1 0

[Monetdb-developers] Announcement: New bug fix release of MonetDB suite
by Sjoerd Mullender 23 Jan '09

23 Jan '09

The MonetDB team at CWI is pleased to announce the Nov2008-SP2 bug fix release of the MonetDB suite of programs. This is a bug fix release of the Nov2008 release. More information on this release can be found at <http://monetdb.cwi.nl/projects/monetdb/Development/Releases/Nov2008/#Nov200…>. The sources and binaries are available at <http://monetdb.cwi.nl/projects/monetdb/Download/>. -- Sjoerd Mullender

1 0

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime serialize.mx, , 1.119, 1.120 xrpc_server.mx, , 1.85, 1.86
by Stefan Manegold 22 Jan '09

22 Jan '09

On Thu, Jan 22, 2009 at 03:06:44PM +0100, Ying Zhang wrote: > On Thu, Jan 22, 2009 at 12:30:21PM +0000, Ying Zhang wrote: > > Update of /cvsroot/monetdb/pathfinder/runtime > > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv16608 > > > > Modified Files: > > serialize.mx xrpc_server.mx > > Log Message: > > Added a new env. var. to make the root directory, from which the XRPC HTTP > > server serves files, configurable. > > > > This was (mainly) needed to get the xrpc admin tests run with RunMtest, since RunMtest doesn't have monet installed yet (i.e., there is no $prefix/share/ MonetDB/xrpc/ yet), so we need a way to tell the xrpc server to serve file from > > a different directory, in this case, from $srcdir/runtime/xrpc. > > Sorry, I forgot to mention that this checkin requires an update and a > rebuild of MonetDB4, because changes in MonetDB4/conf/MonetDB.conf.in > are needed. question is why are they needed? or differently, why is the default (only) set in MonetDB.conf, rather then where it is (first) used in Pathfinder/XRPC? I would have expected that the newly added code in Pathfinder/XRPC, just checks, whether the new environment variable is set; if so, it uses that value; otherwise, it uses a default (defined there). In case your code does already exactly this, the changes in MonetDB.conf, should not be required for the new PF/XRPC code to work, rather, they mainly document the new variable and its default value. Stefan > Jennie > > > ------------------------------------------------------------------------------ > This SF.net email is sponsored by: > SourcForge Community > SourceForge wants to tell your story. > http://p.sf.net/sfu/sf-spreadtheword > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime serialize.mx, , 1.119, 1.120 xrpc_server.mx, , 1.85, 1.86
by Stefan Manegold 22 Jan '09

22 Jan '09

On Thu, Jan 22, 2009 at 12:30:21PM +0000, Ying Zhang wrote: > Update of /cvsroot/monetdb/pathfinder/runtime > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv16608 > > Modified Files: > serialize.mx xrpc_server.mx > Log Message: > Added a new env. var. to make the root directory, from which the XRPC HTTP > server serves files, configurable. > > This was (mainly) needed to get the xrpc admin tests run with RunMtest, since RunMtest doesn't have monet installed yet (i.e., there is no $prefix/share/ MonetDB/xrpc/ yet), so we need a way to tell the xrpc server to serve file from > a different directory, in this case, from $srcdir/runtime/xrpc. > [...] > > U xrpc_server.mx > Index: xrpc_server.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/xrpc_server.mx,v > retrieving revision 1.85 > retrieving revision 1.86 > diff -u -d -r1.85 -r1.86 > --- xrpc_server.mx 8 Jan 2009 16:54:18 -0000 1.85 > +++ xrpc_server.mx 22 Jan 2009 12:30:19 -0000 1.86 > @@ -89,6 +89,10 @@ > xrpc_trusted.append(prefix); > } > > +proc set_xrpc_docroot(str new_docroot) : void { > + > +} > + Jennie, what is the purpose of the above *empty* MIL proc? Stefan > proc get_xrpc_open() : bit { > if (monet_environment.exist("xrpc_open")){ > return bit(monet_environment.find("xrpc_open")); [...] -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 1

[Monetdb-developers] Results of FastBit Experiment
by Ross Bates 21 Jan '09

21 Jan '09

A question for any of the core team who were involved.... what resulted when FastBit was plugged into the backend of MonetDB? -Ross

1 0

[Monetdb-developers] mclient running out of memory, crashing mserver5
by sylver_b 21 Jan '09

21 Jan '09

Hi, I've finally found a better way to bulk load records via perl & odbc - a lot more stable. However, I've noticed many times mclient crashing mserver5 when loading records via the interactive interface. For instance, I tried to load about 130 inserts from a sql file using this command : mclient -lsql -d mydb < public_sysuser.sql After few seconds I started to see : #BBPTRIM: memtarget=17978753 vmtarget=0 #BBPTRIM_EXIT: memsize=24100864,vmsize=273741960 [ 1 ] [ 1 ] [ 1 ] [ 1 ] [ 1 ] [ 1 ] [ 1 ] [ 1 ] [ 1 ] #BBPTRIM_ENTER: memsize=275281456,vmsize=275166768 #BBPTRIM: memtarget=19273201 vmtarget=0 #BBPTRIM_EXIT: memsize=24100864,vmsize=275158816 [ 1 ] [ 1 ] [ 1 ] [ 1 ] [ 1 ] [ 1 ] [ 1 ] #BBPTRIM_ENTER: memsize=276161752,vmsize=276047064 #BBPTRIM: memtarget=20153497 vmtarget=0 #BBPTRIM_EXIT: memsize=24100864,vmsize=276056368 [ 1 ] #BBPTRIM_ENTER: memsize=276322608,vmsize=276207920 #BBPTRIM: memtarget=20314353 vmtarget=0 #BBPTRIM_EXIT: memsize=24100864,vmsize=276198064 Top is showing : top - 23:26:14 up 249 days, 9:57, 5 users, load average: 0.00, 0.20, 0.45 Tasks: 96 total, 1 running, 95 sleeping, 0 stopped, 0 zombie Cpu(s): 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 4025600k total, 2433324k used, 1592276k free, 191788k buffers Swap: 9502408k total, 92k used, 9502316k free, 1993392k cached I don't think i'm running out of memory. I had to stop and restart mserver5, flush the table, then re-run the command - all the records were inserted. Also I noticed this type of messages on the mserver5 console when piping records through the mclient (over 1million records) . It seems that the mclient is quiet instable (cf my post about "COPY, terminating connection" few months ago). So, is there a way to increase the memory limit or make sure mserver5 won't crash when running bash inserts via a cron job ? Please advise, Thank you SB

3 9