developers-list January 2010

developers-list@monetdb.org

12 participants
18 discussions

[Monetdb-developers] encoding of strings returned by Python client

by Mark Bucciarelli

Any chance the Python client could be modified to return strings in Unicode instead of UTF-8? Seems like a better fit for Python; for example, that's how Python's sqlite3 module behaves. An alternative approach is to expose a set_encoding method on the connection that let's you specify the encoding of all text fields returned. If backward compatibility is an issue, the latter idea seems better even though it adds a knob. Any comments? (Is the maintainer of the Python adapter active even on this list? I didn't see many Python-specific posts in the archives.) Thanks, m

12 years, 1 month

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/gdk gdk_storage.mx, Feb2010, 1.149.2.5, 1.149.2.6

by Stefan Manegold

On Sun, Jan 31, 2010 at 04:43:29PM +0000, Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB/src/gdk > In directory sfp-cvsdas-1.v30.ch3.sourceforge.com:/tmp/cvs-serv28035 > > Modified Files: > Tag: Feb2010 > gdk_storage.mx > Log Message: > A pity that the first code was not properly checked, because it did not > even tried to force the preload by memory touching properly. > As per request (though admittedly belated ...): Wouldn't it be sufficient (and faster and less memory consuming) to preload only that part of the heaps that is indeed used (i.e., holds useful/relevant data) instead of preloading the whole (possibly consciously over-allocated) heaps? I.e., replacing ...heap->size by ...heap->free (as it was in the original code, only that the original code incorrecly assumed ...heap->free to be an absolute address instead of a relative byte offset ...) Stefan > > Index: gdk_storage.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_storage.mx,v > retrieving revision 1.149.2.5 > retrieving revision 1.149.2.6 > diff -u -d -r1.149.2.5 -r1.149.2.6 > --- gdk_storage.mx 31 Jan 2010 15:50:19 -0000 1.149.2.5 > +++ gdk_storage.mx 31 Jan 2010 16:43:27 -0000 1.149.2.6 > @@ -703,28 +703,32 @@ > @c > size_t > BATpreload_(BAT *b) { > - size_t dummy = 0, *i; > + size_t dummy = 0, *i, *limit; > size_t step = MT_pagesize()/sizeof(size_t); > size_t pages = (size_t) (0.8 * MT_npages()); > > - if ( b->T->vheap && b->T->vheap->base ){ > - IODEBUG THRprintf(GDKout,"#BATpreload(%s): T->vheap\n", BATgetId(b)); > - for ( i = (size_t *) b->T->vheap->base; i < (size_t *) b->T->vheap->free && pages > 0; i+= step, pages--) > + if ( b->T->vheap && b->T->vheap->base && pages){ > + IODEBUG THRprintf(GDKout,"#BATpreloaded(%s): T->vheap\n", BATgetId(b)); > + limit = (size_t *) (b->T->vheap->base + b->T->vheap->size); > + for ( i = (size_t *) b->T->vheap->base; i < limit && pages > 0; i+= step, pages--) > dummy += *i; > } > - if ( b->H->vheap && b->H->vheap->base ){ > - IODEBUG THRprintf(GDKout,"#BATpreload(%s): H->vheap\n", BATgetId(b)); > - for ( i = (size_t *) b->H->vheap->base; i < (size_t *) b->H->vheap->free && pages > 0 ; i+= step, pages--) > + if ( b->T->heap.base && pages ){ > + IODEBUG THRprintf(GDKout,"#BATpreload(%s): T->heap\n", BATgetId(b)); > + limit = (size_t *) (b->T->heap.base + b->T->heap.size); > + for ( i = (size_t *) b->T->heap.base; i < limit && pages > 0; i+= step, pages--) > dummy += *i; > } > - if ( b->hheap && b->hheap->base ){ > - IODEBUG THRprintf(GDKout,"#BATpreload(%s): hheap\n", BATgetId(b)); > - for ( i = (size_t *) b->hheap->base; i < (size_t *)b->hheap->free && pages > 0 ; i+= step, pages--) > + if ( b->H->heap.base && pages ){ > + IODEBUG THRprintf(GDKout,"#BATpreload(%s): H->heap\n", BATgetId(b)); > + limit = (size_t *) (b->H->heap.base + b->H->heap.size); > + for ( i = (size_t *) b->H->heap.base; i < limit && pages > 0 ; i+= step, pages--) > dummy += *i; > } > - if ( b->theap && b->theap->base ){ > - IODEBUG THRprintf(GDKout,"#BATpreload(%s): theap\n", BATgetId(b)); > - for ( i = (size_t *) b->theap->base; i < (size_t *) b->theap->free && pages > 0; i+= step, pages--) > + if ( b->H->vheap && b->H->vheap->base && pages ){ > + IODEBUG THRprintf(GDKout,"#BATpreload(%s): H->vheap\n", BATgetId(b)); > + limit = (size_t *) (b->H->vheap->base + b->H->vheap->size); > + for ( i = (size_t *) b->H->vheap->base; i < limit && pages > 0 ; i+= step, pages--) > dummy += *i; > } > return dummy; > > > ------------------------------------------------------------------------------ > The Planet: dedicated and managed hosting, cloud storage, colocation > Stay online with enterprise data centers and the best network in the business > Choose flexible plans and management services without long-term contracts > Personal 24x7 support from experience hosting pros just a phone call away. > http://p.sf.net/sfu/theplanet-com > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > > > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |

12 years, 5 months

[Monetdb-developers] How to not print on screen?

by Mo Zhou

Hi, I wonder if there is a way to not print query results on screen? -- Thanks, Mo

12 years, 6 months

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/kernel aggr.mx, , 1.41, 1.42

by Stefan Manegold

On Fri, Jan 29, 2010 at 04:41:00AM +0000, Stefan de Konink wrote: > Update of /cvsroot/monetdb/MonetDB5/src/modules/kernel > In directory sfp-cvsdas-1.v30.ch3.sourceforge.com:/tmp/cvs-serv5117 > > Modified Files: > aggr.mx > Log Message: > Fixes typo's and/or inconsistent throw references to function names. > Please propagate to stable. The standard policy is to check in bug fixes to the latest release branch; propagation will than happen automa[tgn]ically. Back-porting such changes by hand creates quite some overhead. Stefan > > > Index: aggr.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/kernel/aggr.mx,v > retrieving revision 1.41 > retrieving revision 1.42 > diff -u -d -r1.41 -r1.42 > --- aggr.mx 7 Jan 2010 15:24:26 -0000 1.41 > +++ aggr.mx 29 Jan 2010 04:40:58 -0000 1.42 > @@ -276,7 +276,7 @@ > } > BBPreleaseref(b->batCacheid); > BBPreleaseref(e->batCacheid); > - throw(MAL, "aggrX3_sum.@4",GDK_EXCEPTION); > + throw(MAL, "aggrX3_prod.@4",GDK_EXCEPTION); > } > @- > @= AX3sumprod > @@ -548,7 +548,7 @@ > BBPreleaseref(b->batCacheid); > BBPreleaseref(e->batCacheid); > BBPreleaseref(g->batCacheid); > - throw(MAL, "aggrX3_prod3",GDK_EXCEPTION); > + throw(MAL, "aggrX3_avg3",GDK_EXCEPTION); > } > @c > @:AX3arithavg3(bte,loc)@ > @@ -590,7 +590,7 @@ > BBPreleaseref(b->batCacheid); > BBPreleaseref(e->batCacheid); > BBPreleaseref(g->batCacheid); > - throw(MAL, "aggrX3_prod3",GDK_EXCEPTION); > + throw(MAL, "aggrX3_@13",GDK_EXCEPTION); > } > @c > @:AX3extreme3(min,<)@ > @@ -627,7 +627,7 @@ > BBPreleaseref(b->batCacheid); > BBPreleaseref(e->batCacheid); > BBPreleaseref(g->batCacheid); > - throw(MAL, "aggrX3_prod3", GDK_EXCEPTION); > + throw(MAL, "aggrX3_count3", GDK_EXCEPTION); > } > @- > The remainder contains the few MIL proc translations > > > ------------------------------------------------------------------------------ > The Planet: dedicated and managed hosting, cloud storage, colocation > Stay online with enterprise data centers and the best network in the business > Choose flexible plans and management services without long-term contracts > Personal 24x7 support from experience hosting pros just a phone call away. > http://p.sf.net/sfu/theplanet-com > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |

12 years, 6 months

[Monetdb-developers] Announcement: New Nov2009-SP2 bugfix release of MonetDB suite

by Sjoerd Mullender

The MonetDB team at CWI/MonetDB BV is pleased to announce the Nov2009-SP2 bug fix release of the MonetDB suite of programs. More information on this release will be available at <http://monetdb.cwi.nl/Development/Releases/Nov2009/>. Fixes include: - No need to require bash for running the *-config scripts. (SF bug #2914563) - MonetDB5 development package now contains all development include files. (SF RFE #2929558) - Added support for compiling on OpenBSD (contributed, i.e. not tested by us). - General code improvements, partly inspired by running Coverity Scan. - mclient -D will exit with a non-zero exit code if dumping failed. (SF bug #2925674) - The Python interface now uses the same socket options as the C version, giving a boost in performance. (SF bug #2925750) - Fixed a bug in the Python interface to deal with fields with embedded newline characters. (SF bug #2917219) - Added initial support to the Python client interface for specifying the encoding. - Fixed a bug in execute argument parsing in Python interface. - Added missing .sql files to the Windows installer. - In MonetDB/SQL, fixed bug in not(invalidtable). (SF bug #2927174) -- Sjoerd Mullender

12 years, 6 months

[Monetdb-developers] Regarding indexes in MonetDB XQuery

by Mo Zhou

Hi, Does MonetDB XQuery support any type of structural indexes? Or caching results of a frequently queried path (with branches)? -- Thanks, M

12 years, 6 months

[Monetdb-developers] mmap region resizing

by Isidor Zeuner

While figuring out some things related to memory management, I stumbled upon a line in MonetDB/src/gdk/gdk_posix.mx which I don't fully understand: munmap(p + oldsize, oldsize - newsize); ..which is called from MT_vmrealloc when the old size is larger than the new size. This would unmap a region as large as the size difference between both sizes, but after the currently mapped region [p; p + oldsize[ (which is probably not mapped, or is mapped to something unrelated). The attached patch changes this behavior so the resulting mapped region will be [p; p + newsize[, with the now unneeded [p + newsize; p + oldsize[ region being unmapped (which is a behavior I find more intuitive). Please review and use as it fits. Best regards, Isidor Zeuner

12 years, 6 months

[Monetdb-developers] GDKupgradevarheap allocate/copy inconsistency

by Isidor Zeuner

Dear MonetDB developers, please see the newly created tracker item [1]. I haven't seen relevant design documentation which would have allowed me to deduce if BATs from VIEWcreate_ are actually supposed to end up in GDKupgradevarheap, as it happened to me. Still, I could imagine the suggested fix to be a step in the right direction as it helps modularity by making the function working with less preassumptions on the inputs. However, if it fits into the global design, the capacity argument can (should?) be removed from the function signature. Best regards, Isidor Zeuner [1] https://sourceforge.net/tracker/?func=detail&aid=2936616&group_id=56967&ati…

12 years, 6 months

[Monetdb-developers] sql_delta ibase issue

by Isidor Zeuner

Dear MonetDB developers, using MonetDB-SQL Feb2010, I occasionally encounter this error message: bat_storage.mx:1274: tr_update_delta: Assertion `obat->ibase == cbat->ibase' failed It occurs only seldomly, so I'll probably not have an isolated test case all too soon. I wonder if someone with more insight into the SQL engine has some pointers on where to look at. Many thanks, Isidor Zeuner

12 years, 6 months

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/mal groupby.mx, , 1.2, 1.3

by Lefteris

it still calls GROUPmin and not GROUPavg is that the intention? On Sun, Jan 17, 2010 at 8:32 PM, Niels Nes <nielsnes(a)users.sourceforge.net> wrote: > Update of /cvsroot/monetdb/MonetDB5/src/modules/mal > In directory sfp-cvsdas-1.v30.ch3.sourceforge.com:/tmp/cvs-serv30908/src/modules/mal > > Modified Files: > groupby.mx > Log Message: > fix cut/paste bug > > > Index: groupby.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/mal/groupby.mx,v > retrieving revision 1.2 > retrieving revision 1.3 > diff -u -d -r1.2 -r1.3 > --- groupby.mx 7 Jan 2010 15:24:26 -0000 1.2 > +++ groupby.mx 17 Jan 2010 19:32:16 -0000 1.3 > @@ -74,7 +74,7 @@ > > pattern avg(pivot:bat[:oid,:any_1],b:bat[:oid,:any_2]...) :bat[:oid,:any_2] > address GROUPmin > -comment "Derive the grouping of all tuples in the BATs and determine minimum value. > +comment "Derive the grouping of all tuples in the BATs and determine average value. > The group oid is materialized."; > > @- > > > ------------------------------------------------------------------------------ > Throughout its 18-year history, RSA Conference consistently attracts the > world's best and brightest in the field, creating opportunities for Conference > attendees to learn about information security's most important issues through > interactions with peers, luminaries and emerging and established companies. > http://p.sf.net/sfu/rsaconf-dev2dev > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins >

12 years, 6 months

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

developers-list January 2010