[Monetdb-developers] [Monetdb-checkins] MonetDB/src/gdk gdk_utils.mx, MonetDB_1-22, 1.206, 1.206.2.1

Stefan Manegold Stefan.Manegold at cwi.nl
Sun Feb 10 13:33:04 CET 2008


Peter: Thanks for disabling mem_bigsize / vmalloc().

All:
(a) 
Unfortunately, this does not (yet) solve 
[ 1872685 ] PF(tijah): HEAPsetmmap: Assertion `p' failed.
http://sourceforge.net/tracker/index.php?func=detail&aid=1872685&group_id=56967&atid=482468

But this appears to be more vm_minsize related than mem_bigsize related.


(b)
We should check all occurances of mem_bigsize and remove / "hide" those that
have become irrelevant with the deactivation of mem_bigsize / vmalloc():
========
$ cvsfiles * | egrep -v ' |/Tests/' | xargs grep -n --color 'mem_bigsize'
--------
MonetDB4/conf/MonetDB.conf.in:29:# gdk_mem_bigsize & gdk_vm_minsize will be set/limited to 
MonetDB4/conf/MonetDB.conf.in:32:# memory chunks of size >= gdk_mem_bigsize (in bytes) will be mmaped anonymously
MonetDB4/conf/MonetDB.conf.in:33:#gdk_mem_bigsize=262144
MonetDB5/conf/monetdb5.conf.in:33:# gdk_mem_bigsize & gdk_vm_minsize will be set/limited to
MonetDB5/conf/monetdb5.conf.in:36:# memory chunks of size >= gdk_mem_bigsize (in bytes) will be mmaped anonymously
MonetDB5/conf/monetdb5.conf.in:37:#gdk_mem_bigsize=262144
MonetDB/src/common/monet_options.mx:410:        set[i].name = strdup("gdk_mem_bigsize");
MonetDB/src/common/monet_options.py.in:39:    # gdk_mem_bigsize & gdk_vm_minsize will be set/limited to
MonetDB/src/common/monet_options.py.in:42:    gdk_mem_bigsize = '256K'
MonetDB/src/gdk/gdk_heap.mx:121:        if (h->size > GDK_mem_bigsize) {
MonetDB/src/gdk/gdk_heap.mx:189:                int can_mmap = (h->filename && size >= GDK_mem_bigsize);
MonetDB/src/gdk/gdk.mx:2210:gdk_export size_t GDK_mem_bigsize;  /* size after which we use VM rather than heap */
MonetDB/src/gdk/gdk_utils.mx:396:size_t GDK_mem_bigsize = 1 << 20;
MonetDB/src/gdk/gdk_utils.mx:964:larger than GDK_mem_bigsize) to anonymous virtual memory. The seamless
MonetDB/src/gdk/gdk_utils.mx:1054:      if (size > GDK_mem_bigsize) {
MonetDB/src/gdk/gdk_utils.mx:1191:      if (size <= GDK_mem_bigsize) {
MonetDB/src/gdk/gdk_utils.mx:1529:      if ((p = GDKgetenv("gdk_mem_bigsize"))) {
MonetDB/src/gdk/gdk_utils.mx:1531:              lng max_mem_bigsize = GDK_mem_maxsize/16;
MonetDB/src/gdk/gdk_utils.mx:1534:              GDK_mem_bigsize = (size_t) MIN(max_mem_bigsize, strtol(p, NULL, 10));
monetweb/Docs/XQuery/MonetDB.conf.texi:115:@item @code{gdk_mem_bigsize}: minimum size for memory-mapped columns, e.g. @code{262144}.
MonetDB4/src/modules/plain/sys.mx:96:.COMMAND mem_bigsize() : lng = get_mem_bigsize;
MonetDB4/src/modules/plain/sys.mx:98:.COMMAND mem_bigsize(lng) : void = set_mem_bigsize;
MonetDB4/src/modules/plain/sys.mx:1430:get_mem_bigsize(lng *num)
MonetDB4/src/modules/plain/sys.mx:1432: *num = GDK_mem_bigsize;
MonetDB4/src/modules/plain/sys.mx:1437:set_mem_bigsize(lng *num)
MonetDB4/src/modules/plain/sys.mx:1439: @:num2sze(mem_bigsize)@
MonetDB4/src/modules/plain/sys.mx:1440: GDK_mem_bigsize = MAX(32768, sze);
MonetDB4/src/modules/plain/sys.mx:1462: if (sze < GDK_mem_bigsize)
MonetDB4/src/modules/plain/sys.mx:1463:         set_mem_bigsize(num);
MonetDB4/src/modules/plain/sys.mx:1464: GDK_mem_maxsize = MAX(GDK_mem_bigsize, sze);
MonetDB4/src/modules/plain/sys.mx:1486: if (sze < GDK_mem_bigsize)
MonetDB4/src/modules/plain/sys.mx:1487:         set_mem_bigsize(num);
MonetDB4/src/modules/plain/sys.mx:1488: GDK_vm_minsize = MAX(GDK_mem_bigsize, sze);
MonetDB5/src/modules/kernel/status.mx:405:get_mem_bigsize(lng *num)
MonetDB5/src/modules/kernel/status.mx:407:      *num = GDK_mem_bigsize;
MonetDB5/src/modules/kernel/status.mx:412:set_mem_bigsize(lng *num)
MonetDB5/src/modules/kernel/status.mx:414:      @:num2sze(mem_bigsize)@
MonetDB5/src/modules/kernel/status.mx:415:      GDK_mem_bigsize = MAX(32768, sze);
MonetDB5/src/modules/kernel/status.mx:439:      if (sze < GDK_mem_bigsize)
MonetDB5/src/modules/kernel/status.mx:440:              set_mem_bigsize(num);
MonetDB5/src/modules/kernel/status.mx:441:      GDK_mem_maxsize = MAX(GDK_mem_bigsize, sze);
MonetDB5/src/modules/kernel/status.mx:466:      if (sze < GDK_mem_bigsize)
MonetDB5/src/modules/kernel/status.mx:467:              set_mem_bigsize(num);
MonetDB5/src/modules/kernel/status.mx:468:      GDK_vm_minsize = MAX(GDK_mem_bigsize, sze);
monetweb/MonetDB/QuickTour/HelloWorld/content.shtml:42:[ "gdk_mem_bigsize",      "262144"
========

more tomorrow during MADAM ...

Stefan


On Sun, Feb 10, 2008 at 10:37:54AM +0100, Peter Boncz wrote:
> no consequences.
> 
> 
> 
> 
> Peter,
> 
> What about the consequences in:
> build/MonetDB/src/gdk/gdk_heap.c
> build/MonetDB5/src/modules/kernel/status.c
> build/MonetDB/src/gdk/gdk.h
> 
> Peter Boncz wrote:
> > Update of /cvsroot/monetdb/MonetDB/src/gdk
> > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv29651
> >
> > Modified Files:
> >       Tag: MonetDB_1-22
> > 	gdk_utils.mx
> > Log Message:
> > disable use of vmalloc()
> >
> >
> >
> > Index: gdk_utils.mx
> > ===================================================================
> > RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_utils.mx,v
> > retrieving revision 1.206
> > retrieving revision 1.206.2.1
> > diff -u -d -r1.206 -r1.206.2.1
> > --- gdk_utils.mx	14 Jan 2008 10:48:58 -0000	1.206
> > +++ gdk_utils.mx	9 Feb 2008 15:38:11 -0000	1.206.2.1
> > @@ -1050,6 +1050,7 @@
> >  #endif
> >  	}
> >  	size = (size + 7) & ~7;	/* round up to a multiple of eight */
> > +#if 0
> >  	if (size > GDK_mem_bigsize) {
> >  		size_t newsize = size + sizeof(size_t) + sizeof(size_t);
> >  		size_t newmax = MAX(*maxsize, newsize);
> > @@ -1064,6 +1065,7 @@
> >  		*maxsize = newmax - (sizeof(size_t) + sizeof(size_t));
> >  		return (void *) s;
> >  	}
> > +#endif
> >  	CHKMEM(size, 0);
> >  	GDKmalloc_prefixsize(s, size);
> >  	if (s == NULL) {
> >
> >
> > -------------------------------------------------------------------------
> > This SF.net email is sponsored by: Microsoft
> > Defy all challenges. Microsoft(R) Visual Studio 2008.
> > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> > _______________________________________________
> > Monetdb-checkins mailing list
> > Monetdb-checkins at lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
> >
> 
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> Monetdb-developers mailing list
> Monetdb-developers at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/monetdb-developers
> 
> 
> 
> 
> 

-- 
| Dr. Stefan Manegold | mailto:Stefan.Manegold at cwi.nl |
| CWI,  P.O.Box 94079 | http://www.cwi.nl/~manegold/  |
| 1090 GB Amsterdam   | Tel.: +31 (20) 592-4212       |
| The Netherlands     | Fax : +31 (20) 592-4312       |




More information about the developers-list mailing list