[Monetdb-developers] [Monetdb-checkins] MonetDB/src/gdk gdk_heap.mx, Feb2009, 1.107, 1.107.2.1

Stefan Manegold Stefan.Manegold at cwi.nl
Thu Feb 19 23:01:38 CET 2009


On Thu, Feb 19, 2009 at 10:49:24PM +0100, Stefan Manegold wrote:
> On Tue, Feb 10, 2009 at 09:13:21AM +0000, Stefan Manegold wrote:
> > Update of /cvsroot/monetdb/MonetDB/src/gdk
> > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv18180/MonetDB/src/gdk
> > 
> > Modified Files:
> >       Tag: Feb2009
> > 	gdk_heap.mx 
> > Log Message:
> > 
> > [on behalf of Peter]  re-doing
> > 
> > 2009/02/05 - boncz: MonetDB/src/gdk/gdk_heap.mx,1.108
> > - make large allocs fall back again on mmap
> 
> Peter,
> 
> (how) can we make the limit (more) dependend on / adaptive to the actual
> memory size?
> 
> Shouldn't this have an effect only with blocks > gdk_vm_minsize == 128 GB ?
> 
> I have sample scenarios (on Linux) where, e.g., queries produce intermediate
> results of ~ 5GB --- on my 64-bit 8GB machine those queries run in  40 secs
> before this change (i.e., with h->storage = STORE_MMAP;), but take 200 secs
> (i.e., 5 times as long) after this change (i.e., with h->newstorage =
> STORE_MMAP;) ...

... on a 64GB Linux machine, I even see a 10x degradation: 30s -> 300s ...

> Jan R. has seen similar effects with XMark on his Mac ...
> 
> Thanks,
> 
> Stefan
> 
> > now only the actually intended changes
> > and as bug fix, i.e., in the Feb2009 release branch
> > 
> > 
> > Index: gdk_heap.mx
> > ===================================================================
> > RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_heap.mx,v
> > retrieving revision 1.107
> > retrieving revision 1.107.2.1
> > diff -u -d -r1.107 -r1.107.2.1
> > --- gdk_heap.mx	22 Jan 2009 14:47:53 -0000	1.107
> > +++ gdk_heap.mx	10 Feb 2009 09:13:19 -0000	1.107.2.1
> > @@ -115,7 +115,7 @@
> >  		fp = GDKfilelocate(nme, "wb", ext);
> >  		if (fp != NULL) {
> >  			fclose(fp);
> > -			h->storage = STORE_MMAP;
> > +			h->newstorage = STORE_MMAP;
> >  			HEAPload(h, nme, ext, FALSE);
> >  		}
> >  		GDKfree(of);
> > 
> > 
> > ------------------------------------------------------------------------------
> > Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
> > software. With Adobe AIR, Ajax developers can use existing skills and code to
> > build responsive, highly engaging applications that combine the power of local
> > resources and data with the reach of the web. Download the Adobe AIR SDK and
> > Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
> > _______________________________________________
> > Monetdb-checkins mailing list
> > Monetdb-checkins at lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
> 
> -- 
> | Dr. Stefan Manegold | mailto:Stefan.Manegold at cwi.nl |
> | CWI,  P.O.Box 94079 | http://www.cwi.nl/~manegold/  |
> | 1090 GB Amsterdam   | Tel.: +31 (20) 592-4212       |
> | The Netherlands     | Fax : +31 (20) 592-4312       |

-- 
| Dr. Stefan Manegold | mailto:Stefan.Manegold at cwi.nl |
| CWI,  P.O.Box 94079 | http://www.cwi.nl/~manegold/  |
| 1090 GB Amsterdam   | Tel.: +31 (20) 592-4212       |
| The Netherlands     | Fax : +31 (20) 592-4312       |




More information about the developers-list mailing list