On Thu, Feb 19, 2009 at 10:49:24PM +0100, Stefan Manegold wrote:
On Tue, Feb 10, 2009 at 09:13:21AM +0000, Stefan Manegold wrote:
Update of /cvsroot/monetdb/MonetDB/src/gdk In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv18180/MonetDB/src/gdk
Modified Files: Tag: Feb2009 gdk_heap.mx Log Message:
[on behalf of Peter] re-doing
2009/02/05 - boncz: MonetDB/src/gdk/gdk_heap.mx,1.108
- make large allocs fall back again on mmap
Peter,
(how) can we make the limit (more) dependend on / adaptive to the actual memory size?
Shouldn't this have an effect only with blocks > gdk_vm_minsize == 128 GB ?
I have sample scenarios (on Linux) where, e.g., queries produce intermediate results of ~ 5GB --- on my 64-bit 8GB machine those queries run in 40 secs before this change (i.e., with h->storage = STORE_MMAP;), but take 200 secs (i.e., 5 times as long) after this change (i.e., with h->newstorage = STORE_MMAP;) ...
... on a 64GB Linux machine, I even see a 10x degradation: 30s -> 300s ...
Jan R. has seen similar effects with XMark on his Mac ...
Thanks,
Stefan
now only the actually intended changes and as bug fix, i.e., in the Feb2009 release branch
Index: gdk_heap.mx
RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_heap.mx,v retrieving revision 1.107 retrieving revision 1.107.2.1 diff -u -d -r1.107 -r1.107.2.1 --- gdk_heap.mx 22 Jan 2009 14:47:53 -0000 1.107 +++ gdk_heap.mx 10 Feb 2009 09:13:19 -0000 1.107.2.1 @@ -115,7 +115,7 @@ fp = GDKfilelocate(nme, "wb", ext); if (fp != NULL) { fclose(fp);
h->storage = STORE_MMAP;
h->newstorage = STORE_MMAP; HEAPload(h, nme, ext, FALSE);
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM) software. With Adobe AIR, Ajax developers can use existing skills and code to build responsive, highly engaging applications that combine the power of local resources and data with the reach of the web. Download the Adobe AIR SDK and Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com _______________________________________________ Monetdb-checkins mailing list Monetdb-checkins@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
-- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |