GDK_mmap_minsize again

Foteini Alvanaki F.Alvanaki at cwi.nl
Tue Sep 20 23:45:48 CEST 2016


It is interesting that malloc is faster than mmap.
Could you give more details about the setting of your experiment?
How many threads, how many concurrent files (BATs), size of files,
type of data, kind of accesses etc.

----- Original Message -----
From: Roberto Cornacchia <roberto.cornacchia at gmail.com>
To: Communication channel for developers of the MonetDB suite. <developers-list at monetdb.org>
Sent: Tue, 20 Sep 2016 18:40:41 +0200 (CEST)
Subject: Re: GDK_mmap_minsize again

If I may add, that is indeed the default behaviour of the kernel, which can
be disabled with

vm.overcommit_memory = 2
in /etc/sysctl.conf

Perhaps MonetDB could check this system setting and decide on which
strategy to use?

On 20 September 2016 at 18:24, Sjoerd Mullender <sjoerd at acm.org> wrote:

> As far as I understand it, malloc on Linux will happily succeed even if
> there is not enough memory+swap to hold all data.  So you can't rely on
> malloc failures to tell you to switch to mmap.
>
> On 09/20/2016 06:19 PM, Hannes Mühleisen wrote:
> > Hello list,
> >
> > we were wondering about the purpose of GDK_mmap_minsize when creating
> transient columns. The attached patch will always *try* to malloc/realloc a
> transient column but still fall back to memory-mapped files if malloc
> should fail. This dramatically improves performance. Any good reason why
> this should not be the default behaviour?
> >
> > Thanks,
> >
> > Mark and Hannes
> >
> >
> >
>
> --
> Sjoerd Mullender
>
>
>
> --
> Sjoerd Mullender
>
>
> _______________________________________________
> developers-list mailing list
> developers-list at monetdb.org
> https://www.monetdb.org/mailman/listinfo/developers-list
>
>



More information about the developers-list mailing list