hannes.muehleisen at cwi.nl
Thu Sep 22 10:44:58 CEST 2016
> On 20 Sep 2016, at 23:45, Foteini Alvanaki <F.Alvanaki at cwi.nl> wrote:
> It is interesting that malloc is faster than mmap.
Yeah this is one of the magic MonetDB parameters…
> Could you give more details about the setting of your experiment?
just running select (i*2) from table;
i is a column containing 100M integers. Ran on my laptop (OSX) with 4 threads.
> How many threads, how many concurrent files (BATs), size of files,
> type of data, kind of accesses etc.
> ----- Original Message -----
> From: Roberto Cornacchia <roberto.cornacchia at gmail.com>
> To: Communication channel for developers of the MonetDB suite. <developers-list at monetdb.org>
> Sent: Tue, 20 Sep 2016 18:40:41 +0200 (CEST)
> Subject: Re: GDK_mmap_minsize again
> If I may add, that is indeed the default behaviour of the kernel, which can
> be disabled with
> vm.overcommit_memory = 2
> in /etc/sysctl.conf
> Perhaps MonetDB could check this system setting and decide on which
> strategy to use?
> On 20 September 2016 at 18:24, Sjoerd Mullender <sjoerd at acm.org> wrote:
>> As far as I understand it, malloc on Linux will happily succeed even if
>> there is not enough memory+swap to hold all data. So you can't rely on
>> malloc failures to tell you to switch to mmap.
>> On 09/20/2016 06:19 PM, Hannes Mühleisen wrote:
>>> Hello list,
>>> we were wondering about the purpose of GDK_mmap_minsize when creating
>> transient columns. The attached patch will always *try* to malloc/realloc a
>> transient column but still fall back to memory-mapped files if malloc
>> should fail. This dramatically improves performance. Any good reason why
>> this should not be the default behaviour?
>>> Mark and Hannes
>> Sjoerd Mullender
>> Sjoerd Mullender
>> developers-list mailing list
>> developers-list at monetdb.org
> developers-list mailing list
> developers-list at monetdb.org
More information about the developers-list