Hi Arjen,

Thanks for this input! What you say is reasonable. 

I was not very precise about the situation, because what I really meant is that the server is mostly meant for *several instances of* MonetDB. Then it is a lot harder to make them use memory efficiently - they are not really designed to play nicely to each other.

We limit the memory that each instance can see, via cgroups. This has the advantage that no instance can make the others starve. 

We don't just divide the available memory, we overbook a bit. The cgroups setting is just a cap, not preallocated memory, so the sum of the caps is larger than the available memory and that's ok because they won't all use all their quota at the same time.

Still, it does happen that some intermediates are large, or that they happen to use more memory at the same time. And then swapping happens. This is much harder to predict compared to a 1-instance scenario.

This multi-server with occasional overbooking failure scenario is where I wonder what the effects of those kernel settings could be.

About swappiness: would reducing swappiness help? Probably it would only delay the moment in which swapping happens. But if swapping becomes needed then the only options are either do it or get killed, I guess?

About overcommit_memory, I'm still not sure whether MonetDB could benefit from always allowing it (vm.overcommit_memory=1). 

For example, Redis strongly advises that, to support the copy-on-write mechanism of its background backup.

MonetDB actually uses CoW in many situations (I think about shared heaps). I wonder if this setting would have any effect.

In general, it would be great to have a simple set of recommendations like this one for example: https://redis.io/topics/admin

On Fri, 26 Feb 2021 at 09:16, Arjen de Rijke <arjen.de.rijke@cwi.nl> wrote:

Hi Roberto,

I don't have specific recommendations, but i can share my experience with the administration of the scilens cluster. As far as i can remember, i never received requests to change anything related to the vm.overcommit settings. I did receive questions about the swappiness. I never changed the default that fedora uses, but some people wanted to change the value to check some specific use case, including MonetDB, if i remember correctly. But i don't think that it generally matters a lot, i mean, i never heard about that making a big difference. So unless some vendor has a specific reason to suggest some specific version, i would keep the default.

We also kept the amount of swapspace very low, intentionally. In the situation you describe, when MonetDB is the "only" program running, it can use all of the available memory. If the application needs more memory and starts to swap, the performance drop significantly. And likely never finishes. Because in most cases it is some kind of bug. Having a lot of swap only postpones the inevitable, the system runs out of memory. With a lot of swap, it just takes much longer to crash.

Arjen de Rijke

----- Original Message -----
> From: "Roberto Cornacchia" <roberto.cornacchia@gmail.com>
> To: "Communication channel for MonetDB users" <users-list@monetdb.org>
> Sent: Thursday, February 25, 2021 6:31:18 PM
> Subject: Linux kernel vm settings

> Hi there,
>
> I looked around but couldn't find any recommendation about kernel vm settings in
> Linux for MonetDB.
>
> In particular:
>
> - vm.overcommit_memory:
> 0 (default) : a heuristics decides whether overcommitting is allowed
> 1: no check, overcommit is always allowed
> 2: overcommitting is regulated by vm.overcommit_ratio (default = 50%)
>
> Do I understand correctly that using vm.overcommit_memory=1 will only make the
> OOM kill mserver5 when the total VM available is exhausted?
>
> If that is true, should it be reasonably safe to use on a server that is mainly
> intended for MonetDB, as long as sufficient disk space is available?
>
>
> - vm.swappiness
> Generic recommendations are usually 60 for a desktop and 30 for a server.
> Oracle recommends 10.
> Redis recommends 1.
> Are there studies / recommendations for MonetDB?
>
>
>
> _______________________________________________
> users-list mailing list
> users-list@monetdb.org
> https://www.monetdb.org/mailman/listinfo/users-list
_______________________________________________
users-list mailing list
users-list@monetdb.org
https://www.monetdb.org/mailman/listinfo/users-list

_______________________________________________
users-list mailing list
users-list@monetdb.org
https://www.monetdb.org/mailman/listinfo/users-list