Memory management, 8 years later

Roberto Cornacchia roberto.cornacchia at gmail.com
Thu Jun 28 11:39:09 CEST 2018


Hi all,

8 yeas after this interesting thread about using ulimit to limit the data
segment size (
https://www.monetdb.org/pipermail/users-list/2010-March/004007.html), a few
things have changed and I find myself a bit lost again about memory
management.

At some point (can't remember when) mserver5 stopped respecting ulimit. No
matter how I change the limits, it always sees all available physical ram.
Does any other mechanism exist?

The issue I'm facing all the time, and for which I find very little
documentation, is:
- multiple dbfarms, possibly tens of mserver5 instances, running on the
same (virtual) machine, plus other applications
- Not necessarily, but most times each dmfarm is served by a Docker
container
- CPU and memory resources need to be share fairly (ideally, controlling
priorities)
- No mserver5 should ever be killed because of memory issues

Now, especially when using Docker, there are means to control the maximum
memory size allowed to each process. However, the resolution in case of
overusage is to kill the process. That is of course not a solution.

What I'm looking for is: can I set a cap to resources *at process level*,
in a way that mserver5 will just *see* those, make the best out of them,
and never try to use more? (yes, one way would be a VM, but far from ideal
for me)

Does anyone have experience with / tips about multi-dbfarm environments?

Thanks!
Roberto
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.monetdb.org/pipermail/users-list/attachments/20180628/65434f24/attachment.html>


More information about the users-list mailing list