On Thu, 28 Jun 2018 at 12:45 Arjen de Rijke arjen.de.rijke@cwi.nl wrote:
At some point (can't remember when) mserver5 stopped respecting ulimit.
No
matter how I change the limits, it always sees all available physical
ram. Does
any other mechanism exist?
I think this is the correct behaviour of ulimit. The mserver5 process has nothing to do with it, ulimit controls how much resources a shell can use, it does not "fake" the amount of physical memory of the machine. And the behaviour of a process that hits the ulimit setting likely behaves differently than a process on a machine that runs out of real memory. And you also have to take into account that you can have ulimit settings for users, processes and shells.
Yes, that is the correct behaviour. MonetDB used to exploit that information to self-enforce limits on the available resources. Now it doesn't, and it's ok, I'm only poinint out that these is no other mechanism to enforce limits.
I think you are right that the only available solution is vm's. Otherwise monetdb would have to enforce the memory usage per process itself. But i am not sure if this information is available internally. And even if it is, how would you handle for example a situation where one of the dbfarms runs out of memory during a query that is split over multiple dbfarms?
Indeed, I wonder why there is no configuration option to have some degree of control on how aggressively memory should be used. I would expect something like "max 2GB resident memory", and if this means more memory mapping is happening (thus lower performance), then be it. But no control at all with many mserver5 running leaves me a bit frustrated.
Also, do you think there could be recommendations about swap memory (in particular in this shared environment)? I confess I don't have very clear in which occasions swap memory is used. Naively, I always thought that memory mapping (provided you have enough disk space) would make swap memory not necessary to use. But clearly it's not really like that.
Arjen
Does anyone have experience with / tips about multi-dbfarm environments?
Thanks! Roberto
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list