Can MonetDB be slowed down by a single query?

Vijay Krishna vijayakrishna55 at gmail.com
Thu Apr 14 11:14:12 CEST 2016


Hi Martin,

Thanks for the reply.

Is there any workaround to restrict the memory used by a single query or
client connection - like some configuration variables?

I see a lot of worker threads executing a single query in parallel. I
studied that the number of worker threads can be controlled by the
gdk_threads variable. But is there anyway to restrict the maximum memory
that can be used by any particular worker thread?

Or can this be controlled at the OS level?
We thought of restricting the memory used by the entire mserver with
ulimit, but that still doesn't solve the problem of the possibility of a
single query causing a trouble. Is there any other way that this can be
controlled as the OS level?

@Martin and Others,
If both are NO, then what are the safety measures to be followed in
production to avoid such problems?
(Or, I may sound little dumb - is this not a production problem at all?)
Has anyone faced any such issues?

Thanks in advance.




Thanks & Regards,

Vijayakrishna.P.
Mobile : (+91) 9500402305.

On Thu, Apr 14, 2016 at 1:20 PM, Vijay Krishna <vijayakrishna55 at gmail.com>
wrote:

> Hi Martin,
>
> Thanks for the reply.
>
> Is there any workaround to restrict the memory used by a single query or
> client connection - like some configuration variables?
>
> I see a lot of worker threads executing a single query in parallel (please
> refer screenshot attached). I studied that the number of worker threads can
> be controlled by the gdk_threads variable. But is there anyway to restrict
> the maximum memory that can be used by any particular worker thread?
>
> Or can this be controlled at the OS level?
> We thought of restricting the memory used by the entire mserver with
> ulimit, but that still doesn't solve the problem of the possibility of a
> single query causing a trouble. Is there any other way that this can be
> controlled as the OS level?
>
> @Martin and Others,
> If both are NO, then what are the safety measures to be followed in
> production to avoid such problems?
> (Or, I may sound little dumb - is this not a production problem at all?)
> Has anyone faced any such issues?
>
> Thanks in advance.
>
>
>
>
> Thanks & Regards,
>
> Vijayakrishna.P.
> Mobile : (+91) 9500402305.
>
> On Thu, Apr 14, 2016 at 12:15 PM, Martin Kersten <martin.kersten at cwi.nl>
> wrote:
>
>> On 14/04/16 08:30, Vijay Krishna wrote:
>>
>>> Hi,
>>>
>>> As per MonetDB's architecture, the memory allocated to a single query
>>> (thread) is left with the OS and is not controlled by the mserver.
>>>
>>> If so, I am wondering how MonetDB would react to the following scenario.
>>>
>>> I receive a client connection with a single costly query whose execution
>>> would require all of the RAM's memory.
>>>
>> Hi
>>
>> Both queries will run concurrently and fight for the total memory.
>> However, after some time, the oldest query is put on lower priority
>>
>>
>>> Now, when I receive a second connection that runs a similar costly
>>> operation, will MonetDB start processing this second query by using the
>>> available meagre memory (RAM leftovers or from disk swap) or will MonetDB
>>> wait till the first query releases some of
>>> its memory?
>>>
>>>
>>>
>>> Thanks & Regards,
>>>
>>> Vijayakrishna.P.
>>> Mobile : (+91) 9500402305.
>>>
>>>
>>> _______________________________________________
>>> users-list mailing list
>>> users-list at monetdb.org
>>> https://www.monetdb.org/mailman/listinfo/users-list
>>>
>>>
>> _______________________________________________
>> users-list mailing list
>> users-list at monetdb.org
>> https://www.monetdb.org/mailman/listinfo/users-list
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.monetdb.org/pipermail/users-list/attachments/20160414/72d96b05/attachment.html>


More information about the users-list mailing list