Hi Jennie,

Apologies, I have MonetDB-11.25.23 installed. I guess this version has the memory leaks issues fixed. Let me know if there is any other version of SP available.

Meanwhile I will send across the details.

Regards, Sreejith

-----Original Message----- From: users-list [mailto:users-list-bounces+sreejith.sharma=harman.com@monetdb.org] On Behalf Of Ying Zhang Sent: Thursday, August 10, 2017 8:36 PM To: Communication channel for MonetDB users Subject: [EXTERNAL] Re: Memory bloat on concurrency - Any fix to release memory post query run

Hai Sreejith,

There is no way to manually force MonetDB to release memory. However, what you described is not supposed to happen. Not very long ago (but should be before the version you use), we have put quite some efforts in release unused memory earlier (i.e. free a variable as soon as it’s not needed, instead of waiting until the end of query execution to release all variables at once) so as to reduce MonetDB’s virtual memory usage.

Can you please upgrade to the latest release Jul2017-SP1 (11.27.5)? The version you’re using is a bit old. In the bug-fix releases following that, several memory leaks were fixed [1].

If your problem persists, can you please give us the necessary data sample, schema, queries and script to reproduce your problem?

Regards, Jennie

[1] https://clicktime.symantec.com/a/1/2SmZnQfkzCIOWj3BYqEvQrPIz-CbpXjoikixwTcM6...

On 10 Aug 2017, at 11:26, Sharma, Sreejith Sreejith.Sharma@harman.com wrote:

Hi,

I have been using monetdb (11.25.3) in CentOS for one of application. I have been working on a PSR testing for this application and have noticed on high concurrency and on same pattern of query running again and again causing memory to bloat up and doesn’t release once the query run completes. I understand that the data is lifted into memory and the result set is also cached, however I assume this should be release once the system is idle which I don’t see it as of now.

Is there any way or parameter to release the unused memory blocks? I have asked this before, but couldn't get to the root to fix this.

Regards, Sreejith _______________________________________________ users-list mailing list users-list@monetdb.org https://clicktime.symantec.com/a/1/4arWuVyzE5LkZgTnAIJx3HZzMUb40yXaiaLm4ghdQ...

_______________________________________________ users-list mailing list users-list@monetdb.org https://clicktime.symantec.com/a/1/4arWuVyzE5LkZgTnAIJx3HZzMUb40yXaiaLm4ghdQ...

On 10 Aug 2017, at 17:27, Anton Kravchenko kravchenko.anton86@gmail.com wrote:

Hi Jennie,

...

we have put quite some efforts in release unused memory earlier (i.e. free a variable as soon as it’s not needed, instead of waiting until the end of query execution to release all variables at once)

Have it been tested against a query that is stopped by "call sys.stop(qtag)" and would be a memory properly released for this case scenario?

Hai Anton,

These two things are orthogonal to each other. The earlier-release of memory happens *during* the query execution, so this will be done no matter how a query is stopped (i.e. by a sys.stop() or because its execution has finished).

After the end of a query execution, either because it has finished or because sys.stop() was called on this query, MonetDB should free up whatever memory used for this query. If that’s not the case, there’s probably a bug we should fix.

However, please be aware MonetDB cannot stop a query execution when it’s in the middel of processing a MAL statement. A sys.stop() will only stop MonetDB from executing the next MAL statement. So, if MonetDB is executing a long running MAL statement, it can take some time before sys.stop() is effective.

Best, Jennie

Regards, Anton

On Thu, Aug 10, 2017 at 8:05 AM, Ying Zhang Y.Zhang@cwi.nl wrote: Hai Sreejith,

There is no way to manually force MonetDB to release memory. However, what you described is not supposed to happen. Not very long ago (but should be before the version you use), we have put quite some efforts in release unused memory earlier (i.e. free a variable as soon as it’s not needed, instead of waiting until the end of query execution to release all variables at once) so as to reduce MonetDB’s virtual memory usage.

Can you please upgrade to the latest release Jul2017-SP1 (11.27.5)? The version you’re using is a bit old. In the bug-fix releases following that, several memory leaks were fixed [1].

If your problem persists, can you please give us the necessary data sample, schema, queries and script to reproduce your problem?

Regards, Jennie

[1] https://www.monetdb.org/OldReleaseNotes/Dec2016

...

On 10 Aug 2017, at 11:26, Sharma, Sreejith Sreejith.Sharma@harman.com wrote:

Hi,

I have been using monetdb (11.25.3) in CentOS for one of application. I have been working on a PSR testing for this application and have noticed on high concurrency and on same pattern of query running again and again causing memory to bloat up and doesn’t release once the query run completes. I understand that the data is lifted into memory and the result set is also cached, however I assume this should be release once the system is idle which I don’t see it as of now.

Is there any way or parameter to release the unused memory blocks? I have asked this before, but couldn't get to the root to fix this.

Regards, Sreejith _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

---

users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

---

users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list