[MonetDB-users] Memory Leak in mserver5.exe
Hi,
I have a medium sized dataset of about 50 MB text data. When I repeatedly call
SELECT * FROM entry WHERE LOWER(note) LIKE '%hui%'
I get at some time not my result, but the following error in DbVisualizer:
protocol violation, unexpected line: VirtualAlloc(1a0a0000,29806592,MEM_COMMIT,PAGE_READWRITE): failed
Repeatedly calling the query from the command line like this
mclient.exe -lsql -s"SELECT * FROM entry WHERE LOWER(note) LIKE '%hui%'" > result.txt
produces the attached memory usage graph. After 11 times calling the query the memory is full, and some kind of garbage collection seems to happen. However, after only 6 additional times the memory is again full and the garbage collection is quite less effective than before. Subsequent calls reach the memory limit every time.
Markus
P.S.: I have disabled the virtual memory on my Windows XP because when the OS starts paging, it becomes unbearably slow.
The behavior was observed using MonetDB5-SQL-Installer-i386-20071120.msi running on Windows XP SP 2.
On Nov 24, 2007 12:56 AM, Markus Gritsch m.gritsch@gmail.com wrote:
Hi,
I have a medium sized dataset of about 50 MB text data. When I repeatedly call
SELECT * FROM entry WHERE LOWER(note) LIKE '%hui%'
I get at some time not my result, but the following error in DbVisualizer:
protocol violation, unexpected line: VirtualAlloc(1a0a0000,29806592,MEM_COMMIT,PAGE_READWRITE): failed
Repeatedly calling the query from the command line like this
mclient.exe -lsql -s"SELECT * FROM entry WHERE LOWER(note) LIKE '%hui%'" > result.txt
produces the attached memory usage graph. After 11 times calling the query the memory is full, and some kind of garbage collection seems to happen. However, after only 6 additional times the memory is again full and the garbage collection is quite less effective than before. Subsequent calls reach the memory limit every time.
Markus
P.S.: I have disabled the virtual memory on my Windows XP because when the OS starts paging, it becomes unbearably slow.
Markus Gritsch wrote:
The behavior was observed using MonetDB5-SQL-Installer-i386-20071120.msi running on Windows XP SP 2.
Thanks. It seems I'm already on the track to a solution.
On Nov 24, 2007 12:56 AM, Markus Gritsch m.gritsch@gmail.com wrote:
Hi,
I have a medium sized dataset of about 50 MB text data. When I repeatedly call
SELECT * FROM entry WHERE LOWER(note) LIKE '%hui%'
I get at some time not my result, but the following error in DbVisualizer:
protocol violation, unexpected line: VirtualAlloc(1a0a0000,29806592,MEM_COMMIT,PAGE_READWRITE): failed
Repeatedly calling the query from the command line like this
mclient.exe -lsql -s"SELECT * FROM entry WHERE LOWER(note) LIKE '%hui%'" > result.txt
produces the attached memory usage graph. After 11 times calling the query the memory is full, and some kind of garbage collection seems to happen. However, after only 6 additional times the memory is again full and the garbage collection is quite less effective than before. Subsequent calls reach the memory limit every time.
Markus
P.S.: I have disabled the virtual memory on my Windows XP because when the OS starts paging, it becomes unbearably slow.
This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
A memory leak bug has been found in the string routines lower/upper/rtrim/ltrim. Unfortunately, for you, we can not yet produce nightly installers for windows. So this patch is included in the next (bug-fix) release.
Martin Kersten wrote:
Markus Gritsch wrote:
The behavior was observed using MonetDB5-SQL-Installer-i386-20071120.msi running on Windows XP SP 2.
Thanks. It seems I'm already on the track to a solution.
On Nov 24, 2007 12:56 AM, Markus Gritsch m.gritsch@gmail.com wrote:
Hi,
I have a medium sized dataset of about 50 MB text data. When I repeatedly call
SELECT * FROM entry WHERE LOWER(note) LIKE '%hui%'
I get at some time not my result, but the following error in DbVisualizer:
protocol violation, unexpected line: VirtualAlloc(1a0a0000,29806592,MEM_COMMIT,PAGE_READWRITE): failed
Repeatedly calling the query from the command line like this
mclient.exe -lsql -s"SELECT * FROM entry WHERE LOWER(note) LIKE '%hui%'" > result.txt
produces the attached memory usage graph. After 11 times calling the query the memory is full, and some kind of garbage collection seems to happen. However, after only 6 additional times the memory is again full and the garbage collection is quite less effective than before. Subsequent calls reach the memory limit every time.
Markus
P.S.: I have disabled the virtual memory on my Windows XP because when the OS starts paging, it becomes unbearably slow.
This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
On Nov 24, 2007 11:59 AM, Martin Kersten Martin.Kersten@cwi.nl wrote:
A memory leak bug has been found in the string routines lower/upper/rtrim/ltrim. Unfortunately, for you, we can not yet produce nightly installers for windows. So this patch is included in the next (bug-fix) release.
That's ok, I can wait a bit if it does not take month :) BTW, thank you all for being so quick with fixes and responses. Keep up the good work!
Markus
Markus Gritsch wrote:
On Nov 24, 2007 11:59 AM, Martin Kersten Martin.Kersten@cwi.nl wrote:
A memory leak bug has been found in the string routines lower/upper/rtrim/ltrim. Unfortunately, for you, we can not yet produce nightly installers for windows. So this patch is included in the next (bug-fix) release.
a 'hack' around it is to extend your table with a lower case version of the text yuo wanna search. Also avoids lowercase conversion on each call as slightly more space.
That's ok, I can wait a bit if it does not take month :) BTW, thank you all for being so quick with fixes and responses. Keep up the good work!
Markus
This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
participants (2)
-
Markus Gritsch -
Martin Kersten