Possible to reset or flush out RAM usage in Windows without closing and re-opening mserver.exe?
Hi, I'm running a two-day-long program on Windows x64 with MonetDB.R that breaks somewhere into the second day due to what I believe is overloaded RAM? I was under the impression that this should not really happen with MonetDB, since it's a database that doesn't load everything into RAM.. Here's what's happening:
A big, complicated series of importations, merges, and recodes all work for the first two days on about 250,000,000 records of data.. but all the while my Windows Task Manager will slowly show the physical memory usage meter ticking up in the "Performance" tab. mserver.exe does NOT appear to be using RAM in the "Memory" column of the "Processes" tab, so it looks like there's a ghost using RAM.
Then, after two days, the script will break (mserver output shown below) - returning a GDK error on a seemingly-benign line of code, just a simple recode:
Error in .local(conn, statement, ..., async = async) : UPDATE ep10 SET rx_cso_cov = CVRD_D_PLAN_PD_AMT WHERE DRUG_CVRG_STUS_CD IN ( 'C','S','O' ) failed! Server says: !GDKerror:!ERROR: GDKmallocmax: failed for 589484032 bytes
After that, if I close mserver.exe and open it up again, it takes about twenty minutes to get the ">" mserver prompt. But it does open up eventually.. and at that point, if I re-connect to the server with R and run the exact same line, it works without error.
So it appears I just need to flush out that ghost/mystery RAM usage somehow? Does this error make sense?
There is no way this is due to lack of hard disk space - this server's C drive has 15GB free and the attached drive actually hosting all of this data has 235GB free. I don't want to rule it out, but I also doubt this problem is related to MonetDB.R, since the scenario I described does not require that I restart my R console, just mserver.exe.
I can work around the problem by manually killing and re-starting mserver.exe at a few logical breakpoints in my program, but I didn't think this was supposed to happen in the first place?
Thanks!!! Anthony
# MonetDB 5 server v11.15.7 "Feb2013-SP2" # Serving database 'medicare_sample', using 4 threads # Compiled for x86_64-pc-winnt/64bit with 64bit OIDs dynamically linked # Found 20.000 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2013 MonetDB B.V., all rights reserved # Visit http://www.monetdb.org/ for further information # Listening for connection requests on mapi:monetdb://127.0.0.1:49800/ # MonetDB/JAQL module loaded # MonetDB/SQL module loaded
# SQL catalog created, loading sql scripts once
# loading sql script: 09_like.sql # loading sql script: 10_math.sql # loading sql script: 11_times.sql # loading sql script: 12_url.sql # loading sql script: 13_date.sql # loading sql script: 14_inet.sql # loading sql script: 15_history.sql # loading sql script: 16_tracelog.sql # loading sql script: 17_compress.sql # loading sql script: 18_dictionary.sql # loading sql script: 19_cluster.sql # SQL catalog created, loading sql scripts once # loading sql script: 09_like.sql # loading sql script: 20_vacuum.sql # loading sql script: 10_math.sql # loading sql script: 11_times.sql # loading sql script: 12_url.sql # loading sql script: 21_dependency_functions.sql # loading sql script: 13_date.sql # loading sql script: 14_inet.sql # loading sql script: 15_history.sql # loading sql script: 16_tracelog.sql # loading sql script: 17_compress.sql # loading sql script: 18_dictionary.sql # loading sql script: 19_cluster.sql # loading sql script: 20_vacuum.sql # loading sql script: 21_dependency_functions.sql # loading sql script: 22_clients.sql # loading sql script: 23_skyserver.sql # loading sql script: 24_zorder.sql # loading sql script: 25_debug.sql # loading sql script: 22_clients.sql # loading sql script: 23_skyserver.sql # loading sql script: 39_analytics.sql # loading sql script: 24_zorder.sql # loading sql script: 25_debug.sql # loading sql script: 39_analytics.sql # loading sql script: 75_storagemodel.sql # loading sql script: 75_storagemodel.sql # loading sql script: 80_udf.sql # loading sql script: 80_udf.sql # loading sql script: 99_system.sql # loading sql script: 99_system.sql #GDKmalloc(589484032) fails, try to free up space [memory in use=71647756904,vir tual memory in use=72723595880] #GDKmalloc(589484032) result [mem=71615637640,vm=72198121608]
Hi,
A first reaction. The symptoms indicate memory fragmentation.
The message itself is relatively clear. The system attempts to allocate 589MB from its swap space, which fails, eg caused by memory fragmentation.
The recovery step is correct, because you start with a fresh malloc pool. The length of the delay step is correlated with the updates that might be re-applied from the log and any cleanup.
regards, Martin
On 5/23/13 6:12 PM, Anthony Damico wrote:
Hi, I'm running a two-day-long program on Windows x64 with MonetDB.R that breaks somewhere into the second day due to what I believe is overloaded RAM? I was under the impression that this should not really happen with MonetDB, since it's a database that doesn't load everything into RAM.. Here's what's happening:
A big, complicated series of importations, merges, and recodes all work for the first two days on about 250,000,000 records of data.. but all the while my Windows Task Manager will slowly show the physical memory usage meter ticking up in the "Performance" tab. mserver.exe does NOT appear to be using RAM in the "Memory" column of the "Processes" tab, so it looks like there's a ghost using RAM.
Then, after two days, the script will break (mserver output shown below) - returning a GDK error on a seemingly-benign line of code, just a simple recode:
Error in .local(conn, statement, ..., async = async) : UPDATE ep10 SET rx_cso_cov = CVRD_D_PLAN_PD_AMT WHERE DRUG_CVRG_STUS_CD IN ( 'C','S','O' ) failed! Server says: !GDKerror:!ERROR: GDKmallocmax: failed for 589484032 bytes
After that, if I close mserver.exe and open it up again, it takes about twenty minutes to get the ">" mserver prompt. But it does open up eventually.. and at that point, if I re-connect to the server with R and run the exact same line, it works without error.
So it appears I just need to flush out that ghost/mystery RAM usage somehow? Does this error make sense?
There is no way this is due to lack of hard disk space - this server's C drive has 15GB free and the attached drive actually hosting all of this data has 235GB free. I don't want to rule it out, but I also doubt this problem is related to MonetDB.R, since the scenario I described does not require that I restart my R console, just mserver.exe.
I can work around the problem by manually killing and re-starting mserver.exe at a few logical breakpoints in my program, but I didn't think this was supposed to happen in the first place?
Thanks!!! Anthony
# MonetDB 5 server v11.15.7 "Feb2013-SP2" # Serving database 'medicare_sample', using 4 threads # Compiled for x86_64-pc-winnt/64bit with 64bit OIDs dynamically linked # Found 20.000 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2013 MonetDB B.V., all rights reserved # Visit http://www.monetdb.org/ for further information # Listening for connection requests on mapi:monetdb://127.0.0.1:49800/ http://127.0.0.1:49800/ # MonetDB/JAQL module loaded # MonetDB/SQL module loaded
# SQL catalog created, loading sql scripts once
# loading sql script: 09_like.sql # loading sql script: 10_math.sql # loading sql script: 11_times.sql # loading sql script: 12_url.sql # loading sql script: 13_date.sql # loading sql script: 14_inet.sql # loading sql script: 15_history.sql # loading sql script: 16_tracelog.sql # loading sql script: 17_compress.sql # loading sql script: 18_dictionary.sql # loading sql script: 19_cluster.sql # SQL catalog created, loading sql scripts once # loading sql script: 09_like.sql # loading sql script: 20_vacuum.sql # loading sql script: 10_math.sql # loading sql script: 11_times.sql # loading sql script: 12_url.sql # loading sql script: 21_dependency_functions.sql # loading sql script: 13_date.sql # loading sql script: 14_inet.sql # loading sql script: 15_history.sql # loading sql script: 16_tracelog.sql # loading sql script: 17_compress.sql # loading sql script: 18_dictionary.sql # loading sql script: 19_cluster.sql # loading sql script: 20_vacuum.sql # loading sql script: 21_dependency_functions.sql # loading sql script: 22_clients.sql # loading sql script: 23_skyserver.sql # loading sql script: 24_zorder.sql # loading sql script: 25_debug.sql # loading sql script: 22_clients.sql # loading sql script: 23_skyserver.sql # loading sql script: 39_analytics.sql # loading sql script: 24_zorder.sql # loading sql script: 25_debug.sql # loading sql script: 39_analytics.sql # loading sql script: 75_storagemodel.sql # loading sql script: 75_storagemodel.sql # loading sql script: 80_udf.sql # loading sql script: 80_udf.sql # loading sql script: 99_system.sql # loading sql script: 99_system.sql #GDKmalloc(589484032) fails, try to free up space [memory in use=71647756904,vir tual memory in use=72723595880] #GDKmalloc(589484032) result [mem=71615637640,vm=72198121608]
users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list
Thanks Martin! So is there any MonetDB command to initiate the log/cleanup and start with a fresh malloc pool? Or is the appropriate answer to this just shutting down the server and re-starting it? :)
On Thu, May 23, 2013 at 4:48 PM, Martin Kersten martin@monetdb.org wrote:
Hi,
A first reaction. The symptoms indicate memory fragmentation.
The message itself is relatively clear. The system attempts to allocate 589MB from its swap space, which fails, eg caused by memory fragmentation.
The recovery step is correct, because you start with a fresh malloc pool. The length of the delay step is correlated with the updates that might be re-applied from the log and any cleanup.
regards, Martin
On 5/23/13 6:12 PM, Anthony Damico wrote:
Hi, I'm running a two-day-long program on Windows x64 with MonetDB.R that breaks somewhere into the second day due to what I believe is overloaded RAM? I was under the impression that this should not really happen with MonetDB, since it's a database that doesn't load everything into RAM.. Here's what's happening:
A big, complicated series of importations, merges, and recodes all work for the first two days on about 250,000,000 records of data.. but all the while my Windows Task Manager will slowly show the physical memory usage meter ticking up in the "Performance" tab. mserver.exe does NOT appear to be using RAM in the "Memory" column of the "Processes" tab, so it looks like there's a ghost using RAM.
Then, after two days, the script will break (mserver output shown below)
- returning a GDK error on a seemingly-benign line of code, just a simple
recode:
Error in .local(conn, statement, ..., async = async) : UPDATE ep10 SET rx_cso_cov = CVRD_D_PLAN_PD_AMT WHERE DRUG_CVRG_STUS_CD IN ( 'C','S','O' ) failed! Server says: !GDKerror:!ERROR: GDKmallocmax: failed for 589484032 bytes
After that, if I close mserver.exe and open it up again, it takes about twenty minutes to get the ">" mserver prompt. But it does open up eventually.. and at that point, if I re-connect to the server with R and run the exact same line, it works without error.
So it appears I just need to flush out that ghost/mystery RAM usage somehow? Does this error make sense?
There is no way this is due to lack of hard disk space - this server's C drive has 15GB free and the attached drive actually hosting all of this data has 235GB free. I don't want to rule it out, but I also doubt this problem is related to MonetDB.R, since the scenario I described does not require that I restart my R console, just mserver.exe.
I can work around the problem by manually killing and re-starting mserver.exe at a few logical breakpoints in my program, but I didn't think this was supposed to happen in the first place?
Thanks!!! Anthony
# MonetDB 5 server v11.15.7 "Feb2013-SP2" # Serving database 'medicare_sample', using 4 threads # Compiled for x86_64-pc-winnt/64bit with 64bit OIDs dynamically linked # Found 20.000 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2013 MonetDB B.V., all rights reserved # Visit http://www.monetdb.org/ for further information # Listening for connection requests on mapi:monetdb://127.0.0.1:49800/ # MonetDB/JAQL module loaded # MonetDB/SQL module loaded
# SQL catalog created, loading sql scripts once
# loading sql script: 09_like.sql # loading sql script: 10_math.sql # loading sql script: 11_times.sql # loading sql script: 12_url.sql # loading sql script: 13_date.sql # loading sql script: 14_inet.sql # loading sql script: 15_history.sql # loading sql script: 16_tracelog.sql # loading sql script: 17_compress.sql # loading sql script: 18_dictionary.sql # loading sql script: 19_cluster.sql # SQL catalog created, loading sql scripts once # loading sql script: 09_like.sql # loading sql script: 20_vacuum.sql # loading sql script: 10_math.sql # loading sql script: 11_times.sql # loading sql script: 12_url.sql # loading sql script: 21_dependency_functions.sql # loading sql script: 13_date.sql # loading sql script: 14_inet.sql # loading sql script: 15_history.sql # loading sql script: 16_tracelog.sql # loading sql script: 17_compress.sql # loading sql script: 18_dictionary.sql # loading sql script: 19_cluster.sql # loading sql script: 20_vacuum.sql # loading sql script: 21_dependency_functions.sql # loading sql script: 22_clients.sql # loading sql script: 23_skyserver.sql # loading sql script: 24_zorder.sql # loading sql script: 25_debug.sql # loading sql script: 22_clients.sql # loading sql script: 23_skyserver.sql # loading sql script: 39_analytics.sql # loading sql script: 24_zorder.sql # loading sql script: 25_debug.sql # loading sql script: 39_analytics.sql # loading sql script: 75_storagemodel.sql # loading sql script: 75_storagemodel.sql # loading sql script: 80_udf.sql # loading sql script: 80_udf.sql # loading sql script: 99_system.sql # loading sql script: 99_system.sql #GDKmalloc(589484032) fails, try to free up space [memory in use=71647756904,vir tual memory in use=72723595880] #GDKmalloc(589484032) result [mem=71615637640,vm=72198121608]
users-list mailing listusers-list@monetdb.orghttp://mail.monetdb.org/mailman/listinfo/users-list
users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list
participants (2)
-
Anthony Damico -
Martin Kersten