Another huge issue is that deleting records from a table still won't shrink
the size of the table. I like to keep some of my raw data tables small so
once a day I delete some already processed records out of them, dump the
table, and then reload it.
To manage the rest of the database I have a series of scripts that check to
see if the database is using a stupid amount of memory (80%+ of physical
memory on the machine) or to see if the database is deadlocked (this happens
several times a day). If either of these situations exist then I SIGHUP the
merovingian and mserver processes... when the database comes back up and
gets primed then it will be using a reasonable amount of memory for about 8
hours or so.
This is all acceptable at the moment because the database is ridiculously
fast.
73,
Matthew W. Jones (KI4ZIB)
http://matburt.net
On Mon, May 17, 2010 at 9:54 AM, Stefan de Konink
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512
Op 17-05-10 15:39, Matthew Jones schreef:
You're still going to have to be very careful with MonetDB, depending on the volume of incoming data you'll need to restart the database at least a couple of times a day.
One new developer already managed to get 90GB of used memory on a table having only 22MB, consisting of integers and timestamps. I did congratulate this user with this achievement, but want to reproduce it.
Interestingly, after a restart (ok a kill) everything went back to acceptable levels: 1191MB (virtual), 289MB (resources), 11MB (shared) it is interesting to see how such things can expand.
Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.15 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEAREKAAYFAkvxSoEACgkQYH1+F2Rqwn3LdgCdEXJdCs1uGH+1CVFTAIdTt6xs PQcAoI7AYnwTn5dFf8MB4GrCdqAyUU8M =pB2H -----END PGP SIGNATURE-----
------------------------------------------------------------------------------
_______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users