Thanks Fabian, you are right, it lives again. Tomorrow I will try to kill it once again having more free disk space.
2005/10/3, Fabian Fabian.Groffen@cwi.nl:
I think your problem right now is that there is still a "log" directory. Remove both dbfarm and log directories and try again. This should at least revive MonetDB a bit ;)
Andrei Martsinchyk wrote:
Thanks for response, I do not believe that 2GB BAT size limit was hit. Largest column in the table was defined as varchar(44), I had 12,000,000 records loaded and wanted to add 6,000,000 more records. So, there should be 18 * 44=792MBytes. It depends how BAT is represented internally, though. I am afraid I ran out of memory or disk space. Anyway the error has not been handled and now I can not start server to run tests once again and clarify the problem. Could anybody help me to find out what happened to server ?
2005/10/3, Fabian Fabian.Groffen@cwi.nl:
Andrei,
Sorry that your previous post (seemed to be) ignored. Unfortunately nobody was able to respond in time.
I am not sure on this answer, but if I recall correctly MonetDB cannot possibly process columns which are larger than the addresseable memory. This is due to the fact that MonetDB is a main-memory database.
Please someone correct me if I'm wrong here.
Regards, Fabian
Andrei Martsinchyk wrote:
Hello,
It seems MonetDB has problem with handling of huge tables (larger than amount of memory available). I tried to load large amount of data into a table using COPY command. I loaded same data (6 millions records, 700+MBytes) few times. Loading was successful two times, on third time server crushed. After crush I could not start the server again, it output messages
!WARNING: GDKlockHome: ignoring empty or invalid .gdk_lock. !FATAL: BBPinit: cannot properly process bat/BACKUP/.
and exited. Removing of the database data directory did not help much - server output messages
!ERROR:mvc_init: unable to create system tables !WARNING: BBPincref: range error 410
Couple minutes later it output
!ERROR: BATSIGcrash: Mserver internal error (Segmentation fault), please restart . !ERROR: (One potential cause could be that your disk might be full...)
Well, insufficient disk space could be a cause, before server crashed I had approximately 2GB free. But I have cleaned up 10 GB and problem persists.
How can I troubleshoot and fix the server ? What are memory / disk space requirements when bulk loading data ?
I tested with 32-bit system, 512 MB RAM + 1GB swap, Ubuntu Linux, MonetDB-4.9.3, nightly build dated September 30.
-- Best regards, Andrei Martsinchyk mailto:andrei.martsinchyk@gmail.com
This SF.Net email is sponsored by: Power Architecture Resource Center: Free content, downloads, discussions, and more. http://solutions.newsforge.com/ibmarch.tmpl _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
This SF.Net email is sponsored by: Power Architecture Resource Center: Free content, downloads, discussions, and more. http://solutions.newsforge.com/ibmarch.tmpl _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
-- Best regards, Andrei Martsinchyk mailto:andrei.martsinchyk@gmail.com
This SF.Net email is sponsored by: Power Architecture Resource Center: Free content, downloads, discussions, and more. http://solutions.newsforge.com/ibmarch.tmpl _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
-- Best regards, Andrei Martsinchyk mailto:andrei.martsinchyk@gmail.com