De : Stefan Manegold <Stefan.Manegold@cwi.nl>
À : sylver_b <sylver_b@yahoo.com>
Cc : monetdb-developers@lists.sourceforge.net
Envoyé le : Mercredi, 21 Janvier 2009, 11h48mn 42s
Objet : Re : [Monetdb-developers] Re : mclient running out of memory, crashing mserver5
On Wed, Jan 21, 2009 at 03:12:20AM -0800, sylver_b wrote:
> Hi Stephan,
>
> I will reply below :
>
>
>
> ________________________________
>
> De : Stefan Manegold <
Stefan.Manegold@cwi.nl>
> À : sylver_b <
sylver_b@yahoo.com>
> Cc :
monetdb-developers@lists.sourceforge.net> Envoyé le : Mercredi, 21 Janvier 2009, 10h28mn 41s
> Objet : Re: [Monetdb-developers] Re : mclient running out of memory, crashing mserver5
>
>
> >> Yes
>
> How big is/was your mserver5 when running or just before crashing?
>
> >> missed to check that, but this is what i have at the moment :
>
> USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
> root 31679
0.0 3.2 1950816 131056 pts/3 Sl+ 10:05 0:00 | \_ mserver5 --dbinit=include sql;
This while being idle or while loading data?
In any case, with almost 2 GB vitual size, your mserver5 has obviously
reached the addressable limit on a 32-bit system (well, theoretically,
32-bit systems can address 4 GB, but in practice processes are often only
allowed to address 2 GB).
> Your machine and OS seem to be 64-bit, right?
> >> 32-bit
> What about your MonetDB installation?
> Could you please provide us with the output of
> `mserver5 --version`
> ?
> >> # mserver5 --version
> MonetDB server v5.8.0 (32-bit), based on kernel v1.26.0 (32-bit oids)
> Copyright (c) 1993-July
2008 CWI
> Copyright (c) August 2008- MonetDB B.V., all rights reserved
> Visit
http://monetdb.cwi.nl/for further information
> Configured for prefix: /root/MonetDB
> Libraries:
> openssl: OpenSSL 0.9.8b 04 May 2006 (compiled with OpenSSL 0.9.8b 04 May 2006)
> Compiled by: root@ulys
> Compilation: gcc -O2 -std=c99
> Linking : /usr/bin/ld
Ok, on a 32-bit system, 32-bit MonetDB can only handle data sizes up to 2 GB
at a time. Th efollwoing two limitations apply: each column of your tables
(called BAT in MonetDB lingo) cannot exceed 2 GB AND the total size of all
columns (BATs) active at a time while evaluating queries cannot exceed 2 GB,
i.e., the addressable data size on 32-bit systems. When bulkloading data,
all columns (BATs) of a table are active concurrently, hence, the total
table size cannot exceed
2 GB.
The GDKmmap failure in you previous posting suggetst that MonetDB cannot
allocate more memory (or to be more precise: address space), most probably
as the available 2 GB address space is already exhausted.
As Stefan (de Konink) suggested, you can test whether loading your data
succeeds without constraints (primary keys, foreign keys, etc.) defined on
your table(s). If not, your data size might just be too large for a 32-bit
MonetDB, and you'd need to resort to a 64-bit machine + 64-bit MonetDB or
consider fragmenting your data.
Hope this helps ...
Stefan
> How much free space is on your disk partition where your dbfarm is located?
> >> # df -h
> Filesystem Size Used Avail Use% Mounted on
> /dev/sda1 212G 180G
22G 90% /
> varrun 2.0G 56K 2.0G 1% /var/run
> varlock 2.0G 0 2.0G 0% /var/lock
> procbususb 10M 52K 10M 1% /proc/bus/usb
> udev 10M 52K 10M 1% /dev
> devshm 2.0G 0 2.0G 0% /dev/shm
>
>
> > Is there anything to do to prevent this ? should the DB swap at all ?
>
>
MonetDB does in-memory processing, i.e., all columns that are active at a time
> need to be in the processes address space, either being loaded or being
> memory mapped; so yes, with huge amount of data MonetDB will also use
> virtual memory, either as swap or as memory-mapped files.
>
> Stefan
>
> >
> >
> >
> >
> >
> > ________________________________
> > De : Stefan de Konink <
stefan@konink.de>
> > À : sylver_b <
sylver_b@yahoo.com>
> > Cc :
monetdb-developers@lists.sourceforge.net> > Envoyé le : Mercredi, 21 Janvier 2009, 1h09mn 13s
>
> Objet : Re: [Monetdb-developers] mclient running out of memory, crashing mserver5
> >
> > sylver_b wrote:
> > > I don't think i'm running out of memory. I had to stop and restart mserver5, flush the table, then re-run the command - all the records were inserted. Also I noticed this type of messages on the mserver5 console when piping records through the mclient (over 1million records) . It seems that the mclient is quiet instable (cf my post about "COPY, terminating connection" few months ago).
> > > So, is there a way to increase the memory limit or make sure mserver5 won't crash when running bash inserts via a cron job ?
> >
> > The only thing I was able to do against that; more swap for the job. And of course the standard stuff as loading CSV files, preferably no indices on the table you are loading huge amounts of data to.
> >
> >
> >
Stefan
> >
> >
> >
> >
> > ------------------------------------------------------------------------------
> > This SF.net email is sponsored by:
> > SourcForge Community
> > SourceForge wants to tell your story.
> >
http://p.sf.net/sfu/sf-spreadtheword> > _______________________________________________
> > Monetdb-developers mailing list
> >
Monetdb-developers@lists.sourceforge.net> >
https://lists.sourceforge.net/lists/listinfo/monetdb-developers>
>
> --
> | Dr. Stefan Manegold | mailto:
Stefan.Manegold@cwi.nl |
> | CWI, P.O.Box 94079 |
http://www.cwi.nl/~manegold/ |
> | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 |
> | The Netherlands | Fax : +31 (20) 592-4312 |
>
> On Wed, Jan 21, 2009 at 02:07:04AM -0800, sylver_b wrote:
> > Hi
> >
> > Interesting..
> >
> > Even worst, this morning i found out that all the inserts of last night failed !! :(
> >
> >
> > #BBPTRIM_ENTER: memsize=1144385432,vmsize=1599613808
> > #BBPTRIM: memtarget=819717481 vmtarget=0
> > #BBPTRIM_EXIT: memsize=366265384,vmsize=1599605544
> > #BBPTRIM_ENTER: memsize=1173710640,vmsize=1628939016
> > #BBPTRIM:
memtarget=849042689 vmtarget=0
> > #BBPTRIM_EXIT: memsize=366265384,vmsize=1628939016
> > #BBPTRIM_ENTER: memsize=1143635024,vmsize=1629665320
> > #BBPTRIM: memtarget=0 vmtarget=19052584
> > #BBPTRIM_EXIT: memsize=366265384,vmsize=1598862320
> > #BBPTRIM_ENTER: memsize=1143633968,vmsize=1645786120
> > #BBPTRIM: memtarget=0 vmtarget=35173384
> > #BBPTRIM_EXIT: memsize=366265384,vmsize=1645786120
> > #BBPTRIM_ENTER: memsize=1143633992,vmsize=1676588064
> > #BBPTRIM: memtarget=0 vmtarget=65975328
> > #BBPTRIM_EXIT: memsize=366265384,vmsize=1598862320
> > #BBPTRIM_ENTER: memsize=1143633944,vmsize=1613149168
> > #BBPTRIM: memtarget=0 vmtarget=2536432
> > #BBPTRIM_EXIT: memsize=366265384,vmsize=1582347224
> > #BBPTRIM_ENTER: memsize=1143633920,vmsize=1620095960
> > #BBPTRIM: memtarget=0 vmtarget=9483224
> >
#BBPTRIM_EXIT: memsize=366265384,vmsize=1620095960
> > #GDKmmap(148111360) fail => BBPtrim(enter) usage[mem=1143633920,vm=1496757208]
> > #BBPTRIM_ENTER: memsize=1143633920,vmsize=1496757208
> > #BBPTRIM: memtarget=0 vmtarget=1073741824
> > #BBPTRIM_EXIT: memsize=366265384,vmsize=1496757208
> > #GDKmmap(148111360) fail => BBPtrim(ready) usage[mem=1143633920,vm=1496757208]
> > !ERROR: GDKload: cannot mmap(): name=32/3267, ext=theap
> > !OS: Cannot allocate memory
> > !ERROR: GDKload: failed name=32/3267, ext=theap
> >
> > top - 10:00:34 up 249 days, 20:31, 4 users, load average: 0.28, 0.12, 0.04
> > Tasks: 86 total, 1 running, 85 sleeping, 0 stopped, 0 zombie
> > Cpu(s): 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>
> Mem: 4025600k total, 3633228k used, 392372k free, 107544k buffers
> > Swap: 9502408k total, 92k used, 9502316k free, 1520104k cached
>
> Is that after mserver5 had crashed?
>
>
>
--
| Dr. Stefan Manegold | mailto:
Stefan.Manegold@cwi.nl |
| CWI, P.O.Box 94079 |
http://www.cwi.nl/~manegold/ |
| 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 |
| The Netherlands | Fax : +31 (20) 592-4312 |