[Monetdb-developers] Re : Re : mclient running out of memory, crashing mserver5

sylver_b sylver_b at yahoo.com
Wed Jan 21 14:32:53 CET 2009


Thanks..





________________________________
De : Stefan Manegold <Stefan.Manegold at cwi.nl>
À : sylver_b <sylver_b at yahoo.com>
Cc : monetdb-developers at lists.sourceforge.net
Envoyé le : Mercredi, 21 Janvier 2009, 12h40mn 10s
Objet : Re: [Monetdb-developers] Re : mclient running out of memory, crashing mserver5

On Wed, Jan 21, 2009 at 04:16:00AM -0800, sylver_b wrote:
> By the way ... what will happen if i only have 4GB of memory on a 64-bit system and table size over 10GB (for example) ?

MonetDB will use virtual memory, mostly memory mapped files, partly swap ---
until the available disk capacity or address space is exceeded --- whatever
limit comes first (holds for both 32- & 64-bit systems).

Stefan

> 
> 
> 
> 
> ________________________________
> De : Stefan Manegold <Stefan.Manegold at cwi.nl>
> À : sylver_b <sylver_b at yahoo.com>
> Cc : monetdb-developers at lists.sourceforge.net
> Envoyé le : Mercredi, 21 Janvier 2009, 11h48mn 42s
> Objet : Re : [Monetdb-developers] Re : mclient running out of memory, crashing mserver5
> 
> On Wed, Jan 21, 2009 at 03:12:20AM -0800, sylver_b wrote:
> > Hi Stephan,
> >  
> > I will reply below :
> > 
> > 
> > 
> > ________________________________
> > 
> > De : Stefan Manegold <Stefan.Manegold at cwi.nl>
> > À : sylver_b <sylver_b at yahoo.com>
> > Cc : monetdb-developers at lists.sourceforge.net
> > Envoyé le : Mercredi, 21 Janvier 2009, 10h28mn 41s
> > Objet : Re: [Monetdb-developers] Re : mclient running out of memory, crashing mserver5
> > 
> > 
> > >> Yes
> > 
> > How big is/was your mserver5 when running or just before crashing?
> > 
> > >> missed to check that, but this is what i have at the moment :
> >  
> > USER       PID     %CPU %MEM    VSZ        RSS     TTY      STAT START   TIME COMMAND
> > root          31679  0.0       3.2          1950816 131056 pts/3     Sl+    10:05      0:00  |   \_ mserver5 --dbinit=include sql;
> 
> This while being idle or while loading data?
> 
> In any case, with almost 2 GB vitual size, your mserver5 has obviously
> reached the addressable limit on a 32-bit system (well, theoretically,
> 32-bit systems can address 4 GB, but in practice processes are often only
> allowed to address 2 GB).
> 
> > Your machine and OS seem to be 64-bit, right?
> > >> 32-bit
> 
> > What about your MonetDB installation?
> > Could you please provide us with the output of
> >     `mserver5 --version`
> > ?
> > >> # mserver5 --version
> > MonetDB server v5.8.0 (32-bit), based on kernel v1.26.0 (32-bit oids)
> > Copyright (c) 1993-July 2008 CWI
> > Copyright (c) August 2008- MonetDB B.V., all rights reserved
> > Visit http://monetdb.cwi.nl/for further information
> > Configured for prefix: /root/MonetDB
> > Libraries:
> >   openssl: OpenSSL 0.9.8b 04 May 2006 (compiled with OpenSSL 0.9.8b 04 May 2006)
> > Compiled by: root at ulys
> > Compilation: gcc -O2 -std=c99
> > Linking    : /usr/bin/ld
> 
> Ok, on a 32-bit system, 32-bit MonetDB can only handle data sizes up to 2 GB
> at a time. Th efollwoing two limitations apply: each column of your tables
> (called BAT in MonetDB lingo) cannot exceed 2 GB AND the total size of all
> columns (BATs) active at a time while evaluating queries cannot exceed 2 GB,
> i.e., the addressable data size on 32-bit systems. When bulkloading data,
> all columns (BATs) of a table are active concurrently, hence, the total
> table size cannot exceed 2 GB.
> 
> The GDKmmap failure in you previous posting suggetst that MonetDB cannot
> allocate more memory (or to be more precise: address space), most probably
> as the available 2 GB address space is already exhausted.
> 
> As Stefan (de Konink) suggested, you can test whether loading your data
> succeeds without constraints (primary keys, foreign keys, etc.) defined on
> your table(s). If not, your data size might just be too large for a 32-bit
> MonetDB, and you'd need to resort to a 64-bit machine + 64-bit MonetDB or
> consider fragmenting your data.
> 
> Hope this helps ...
> 
> Stefan
> 
> > How much free space is on your disk partition where your dbfarm is located?
> > >> # df -h
> > Filesystem            Size Used Avail Use% Mounted on
> > /dev/sda1             212G  180G   22G  90% /
> > varrun                2.0G   56K  2.0G   1% /var/run
> > varlock               2.0G     0  2.0G   0% /var/lock
> > procbususb             10M   52K   10M   1% /proc/bus/usb
> > udev                   10M   52K   10M   1% /dev
> > devshm                2.0G     0  2.0G   0% /dev/shm
> > 
> > 
> > > Is there anything to do to prevent this ? should the DB swap at all ?
> > 
> > MonetDB does in-memory processing, i.e., all columns that are active at a time
> > need to be in the processes address space, either being loaded or being
> > memory mapped; so yes, with huge amount of data MonetDB will also use
> > virtual memory, either as swap or as memory-mapped files.
> > 
> > Stefan
> > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > > ________________________________
> > > De : Stefan de Konink <stefan at konink.de>
> > > À : sylver_b <sylver_b at yahoo.com>
> > > Cc : monetdb-developers at lists.sourceforge.net
> > > Envoyé le : Mercredi, 21 Janvier 2009, 1h09mn 13s
> > > Objet : Re: [Monetdb-developers] mclient running out of memory, crashing mserver5
> > > 
> > > sylver_b wrote:
> > > > I don't think i'm running out of memory. I had to stop and restart mserver5, flush the table, then re-run the command - all the records were inserted. Also I noticed this type of messages on the mserver5 console when piping records through the mclient (over 1million records) . It seems that the mclient is quiet instable (cf my post about "COPY, terminating connection" few months ago).
> > > >  So, is there a way to increase the memory limit  or make sure mserver5 won't crash when running bash inserts via a cron job ?
> > > 
> > > The only thing I was able to do against that; more swap for the job. And of course the standard stuff as loading CSV files, preferably no indices on the table you are loading huge amounts of data to.
> > > 
> > > 
> > > Stefan
> > > 
> > > 
> > > 
> > >      
> > > ------------------------------------------------------------------------------
> > > This SF.net email is sponsored by:
> > > SourcForge Community
> > > SourceForge wants to tell your story.
> > > http://p.sf.net/sfu/sf-spreadtheword
> > > _______________________________________________
> > > Monetdb-developers mailing list
> > > Monetdb-developers at lists.sourceforge.net
> > > https://lists.sourceforge.net/lists/listinfo/monetdb-developers
> > 
> > 
> > -- 
> > | Dr. Stefan Manegold | mailto:Stefan.Manegold at cwi.nl |
> > | CWI,  P.O.Box 94079 | http://www.cwi.nl/~manegold/  |
> > | 1090 GB Amsterdam  | Tel.: +31 (20) 592-4212      |
> > | The Netherlands    | Fax : +31 (20) 592-4312      |
> > 
> > On Wed, Jan 21, 2009 at 02:07:04AM -0800, sylver_b wrote:
> > > Hi 
> > > 
> > > Interesting..
> > > 
> > > Even worst, this morning i found out that all the inserts of last night failed !! :(
> > > 
> > > 
> > > #BBPTRIM_ENTER: memsize=1144385432,vmsize=1599613808
> > > #BBPTRIM: memtarget=819717481 vmtarget=0
> > > #BBPTRIM_EXIT: memsize=366265384,vmsize=1599605544
> > > #BBPTRIM_ENTER: memsize=1173710640,vmsize=1628939016
> > > #BBPTRIM: memtarget=849042689 vmtarget=0
> > > #BBPTRIM_EXIT: memsize=366265384,vmsize=1628939016
> > > #BBPTRIM_ENTER: memsize=1143635024,vmsize=1629665320
> > > #BBPTRIM: memtarget=0 vmtarget=19052584
> > > #BBPTRIM_EXIT: memsize=366265384,vmsize=1598862320
> > > #BBPTRIM_ENTER: memsize=1143633968,vmsize=1645786120
> > > #BBPTRIM: memtarget=0 vmtarget=35173384
> > > #BBPTRIM_EXIT: memsize=366265384,vmsize=1645786120
> > > #BBPTRIM_ENTER: memsize=1143633992,vmsize=1676588064
> > > #BBPTRIM: memtarget=0 vmtarget=65975328
> > > #BBPTRIM_EXIT: memsize=366265384,vmsize=1598862320
> > > #BBPTRIM_ENTER: memsize=1143633944,vmsize=1613149168
> > > #BBPTRIM: memtarget=0 vmtarget=2536432
> > > #BBPTRIM_EXIT: memsize=366265384,vmsize=1582347224
> > > #BBPTRIM_ENTER: memsize=1143633920,vmsize=1620095960
> > > #BBPTRIM: memtarget=0 vmtarget=9483224
> > > #BBPTRIM_EXIT: memsize=366265384,vmsize=1620095960
> > > #GDKmmap(148111360) fail => BBPtrim(enter) usage[mem=1143633920,vm=1496757208]
> > > #BBPTRIM_ENTER: memsize=1143633920,vmsize=1496757208
> > > #BBPTRIM: memtarget=0 vmtarget=1073741824
> > > #BBPTRIM_EXIT: memsize=366265384,vmsize=1496757208
> > > #GDKmmap(148111360) fail => BBPtrim(ready) usage[mem=1143633920,vm=1496757208]
> > > !ERROR: GDKload: cannot mmap(): name=32/3267, ext=theap
> > > !OS: Cannot allocate memory
> > > !ERROR: GDKload: failed name=32/3267, ext=theap
> > > 
> > > top - 10:00:34 up 249 days, 20:31,  4 users,  load average: 0.28, 0.12, 0.04
> > > Tasks:  86 total,   1 running,  85 sleeping,   0 stopped,   0 zombie
> > > Cpu(s):  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> > > Mem:   4025600k total,  3633228k used,   392372k free,   107544k buffers
> > > Swap:  9502408k total,       92k used,  9502316k free,  1520104k cached
> > 
> > Is that after mserver5 had crashed?
> > 
> > 
> >      
> -- 
> | Dr. Stefan Manegold | mailto:Stefan.Manegold at cwi.nl |
> | CWI,  P.O.Box 94079 | http://www.cwi.nl/~manegold/  |
> | 1090 GB Amsterdam  | Tel.: +31 (20) 592-4212      |
> | The Netherlands    | Fax : +31 (20) 592-4312      |
> 
> 
> 
>      
-- 
| Dr. Stefan Manegold | mailto:Stefan.Manegold at cwi.nl |
| CWI,  P.O.Box 94079 | http://www.cwi.nl/~manegold/  |
| 1090 GB Amsterdam  | Tel.: +31 (20) 592-4212      |
| The Netherlands    | Fax : +31 (20) 592-4312      |



      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.monetdb.org/pipermail/developers-list/attachments/20090121/7e64620d/attachment.html>


More information about the developers-list mailing list