[Monetdb-developers] Unable to shred 1gb xml file (OS: Not enough space)

kla gw klarinda.gw at gmail.com
Mon Jul 24 11:43:33 CEST 2006


Hi,

I tried MonetDB/XQuery to shred a 1gb xml file, but it failed.

Following is the error message:
MonetDB>shred_doc("D:xbench/output/DC1000catalog.xml", "DC1000catalog.xml");
!ERROR: MT_mmap: MapViewOfFile(6b4, 2, 0, 0, 442499072, 0) failed
!OS: Not enough space
!GDKmmap(442499072) fail => BBPtrim(enter) usage[mem=101297568,vm=794361856]

I use windows XP Professional version 2002 service pack 2, Pentium 4
CPU, 2.40GHz, 512 MB of RAM.

Previously I tried to shred 100mb xml file, and it took 18.532 sec.
For this 1 gb file, I left it overnight so I don't know how long it
takes till the error message occurs.

Can please help me to solve this problem?

Regards,

Klarinda


Below is the complete error message:
MonetDB>shred_doc("D:xbench/output/DC1000catalog.xml", "DC1000catalog.xml");
!ERROR: MT_mmap: MapViewOfFile(6b4, 2, 0, 0, 442499072, 0) failed
!OS: Not enough space
!GDKmmap(442499072) fail => BBPtrim(enter) usage[mem=101297568,vm=794361856]
#
!mallinfo.arena = 15613828
!mallinfo.ordblks = 46134
!mallinfo.smblks = 15492
!mallinfo.hblkhd = 0
!mallinfo.hblks = 0
!mallinfo.usmblks = 13718408
!mallinfo.fsmblks = 899720
!mallinfo.uordblks = 950740
!mallinfo.fordblks = 44960
#BBPTRIM_ENTER: memsize=101297568,vmsize=794361856
#BBPTRIM: memtarget=0 vmtarget=1073741824
#TRIMSCAN: mem=0 vm=1, start=1, limit=1
#TRIMSCAN:      145030          0=tmp_35        (#0)
#TRIMSCAN:      145059          1=tmp_36        (#0)
#TRIMSCAN:      145088          2=tmp_37        (#0)
#TRIMSCAN:      145146          3=tmp_41        (#0)
#TRIMSCAN:      149075          4=doc_query     (#0)
#TRIMSCAN:      149092          5=doc_sema      (#0)
#TRIMSCAN:      155215          6=tmp_374       (#0)
#TRIMSCAN:      155218          7=prop_pre_39   (#0)
#TRIMSCAN:      157895          8=tmp_533       (#0)
#TRIMSCAN:      157898          9=prop_pre_310  (#0)
#TRIMSCAN: end at 1 (size=628)
#TRIMSELECT: dirty = 0
#TRIMSELECT: candidate=tmp_35 BAT*=03D6A230
#            (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=145030,145030
,145030)
#TRIMSELECT: keep tmp_35 [224,0] bytes [224,0] dirty target(mem=0 vm=1073741824)

#TRIMSELECT: candidate=tmp_36 BAT*=03D66E60
#            (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=145059,145059
,145059)
#TRIMSELECT: keep tmp_36 [224,0] bytes [224,0] dirty target(mem=0 vm=1073741824)

#TRIMSELECT: candidate=tmp_37 BAT*=058080B0
#            (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=145088,145088
,145088)
#TRIMSELECT: keep tmp_37 [224,0] bytes [224,0] dirty target(mem=0 vm=1073741824)

#TRIMSELECT: candidate=tmp_41 BAT*=058066F0
#            (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=145146,145146
,145146)
#TRIMSELECT: keep tmp_41 [224,0] bytes [224,0] dirty target(mem=0 vm=1073741824)

#TRIMSELECT: candidate=doc_query BAT*=057F9D10
#            (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=149075,149075
,149075)
#TRIMSELECT: keep doc_query [224,0] bytes [224,0] dirty target(mem=0 vm=10737418
24)
#TRIMSELECT: candidate=doc_sema BAT*=05845370
#            (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=149092,149092
,149092)
#TRIMSELECT: keep doc_sema [224,0] bytes [224,0] dirty target(mem=0 vm=107374182
4)
#TRIMSELECT: candidate=tmp_374 BAT*=03D722F0
#            (cnt=0, mode=4096, refs=0, wait=0, parent=0, lastused=155215,155215
,155215)
#TRIMSELECT: keep tmp_374 [224,0] bytes [0,0] dirty target(mem=0 vm=1073741824)
#TRIMSELECT: candidate=prop_pre_39 BAT*=05845CD0
#            (cnt=0, mode=4096, refs=0, wait=0, parent=0, lastused=155218,155218
,155218)
#TRIMSELECT: keep prop_pre_39 [224,0] bytes [0,0] dirty target(mem=0 vm=10737418
24)
#TRIMSELECT: candidate=tmp_533 BAT*=0583C598
#            (cnt=0, mode=4096, refs=0, wait=0, parent=0, lastused=157895,157895
,157895)
#TRIMSELECT: keep tmp_533 [224,0] bytes [0,0] dirty target(mem=0 vm=1073741824)
#TRIMSELECT: candidate=prop_pre_310 BAT*=057F7E70
#            (cnt=0, mode=4096, refs=0, wait=0, parent=0, lastused=157898,157898
,157898)
#TRIMSELECT: keep prop_pre_310 [224,0] bytes [0,0] dirty target(mem=0 vm=1073741
824)
#TRIMSELECT: end
#TRIMSELECT: dirty = 1
#TRIMSELECT: candidate=tmp_35 BAT*=03D6A230
#            (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=145030,145030
,145030)
#TRIMSELECT: delete tmp_35 from trimlist (does not match trim needs)
#TRIMSELECT: candidate=tmp_36 BAT*=03D66E60
#            (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=145059,145059
,145059)
#TRIMSELECT: delete tmp_36 from trimlist (does not match trim needs)
#TRIMSELECT: candidate=tmp_37 BAT*=058080B0
#            (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=145088,145088
,145088)
#TRIMSELECT: delete tmp_37 from trimlist (does not match trim needs)
#TRIMSELECT: candidate=tmp_41 BAT*=058066F0
#            (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=145146,145146
,145146)
#TRIMSELECT: delete tmp_41 from trimlist (does not match trim needs)
#TRIMSELECT: candidate=doc_query BAT*=057F9D10
#            (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=149075,149075
,149075)
#TRIMSELECT: delete doc_query from trimlist (does not match trim needs)
#TRIMSELECT: candidate=doc_sema BAT*=05845370
#            (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=149092,149092
,149092)
#TRIMSELECT: delete doc_sema from trimlist (does not match trim needs)
#TRIMSELECT: candidate=tmp_374 BAT*=03D722F0
#            (cnt=0, mode=4096, refs=0, wait=0, parent=0, lastused=155215,155215
,155215)
#TRIMSELECT: delete tmp_374 from trimlist (does not match trim needs)
#TRIMSELECT: candidate=prop_pre_39 BAT*=05845CD0
#            (cnt=0, mode=4096, refs=0, wait=0, parent=0, lastused=155218,155218
,155218)
#TRIMSELECT: delete prop_pre_39 from trimlist (does not match trim needs)
#TRIMSELECT: candidate=tmp_533 BAT*=0583C598
#            (cnt=0, mode=4096, refs=0, wait=0, parent=0, lastused=157895,157895
,157895)
#TRIMSELECT: delete tmp_533 from trimlist (does not match trim needs)
#TRIMSELECT: candidate=prop_pre_310 BAT*=057F7E70
#            (cnt=0, mode=4096, refs=0, wait=0, parent=0, lastused=157898,157898
,157898)
#TRIMSELECT: delete prop_pre_310 from trimlist (does not match trim needs)
#TRIMSELECT: end
#BBPTRIM: no more unload candidates!
#BBPTRIM_EXIT: memsize=95140356,vmsize=794361856
!GDKmmap(442499072) fail => BBPtrim(ready) usage[mem=101297568,vm=794361856]
#
!mallinfo.arena = 15613828
!mallinfo.ordblks = 46134
!mallinfo.smblks = 15492
!mallinfo.hblkhd = 0
!mallinfo.hblks = 0
!mallinfo.usmblks = 13718408
!mallinfo.fsmblks = 899720
!mallinfo.uordblks = 950740
!mallinfo.fordblks = 44960
!ERROR: MT_mmap: MapViewOfFile(6b0, 2, 0, 0, 442499072, 0) failed
!OS: Not enough space
!ERROR: GDKload: cannot mmap(): name=05\552, ext=theap.priv
!OS: Not enough space
!ERROR: GDKload failed: name=05\552, ext=theap.priv
!ERROR: shredder.mx:append_str2bat: APPEND-STR[PROP_TEXT](final foxes since the
silent, quick realms should breach never sheaves--ruthless, daring waters beneat
h the close asymptotes c), BUNappend fails
!ERROR: CMDshred2bats: operation failed.
MonetDB>




More information about the developers-list mailing list