August 2006 - developers-list

[Monetdb-developers] [PATCH] Mx Io.c more specific errors
by Fabian Groffen 09 Aug '06

09 Aug '06

Hi devs, I ran into a problem with Mx, where IoWriteToFile seemed to have a problem but didn't tell exactly what. With the attached patch, I was able to figure out what the problem was. I don't like to commit it, as the Mx code is completely unknown to me. Also, I have no ideas on the effects on this highly portable piece of code on Windows. If someone considers it useful to commit, please do.

3 5

[Monetdb-developers] Unable to shred 1gb xml file (OS: Not enough space)
by kla gw 08 Aug '06

08 Aug '06

Hi, I tried MonetDB/XQuery to shred a 1gb xml file, but it failed. Following is the error message: MonetDB>shred_doc("D:xbench/output/DC1000catalog.xml", "DC1000catalog.xml"); !ERROR: MT_mmap: MapViewOfFile(6b4, 2, 0, 0, 442499072, 0) failed !OS: Not enough space !GDKmmap(442499072) fail => BBPtrim(enter) usage[mem=101297568,vm=794361856] I use windows XP Professional version 2002 service pack 2, Pentium 4 CPU, 2.40GHz, 512 MB of RAM. Previously I tried to shred 100mb xml file, and it took 18.532 sec. For this 1 gb file, I left it overnight so I don't know how long it takes till the error message occurs. Can please help me to solve this problem? Regards, Klarinda Below is the complete error message: MonetDB>shred_doc("D:xbench/output/DC1000catalog.xml", "DC1000catalog.xml"); !ERROR: MT_mmap: MapViewOfFile(6b4, 2, 0, 0, 442499072, 0) failed !OS: Not enough space !GDKmmap(442499072) fail => BBPtrim(enter) usage[mem=101297568,vm=794361856] # !mallinfo.arena = 15613828 !mallinfo.ordblks = 46134 !mallinfo.smblks = 15492 !mallinfo.hblkhd = 0 !mallinfo.hblks = 0 !mallinfo.usmblks = 13718408 !mallinfo.fsmblks = 899720 !mallinfo.uordblks = 950740 !mallinfo.fordblks = 44960 #BBPTRIM_ENTER: memsize=101297568,vmsize=794361856 #BBPTRIM: memtarget=0 vmtarget=1073741824 #TRIMSCAN: mem=0 vm=1, start=1, limit=1 #TRIMSCAN: 145030 0=tmp_35 (#0) #TRIMSCAN: 145059 1=tmp_36 (#0) #TRIMSCAN: 145088 2=tmp_37 (#0) #TRIMSCAN: 145146 3=tmp_41 (#0) #TRIMSCAN: 149075 4=doc_query (#0) #TRIMSCAN: 149092 5=doc_sema (#0) #TRIMSCAN: 155215 6=tmp_374 (#0) #TRIMSCAN: 155218 7=prop_pre_39 (#0) #TRIMSCAN: 157895 8=tmp_533 (#0) #TRIMSCAN: 157898 9=prop_pre_310 (#0) #TRIMSCAN: end at 1 (size=628) #TRIMSELECT: dirty = 0 #TRIMSELECT: candidate=tmp_35 BAT*=03D6A230 # (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=145030,145030 ,145030) #TRIMSELECT: keep tmp_35 [224,0] bytes [224,0] dirty target(mem=0 vm=1073741824) #TRIMSELECT: candidate=tmp_36 BAT*=03D66E60 # (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=145059,145059 ,145059) #TRIMSELECT: keep tmp_36 [224,0] bytes [224,0] dirty target(mem=0 vm=1073741824) #TRIMSELECT: candidate=tmp_37 BAT*=058080B0 # (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=145088,145088 ,145088) #TRIMSELECT: keep tmp_37 [224,0] bytes [224,0] dirty target(mem=0 vm=1073741824) #TRIMSELECT: candidate=tmp_41 BAT*=058066F0 # (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=145146,145146 ,145146) #TRIMSELECT: keep tmp_41 [224,0] bytes [224,0] dirty target(mem=0 vm=1073741824) #TRIMSELECT: candidate=doc_query BAT*=057F9D10 # (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=149075,149075 ,149075) #TRIMSELECT: keep doc_query [224,0] bytes [224,0] dirty target(mem=0 vm=10737418 24) #TRIMSELECT: candidate=doc_sema BAT*=05845370 # (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=149092,149092 ,149092) #TRIMSELECT: keep doc_sema [224,0] bytes [224,0] dirty target(mem=0 vm=107374182 4) #TRIMSELECT: candidate=tmp_374 BAT*=03D722F0 # (cnt=0, mode=4096, refs=0, wait=0, parent=0, lastused=155215,155215 ,155215) #TRIMSELECT: keep tmp_374 [224,0] bytes [0,0] dirty target(mem=0 vm=1073741824) #TRIMSELECT: candidate=prop_pre_39 BAT*=05845CD0 # (cnt=0, mode=4096, refs=0, wait=0, parent=0, lastused=155218,155218 ,155218) #TRIMSELECT: keep prop_pre_39 [224,0] bytes [0,0] dirty target(mem=0 vm=10737418 24) #TRIMSELECT: candidate=tmp_533 BAT*=0583C598 # (cnt=0, mode=4096, refs=0, wait=0, parent=0, lastused=157895,157895 ,157895) #TRIMSELECT: keep tmp_533 [224,0] bytes [0,0] dirty target(mem=0 vm=1073741824) #TRIMSELECT: candidate=prop_pre_310 BAT*=057F7E70 # (cnt=0, mode=4096, refs=0, wait=0, parent=0, lastused=157898,157898 ,157898) #TRIMSELECT: keep prop_pre_310 [224,0] bytes [0,0] dirty target(mem=0 vm=1073741 824) #TRIMSELECT: end #TRIMSELECT: dirty = 1 #TRIMSELECT: candidate=tmp_35 BAT*=03D6A230 # (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=145030,145030 ,145030) #TRIMSELECT: delete tmp_35 from trimlist (does not match trim needs) #TRIMSELECT: candidate=tmp_36 BAT*=03D66E60 # (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=145059,145059 ,145059) #TRIMSELECT: delete tmp_36 from trimlist (does not match trim needs) #TRIMSELECT: candidate=tmp_37 BAT*=058080B0 # (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=145088,145088 ,145088) #TRIMSELECT: delete tmp_37 from trimlist (does not match trim needs) #TRIMSELECT: candidate=tmp_41 BAT*=058066F0 # (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=145146,145146 ,145146) #TRIMSELECT: delete tmp_41 from trimlist (does not match trim needs) #TRIMSELECT: candidate=doc_query BAT*=057F9D10 # (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=149075,149075 ,149075) #TRIMSELECT: delete doc_query from trimlist (does not match trim needs) #TRIMSELECT: candidate=doc_sema BAT*=05845370 # (cnt=0, mode=1024, refs=0, wait=0, parent=0, lastused=149092,149092 ,149092) #TRIMSELECT: delete doc_sema from trimlist (does not match trim needs) #TRIMSELECT: candidate=tmp_374 BAT*=03D722F0 # (cnt=0, mode=4096, refs=0, wait=0, parent=0, lastused=155215,155215 ,155215) #TRIMSELECT: delete tmp_374 from trimlist (does not match trim needs) #TRIMSELECT: candidate=prop_pre_39 BAT*=05845CD0 # (cnt=0, mode=4096, refs=0, wait=0, parent=0, lastused=155218,155218 ,155218) #TRIMSELECT: delete prop_pre_39 from trimlist (does not match trim needs) #TRIMSELECT: candidate=tmp_533 BAT*=0583C598 # (cnt=0, mode=4096, refs=0, wait=0, parent=0, lastused=157895,157895 ,157895) #TRIMSELECT: delete tmp_533 from trimlist (does not match trim needs) #TRIMSELECT: candidate=prop_pre_310 BAT*=057F7E70 # (cnt=0, mode=4096, refs=0, wait=0, parent=0, lastused=157898,157898 ,157898) #TRIMSELECT: delete prop_pre_310 from trimlist (does not match trim needs) #TRIMSELECT: end #BBPTRIM: no more unload candidates! #BBPTRIM_EXIT: memsize=95140356,vmsize=794361856 !GDKmmap(442499072) fail => BBPtrim(ready) usage[mem=101297568,vm=794361856] # !mallinfo.arena = 15613828 !mallinfo.ordblks = 46134 !mallinfo.smblks = 15492 !mallinfo.hblkhd = 0 !mallinfo.hblks = 0 !mallinfo.usmblks = 13718408 !mallinfo.fsmblks = 899720 !mallinfo.uordblks = 950740 !mallinfo.fordblks = 44960 !ERROR: MT_mmap: MapViewOfFile(6b0, 2, 0, 0, 442499072, 0) failed !OS: Not enough space !ERROR: GDKload: cannot mmap(): name=05\552, ext=theap.priv !OS: Not enough space !ERROR: GDKload failed: name=05\552, ext=theap.priv !ERROR: shredder.mx:append_str2bat: APPEND-STR[PROP_TEXT](final foxes since the silent, quick realms should breach never sheaves--ruthless, daring waters beneat h the close asymptotes c), BUNappend fails !ERROR: CMDshred2bats: operation failed. MonetDB>

2 3

[Monetdb-developers] M5 inspect05 test
by Fabian Groffen 02 Aug '06

02 Aug '06

Devs, I added sabaoth today, but could not approve inspect05 with manual patching, because there seems to have changed more than just some sabaoth signatures added. This is seems to be so for some time. Whoever knows if those changes are correct (something with sortT) and commits inspect05: the sabaoth additions are correct AFAIT.

1 0