[Monetdb-developers] [MonetDB-users] Large XML instances with MonetDB/XQuery 4.38.5
Stefan.Manegold at cwi.nl
Fri Oct 1 23:51:08 CEST 2010
quick and simple answer, on a 32-bit Windows system, you are running out of
address space, which is in practice limited to 2GB or at most 3GB if you
configured your system accordingly (cf.,
), and know to be very vulnarable to address space fragmentation.
I haven't tried it recently, but you might be able to load the 1 GB XMark
document on a 32-bit Unix-like system.
A 64-bit build of MonetDB/XQuery (on a 64-bit system) should not have any
problems loading a 1 GB XMark document.
On Fri, Oct 01, 2010 at 07:13:53PM +0200, Christian Grün wrote:
> Hi everyone,
> is it still possible to shred large XML instances with MonetDB/XQuery,
> or do I need more memory for shredding? I was just trying to the 1GB
> instance of XMark with WinXP 32Bit and 2GB RAM, and this is what I
> MonetDB>shred_doc("1gb.xml", "xmark");
> #GDKmmap(536870912) fails, try to free up space [memory in use=36143780,virtual
> memory in use=1015480320]
> #GDKmmap(536870912) result [mem=26016940,vm=1005387776]
> !ERROR: [shred_url]: 1 times inserted nil due to errors at tuples 0 at 0.
> !ERROR: [shred_url]: first error was:
> !ERROR: HEAPextend: failed to extend to 536870912 for 06\632theap
> !ERROR: shredBAT_append_str: APPEND-STR[_prop_text]( follow murders prov shoes p
> resent eager delivered defy bend viol here marquis powers complexion pillar cran
> ts agate ), BUNappend fails
> !ERROR: CMDshred_url: operation failed.
> pf:add-doc(...) produces the same output; the XML document itself is
> Start uncovering the many advantages of virtual appliances
> and start using them to simplify application deployment and
> accelerate your shift to cloud computing.
> MonetDB-users mailing list
> MonetDB-users at lists.sourceforge.net
| Dr. Stefan Manegold | mailto:Stefan.Manegold at cwi.nl |
| CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ |
| 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 |
| The Netherlands | Fax : +31 (20) 592-4199 |
More information about the developers-list