Re: [MonetDB-users] [Monetdb-developers] dealing with 1G XML file in MonetDB/XQuery
Mo Zhou wrote:
Hello,
Could you please reply as soon as possible?
From your email it is unclear what platform you are using. I guess it is a Windows 32bit version. The document you use may easily consume more memory when processed, exceeding the 32-bit addressing scheme. A 64-bit version is advised. (See earlier threads on this topic)
regards, Martin
Thanks, Mo On Thu, Apr 1, 2010 at 4:49 PM, Mo Zhou mozhou@umail.iu.edu wrote:
Hi,
When I load a 1G XML file to MonetDB/XQuery, it throws exceptions as follows. I did not find related parameters in the monetDB config file to enlarge the heap size. I wonder how I can solve this problem? Thank you.
#GDKmmap(1073741824) fails, try to free up space [memory in use=156511940,virtual memory in use=1309212672] #GDKmmap(1073741824) result [mem=156511940,vm=1309212672] MAPI = mozhou@localhost:40000 QUERY = pf:add-doc("/.../new1GL1Comp/xmldoc3.xml","1GL1doc3") ERROR = !ERROR: [shred_url]: 1 times inserted nil due to errors at tuples 0@0. !ERROR: [shred_url]: first error was: !ERROR: HEAPextend: failed to extend to 1073741824 for 03/02/30243theap !ERROR: shredBAT_append_str: APPEND-STR[_prop_text](http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/instances/dataFromVendor20/Of...), BUNappend fails !ERROR: CMDshred_url: operation failed.
-- Thanks, Mo
I really appreciate your help. Many thanks for your time and information.
Thanks mo
On Fri, Apr 2, 2010 at 10:20 AM, Martin Kersten Martin.Kersten@cwi.nl wrote:
Mo Zhou wrote:
Hello,
Could you please reply as soon as possible?
From your email it is unclear what platform you are using. I guess it is a Windows 32bit version. The document you use may easily consume more memory when processed, exceeding the 32-bit addressing scheme. A 64-bit version is advised. (See earlier threads on this topic)
regards, Martin
Thanks, Mo On Thu, Apr 1, 2010 at 4:49 PM, Mo Zhou mozhou@umail.iu.edu wrote:
Hi,
When I load a 1G XML file to MonetDB/XQuery, it throws exceptions as follows. I did not find related parameters in the monetDB config file to enlarge the heap size. I wonder how I can solve this problem? Thank you.
#GDKmmap(1073741824) fails, try to free up space [memory in use=156511940,virtual memory in use=1309212672] #GDKmmap(1073741824) result [mem=156511940,vm=1309212672] MAPI = mozhou@localhost:40000 QUERY = pf:add-doc("/.../new1GL1Comp/xmldoc3.xml","1GL1doc3") ERROR = !ERROR: [shred_url]: 1 times inserted nil due to errors at tuples 0@0. !ERROR: [shred_url]: first error was: !ERROR: HEAPextend: failed to extend to 1073741824 for 03/02/30243theap !ERROR: shredBAT_append_str:
APPEND-STR[_prop_text](http://www4.wiwiss.fu-berlin.de/bizer/bsbm/v01/instances/dataFromVendor20/Of...), BUNappend fails !ERROR: CMDshred_url: operation failed.
-- Thanks, Mo
participants (2)
-
Martin Kersten -
Mo Zhou