On 2011-03-31 11:34, Sjoerd Mullender wrote:
On 2011-03-31 11:09, Xianmin Liu wrote:
Hi,
Sorry for missing the software information in last letter. I used MonetDB4-XQuery-Installer-x86_64-20110324 version, it seems to be the latest.
If you have read my last letter, pls ignore followings.
I am testing the performance of MonetDB/XQuery over large data. I use a HP xw8600 workstation, with Intel(R) Xeon(R) CPU X5482 3.20GHz, and 32GB RAM. I generate XMark data as large as 50GB, using standard XMark generator with 500 factor. When I tried to load it into database using "pf:add-doc", error message returned as following.
ERROR = !ERROR: [shred_url]: 1 times inserted nil due to errors at tuples 0@0. !ERROR: [shred_url]: first error was: !ERROR: strPut: string heaps gets larger than 15GB.
Here is the problem. In the past we made the decision to have 32-bit OIDs also on 64 bit binaries on Windows. On Linux, when the binaries are for 64 bits, OIDs are also 64 bits. Because of backward compatibility of databases, when a new release comes out, we never changed this decision.
The upshot is, on Windows you cannot have string columns that contain more than 15 GB of data. Also not if you have plenty of resources (such as memory).
The solution is to build a binary that uses 64 bit OIDs. This is as hard or easy as building a binary for 32 bit OIDs. This binary would then *not* be compatible with versions using 32 bit OIDs.
What I can do for you is try to build such an installer based on the Mar2011 release.
This version is now available at http://dev.monetdb.org/downloads/Windows/Mar2011/.
!ERROR: shredBAT_append_str: APPEND-STR[_prop_text]( !ERROR: catch fain deserves liking knee katharine easy handhelms daughter corn hire amaz plainness curtains staying excuse since tom walls thirty goodman twice corrections flesh behold woundl ess cargo royally zwagger nuncle season repair govern piercing sighs ounce arrest ganymede balthasar perjury !ERROR: ), BUNappend fails !ERROR: CMDshred_url: operation failed. Timer 2633290.282 msec
Can you help me? How can I fix it? Thanks a lot.
-- Xianmin Liu