In fact, I've wrapped monetDB using JENA's api. And tried to load DBPedia's RDF data with it. It runs OK on small dataset. But it always requires memory during the insertion. So it failed when there's no memory to be allocated. Since it has been wrapped by JENA, I know few of the background schema of the database. I go with you it may caused by the os's vm strategy. I'll check it. I've seen that there are some debug option for the config file. Can I use some of them to log something that can let you know my problem? Which one should I use ?
On Wed, Apr 30, 2008 at 2:19 PM, Martin Kersten Martin.Kersten@cwi.nl wrote:
Kang Zhang wrote:
The data's size is about 3 Gigabytes. 21,852,641 triples need to be inserted in total. My machine is 64-bit and only 8G memory for that.
Can you say a little more when and how it crashed. Did you use the SQL COPY command? Is it a victim process selected by the OS, due to excessive virtual memory use? Do you have an gdb stack trace where it happen?
Thank you for your immediate reply. Kang.
On Wed, Apr 30, 2008 at 1:21 PM, Martin Kersten <Martin.Kersten@cwi.nl mailto:Martin.Kersten@cwi.nl> wrote:
Kang Zhang wrote: > I am using the latest version of monetDB from the web-site(thesuper
> source tarball). > My machine is a X86-64 machine with a red-hat installed on it. When I > insert data into monetDB, > it always requires memory from the system. Then it will be crashed > with running out of memory. > Is there a way to set the limit for memory used by monetDB ? I've read > the docs on the web site > and can't get what I need from that. Dear Kang, indeed, there is no way to limit the memory use. All columns are memory mapped files, which give an upperbound on their size. This is particularly problematic for 32-bits machines and very large columns. > > Thank you. > > -- > ------------------------------------------------------------ > Kang Zhang > > Computer Science Dept. > Shanghai Jiao Tong University >
> >
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference > Don't miss this year's exciting event. There's still time to save $100. > Use priority code J8TL2D2. >http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javao...
>
> > _______________________________________________ > MonetDB-users mailing list > MonetDB-users@lists.sourceforge.net <mailto:MonetDB-users@lists.sourceforge.net> > https://lists.sourceforge.net/lists/listinfo/monetdb-users >
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2.http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javao...
_______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net <mailto:MonetDB-users@lists.sourceforge.net> https://lists.sourceforge.net/lists/listinfo/monetdb-users--
Kang Zhang
Computer Science Dept. Shanghai Jiao Tong University
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javao...
MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javao... _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users