[Monetdb-developers] Dealing with large tables
schapira at cs.umass.edu
Thu Jun 14 19:30:38 CEST 2007
>> Once again, here I am, writing for help ;-)
>> We are working with a 'large' table, with 250 million rows, 11
>> columns. This takes up a lot of memory space, as you can imagine:
>> a single column of 250 million 'ints' requires almost a gigabyte,
> I would say 1GB is a minimum. What if you are on a 64bit machine?
> And also, if this table requires a hash index, then add another 1GB/
It is taking 1GB per BAT. It's running on a 64-bit machine, but Monet
is compiled with 32-bit oids, and the column has 'ints', not 'words'...
> Hard to deduce what's happening.
> I would first try to make sure if the commit
> is not sitting in the way.
> Do you need to commit here ?
No, the commit() is not necessary. In fact, the original code didn't
have it; I just thought that maybe calling commit() would help Monet
free up BATs that it didn't need anymore.
Thanks for your answer; I'll keep trying,
More information about the developers-list