[Monetdb-developers] Dealing with large tables

Agustin Schapira schapira at cs.umass.edu
Thu Jun 14 19:30:38 CEST 2007


Martin,


>> Once again, here I am, writing for help ;-)
>> We are working with a 'large' table, with 250 million rows, 11   
>> columns. This takes up a lot of memory space, as you can imagine:  
>> a  single column of 250 million 'ints' requires almost a gigabyte,  
>> and
> I would say 1GB is a minimum. What if you are on a 64bit machine?
> And also, if this table requires a hash index, then add another 1GB/ 
> table.

It is taking 1GB per BAT. It's running on a 64-bit machine, but Monet  
is compiled with 32-bit oids, and the column has 'ints', not 'words'...

> Hard to deduce what's happening.
>> col1.delete(var_0);
>> commit();
>> col2.delete(var_0);
>> commit();
>> col3.delete(var_0);
>> commit();
> I would first try to make sure if the commit
> is not sitting in the way.
> Do you need to commit here ?

No, the commit() is not necessary. In fact, the original code didn't  
have it; I just thought that maybe calling commit() would help Monet  
free up BATs that it didn't need anymore.


Thanks for your answer; I'll keep trying,

-- A




More information about the developers-list mailing list