Greetings!
In the architectural overview, I find the text: "Most memory resource problems in MonetDB are easily avoidable by minimizing your open BAT references (as discussed above) and making your access patterns cache friendly (see the radix module)."
Is there anyplace that actually describes when a BAT is considered open? Is it a scope thing (BATs are closed as soon as they are out of scope)? Is a BAT open when I append/update? Does it close when I perform a commit? If not, is there a way to specify "I am done, clean it out"?
I am noticing a large amount of open memory (approximately 500MB out of a 1GB system) being used by MonetDB. I have hand partitioned my BATs into groups using a naming convention, but I am wondering if during inserts I am somehow keeping open references to the BATs which may cause issues.
Is there a way to find out where memory is being allocated? Which BATs are open?
Also, I am setting some of the larger BATs to STORE_MMAP. I notice in the documentation that if I compress the heaps, then these must be set to STORE_MEM. Is there a reason that when uncompressing the heap, that the uncompressed version cannot be STORE_MMAP to avoid memory problems?
Regards! Ed