Veneta Dobreva wrote:
Hello,
I have the following question about MonetDB: does MonetDB implement some data compression algorithms? I would like to use an RLE (run-length encoding) algorithm for compression of my data. Is this already implemented in MonetDB?
Hello Veneta,
Thanks for your interest in MonetDB.
In the very old version we had a method to (de-)compress bats when they were loaded into memory. Nowadays, most BATs are memory mapped without any decompression.
This does not preclude an activity to re-activate and generalize the concept. But, full integration with the kernel will be hard. The easiest way is to define a toCompress and fromCompress functions that are explicitly called to read and convert the data.
Light-weight compression in the MonetDB project has been studied in the X100 project, whose code base has (and will not) be integrated in the main stream
regards, Martin
Thank you in advace for your help!
Regards, Veneta Dobreva