Hi Alexis,
On 20-07-2011 16:12:49 +0200, Alexis Berlemont wrote:
I am currently trying to evaluate MonetDB as a solution to hold significant amounts of data.
Because of limited storage capacities, I am trying to enable compression in MonetDB.
There is no compressed storage system inside MonetDB, other than "compression" at the value level (duplicates).
I did not manage to find any tutorial on that subject, so I did not go very far. I cannot even tell that I wholly understand the problem.
According to the page "Documentation->User Guide->compression", I understood that I had to inject a compression optimizer inside a pipeline. This "thing" is supposed to be possible via the setting of a sql environment variable named "optimizer".
Thanks, I've removed the confusing paragraph from that page.
In the (generated) source file monetdb5/optimizer/opt_pipes.c, I found a default + compression pipeline. Then, in a mclient session, I typed:
sql>set optimizer='inline,remap,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,constants,commonTerms,joinPath,deadcode,reduce,dataflow,compression,dataflow,history,multiplex,garbageCollector';
After that, I tried to use a sql function which was supposed to compress (thanks to the zlib) a table: sql.gzcompress
sql>call gzcompress(myschema, mytable);
Unfortunately, I am stuck here with the error message 'SELECT: identifier 'myschema' unknown'. I think I located the error message in "sql/server/rel_select.c" but I did not manage to deduce anything interesting.
Do you have any idea ?
You better push your luck with a compressed filesystem. I'm sorry to have put you on a false track here. Compression of the columns is not working.
Best regards, Fabian Groffen