MonetDB taking lot of storage space

Stefan Manegold stefan.manegold at cwi.nl
Sat Jun 17 13:01:17 CEST 2017


Typo in formula:does * columns * (average) width per columnmust readrows * columns * (average) width per column,
-------- Original message --------From: Stefan Manegold <stefan.manegold at cwi.nl> Date: 6/17/17  12:33  (GMT+01:00) To: Communication channel for MonetDB users <users-list at monetdb.org> Subject: Re: MonetDB taking lot of storage space 
Hi Amir,
May I assume "cr" stands for "crore" (c.f.., https://en.m.wikipedia.org/wiki/Crore), I.e. 1 cr = 10 million = 10^7 = 10,000,000 ?
The plain expected size of the database assuming only fixed-width column types and ignoring automatically generated indexes, etc.Can be assumed as does * columns * (average) width per column, e.g.,Assuming all 4 byte numerical columns:(4+5) * 10^7 * 10 * 4 byte = 36 * 10^8 ~= 3.6 GBor twice as much for all 8 byte columns.
Your 196 GB is far from that.
Are you sure there is no other left-over data in your database?As Jennie suggested, did you do a lot of update (insert, delete) operations on these tables?Which (top-level) directories in your dbfarm are the largest?
Best,Stefan

-------- Original message --------From: Ying Zhang <Y.Zhang at cwi.nl> Date: 6/17/17  10:35  (GMT+01:00) To: Communication channel for MonetDB users <users-list at monetdb.org> Subject: Re: MonetDB taking lot of storage space 
Hai, 

MonetDB doesn’t provide any user definable compressions.


What do you mean with “5cr rows”? Compressed Rows?

198GB is indeed pretty big, so it’s expected to contain a lot of table data.  If you don’t have variable length columns, it’s easy to compute the expected storage space: #rows * #columns * (sum of the length of each column).

Are your database fresh or a lot of insert and delete have been conducted on them?  Then, the deletes might have leaved garbages.

Did you happen to have put larges BLOBs or CLOBs in some columns?

Which MonetDB version and OS do you use?

Regards,
Jennie


> On 17 Jun 2017, at 09:38, Amit Ambekar <amit at g-square.in> wrote:
> 
> Hi,
> 
> i have 2 databases in monetdb. first one has 5cr rows and second has 4 cr rows with average 10 columns per table. It takes 196 GB storage space. is it normal ?? most of data consists of numeric values. And How i can compress database size in MonetDB ??
> 
> _______________________________________________
> users-list mailing list
> users-list at monetdb.org
> https://www.monetdb.org/mailman/listinfo/users-list

_______________________________________________
users-list mailing list
users-list at monetdb.org
https://www.monetdb.org/mailman/listinfo/users-list
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.monetdb.org/pipermail/users-list/attachments/20170617/0ac16923/attachment.html>


More information about the users-list mailing list