[MonetDB-users] MonetDB size limitation questions.

Martin Kersten Martin.Kersten at cwi.nl
Mon Dec 17 22:08:09 CET 2007


Thomas Backlund wrote:
>
> Hi,
>
Hi Thomas.

Thanks for your interest in MonetDB.
>
> I stumbled over MonetDB a few days ago and it really cought my 
> interest. But it is hard to find information about it, so I hope You 
> could answer a few questions.
>
> I need a database engine that could handle many rows, about 350*10^9 
> rows of about 1000TB data.
>
This is really large, do you really mean 1 PetaByte?
For MonetDB the first interesting part is that it will automatically
split the table into 350 tables of 1000.000.000 rows, each would then be
be 3 TB.. which means you are using a special column type?
I think we need a snippet of the relation declaration to understand your 
schema

We are currently working on automatic horizontal fragmentation with 
distribution
to indeed handle them as single tables.

To understand if MonetDB is the solution to your problem, it is also 
relevant to
know what kind of queries you intend to run.

We know there are users moving MonetDB into this direction (sizewise)
and we would certainly be interested in learning the problems encountered.
Given, of course, you have the time to help us on this track and you know
of any database system that can already solve the problem out of the box.

regards, Martin Kersten
>
>
> Well I dont really need it right now, but I want to be prepared.
>
> My option up til now has been to use MySQL and to do some kind of 
> sharding using alot of databases.
> But that complicates stuff alot since I still want to do selects in 
> this gigantic table and that will be a bit of a problem doing selects 
> if the data is split up in 8000 different MySQL databases.
>
> Do You think MonetDB could handle this in one single table?
> Even if MonetDB could handle this I still will need a hard disk of 
> 1000TB, do You know of any technique merging many computers together 
> into one virtiualized hard drive that MonetDB would accept?
>
> Thank You very much!
>
> Best regards
> Thomas Backlund,
> Sweden.
>





More information about the users-list mailing list