[Monetdb-developers] Re: X100

Peter Boncz P.Boncz at cwi.nl
Sat Jul 30 20:39:26 CEST 2005


Hi Joel,

About X100. In my opinion this is a very cool project that actually
constitutes an entirely new query engine. It is more scalable than the current
Monet4 engine, and also than the Monet5 engine. There is no main memory
limitation anymore. Let's think about it as Monet6 (though that is not sure). 
The system is quite functional, but especially regarding APIs, database
updates and also binary/source distribution of the code, a lot has to be done.
Currently, we are concentrating on obtaining scientific results. Some
functioanlities (like database updates) still have to be developed. I think we
are more than a year off from a releasable system. Also, it has not been
decided yet whether that will be an open-source release. Maybe X100 has some
commercial future..

Then more general. I just read your posts on this mailing list and I do not
have a full grasp of your plans in order to properly advise you.

Some general tips:
- try to keep all bulk operations in MIL, with interfaces to the lisp engine
that move only little data in and out.
- use unary bats preferable. i/e/ those that have a void head type
- try to seperate data and metadata. Thus, the schema you use, and also the
results, are best kept outside MonetDB. Maybe store them in flat files XML or
something. It makes the architecture easier to maintain. Also, your
application does not depend that strongly on MonetDB repositories.

As for special purpose indexing, often it is possible to represent them as a
set of bats. you will have to do some coding in a new monet extension module
(a .mx file as found in the src/modules/plain/ directory) to write some
primitives that access those bats (that are your index) efficiently.

just my 2 cts.

Peter




More information about the developers-list mailing list