[Monetdb-developers] MAPI protocol suggestion

Martin Kersten Martin.Kersten at cwi.nl
Sat Nov 22 09:19:33 CET 2008


The question about binary versions of MAPI has recently been
raised and a test was performed based on sending a large sequence
of small SQL queries involving integers and strings.

On 21-10-2008 15:47:02 +0200, Fabian Groffen wrote:

> > 100000 "select 1" using 1 connection over standard client utility:
> > psql:    9.2s (2.8 user, 1.1 sys)
> > mysql:   5.4s (1.1 user, 0.9 sys)
> > monetdb: 7.6s (2.1 user, 1.3 sys)
> > 
> > more to follow.

10000 "select 'hello world'"

psql:    9.2s (2.9 user, 1.2 sys)
mysql:   5.4s (1.2 user, 0.8 sys)
monetdb: 7.9s (2.1 user, 1.2 sys)

A Valgrind analysis of MonetDB showed that the
cost for the protocol is around 10% of this task.
A little gain in parsing might be expected when
the protocol carries only numerics and from the
formatting/parsing overhead of tuples being
transmitted (result-sets are mostly small)

Consequently, moving to a binary protocol is not
warranted by comparison, nor effective in terms
of performance improvement. 50% of the cost in
MonetDB for these queries are the SQL parser and
query cache matching.

Given the focus of MonetDB on datawarehousing,
rather then high volume web-interactions, there
has been no steps taken to improve the protocol.

There are other major gains the made.

Stefan de Konink wrote:
> Hi,
> With light on my recent philosophical question on doubles/uints for the 
> lossy storage on coordinates. I wonder the following;
> Would it be possible to implement a binary MAPI protocol that can use 
> the native types of the data that is stored within MonetDB?
> So for example; I create a table:
> CREATE TABLE helloword (hello varchar(13), world integer);
> INSERT INTO helloword (hello, world) VALUES ('hoi', 1);
> And fetch this table:
> SELECT hello, world FROM helloworld;
> And receive the MAPI data in the following (too simplistic) way:
>    (len) (string)
> --> 5 hello 5 world
> --> 3 hoi 4 0x0 0x0 0x0 0x1
> For several reasons the above will not work; the length field is too 
> short (only up to 255) it is also not optimal. I wonder if it can 
> trigger any idea to allow the end user to use native types instead of 
> char[]'s.
> Stefan
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Monetdb-developers mailing list
> Monetdb-developers at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/monetdb-developers

More information about the developers-list mailing list