January 2010 - users-list

[MonetDB-users] compression
by Jason Kinzer 20 Jan '10

20 Jan '10

Hi, Are there any plans on the horizon to roll compression into Monet? The X-100 project looked really interesting in this regard, but as I understand it, that work has been transferred into VectorWise. If there are no plans, is this because it's completely antithetical to the monet architecture (from the papers it seems like X-100 was, to some degree at least, 'integrated' in), or more due to lack of resources? My motivating example here is OLAP: I frequently have 1 relatively large fact table and then many much smaller dimensional tables. If optional compression were available, it we be nice to compress all or some of the BATs for the fact table columns and then have the others work as usual. (Well, at least this sounds good, maybe it makes no sense). Another motivation is there seems to be a lot of anecdotal evidence for companies moving from larger big iron servers to more numerous, smaller machines - so it would be really nice to have this capability for more memory constrained settings. I understand on a basic level how compression conflicts with the relatively simple approach monet uses to load BATs (e.g. memory map), but, dwelling in ignorance, I blithely assume there could be some solution not as complex as X-100 if one were to accept a significant performance cost. For example: decompressing BAT data on the fly as part of a BATiterator. I probably don't have the skills to implement even a basic on-the-fly decompression approach like this, but just wondering aloud: how hard a problem is this? Thanks, Jason

2 3

[MonetDB-users] manually setting gdk_nr_threads to N?
by Nafees Ur Rehman 19 Jan '10

19 Jan '10

Hi, I am using Windows 2003 with two core processors configures, and the gdk_nr_threads does not show the 2nd processor. Can you tell me how can I set that manually? Regards Nafees Ur Rehman

2 1

[MonetDB-users] Standard Deviation tips and hints request
by Dennis Forbes 17 Jan '10

17 Jan '10

Hi all. While it's possible to calculate the stddev with an aggregated cross join, is there perhaps a better way? Any tips or hints on where I should look? Can this be done in MIL or some other tactic to provide a more straightforward, better performing experience? I searched the web before asking and all I've found are unanswered questions, however I would really like to solve this even if it includes extending the system. Dennis W. Forbes Vastardis Capital Services 41 Madison Avenue, 30th Floor New York, NY 10010 P: 646-230-0933 F: 212-243-5251

2 2

[MonetDB-users] mapi_query with MAL code?
by Guillaume Yziquel 17 Jan '10

17 Jan '10

Hello. I've been using an embedded SQL with moderate success with my OCaml binding. Essentially passing the SQL statement through the mapi_query() function call. I would now like to try out some MAL code in a similar setting: embedded MonetDB, code passed via mapi_query(). Could someone provide me with a simple "Hello world" MAL code to try? For now, all I've been able to achieve with MAL are segfaults... All the best, -- Guillaume Yziquel http://yziquel.homelinux.org/

2 1

[MonetDB-users] native socket protocol vs mapi
by Andy Chambers 16 Jan '10

16 Jan '10

Hi, I want to be able to use monetdb from common lisp and was looking at the other language bindings for clues on how to write one for cl. Python and Ruby seem to communicate using a socket rather than use the C api. However, in common lisp it's relatively easy to use C bindings (though CFFI) so I'd rather use them if I thought they were going to continue to be supported. Are there any other factors I should consider in deciding which API to use? Many Thanks, Andy

4 9

[MonetDB-users] Star Schema Benchmark: Can't load SF 5 ou SELECT, LINE_READ Crash
by Rui Anastácio 12 Jan '10

12 Jan '10

Hello, I Have Ubunto 9.10 Karmik Koala installed. On top of this I have installed MonetDB latest version (same problem with previous one). I'm working with Star Schema Benchmark (SSBM). For Scale Factor 1 and 3 I can load the data. For SF 5 our above I get READ_LINE error when loading LINEORDER table. Also when running select Q4.1 I also get the same error and the server crashes. Anyone as a clue? Thanks Rui A.

2 3

Re: [MonetDB-users] monetdb_sql/mal() ?
by Guillaume Yziquel 12 Jan '10

12 Jan '10

Jason Kinzer a écrit : >> >> Besides, It would seem to overlap with Mauricio's work which already >> provides some parsing from his OCaml algebra to SQL. But the thing is, >> you've got the SQL string created at OCaml compile time. I may be wrong about this compile time assertion. It now seems to me that this is runtime. In which case, it would make sense to map Mauricio's relational algebra to MAL's relational algebra. >> What would be nice >> is to have the SQL -> MAL compilation also happen at the same time you >> compile the OCaml program. >> >> (Moreover, it seems to me that MAL comes in two flavour: a human-readable >> one and a machine readable one. Is that correct? If yes, I'm aiming at the >> machine readable one.) >> >> How easy would that be? Do you need a running embedded server for that, or >> simply a subset of the functionalities of the MonetDB shared libraries? > Ah, I see now, so you're talking about actually writing a SQL->MAL compiler > as part of that existing Ocaml SQL frontend (sort of cut out the > "middle-man"). Yes. > That sounds neat, but bear in mind that could be tantamount > to re-writing the existing Monet SQL frontend, depending on your goals In fact, I would like to do an OCaml/C binding to the SQL -> MAL compilation that is already done in MonetDB. This seems to me the least effort path. But I'd like to know where in the code is this compilation / conversion from SQL to MAL done. > - so I let you draw your own conclusions. Try doing an "explain" on various SQL > statements to see the generated MAL. Sorry, I'm quite new to MonetDB and still having a hard time with the documentation. What is this "explain" thing I should be doing? > I would say writing a very naive, > minimal SQL translation is not easy but certainly within raeach. Here > someone other than I should comment, but I think most of the complexity of > the existing frontend lies in the optimization passes, tx handling, > approaches to handle datasets that won't fit in memory, etc. Rewriting an SQL -> MAL translation is not my goal. Reusing an existing one would be one my goals. All the best, -- Guillaume Yziquel http://yziquel.homelinux.org/

2 1

[MonetDB-users] monetdb_sql/mal() ?
by Guillaume Yziquel 11 Jan '10

11 Jan '10

Hello. I've been looking at Debian packages, and the followig things struck me: -1- You have two shared libraries: libembeddedsql.so and libembeddedmal.so which are distributed, but there's only headers for the *sql.so one. You have to write your own for the *mal.so one. -2- In the source, there are two libembeddedclient.c.in files: one for sql, one for mal. The one for sql contains a monetdb_sql() function call. The one for mal does not. Is there a good reason for this? Or is it simply the way it currently is? All the best, -- Guillaume Yziquel http://yziquel.homelinux.org/

2 4

[MonetDB-users] problems with the interpretation of the query history
by Gerrard_56 10 Jan '10

10 Jan '10

Hi. I’m a new MonetDB user and I use the MonetDB server v5.14.2 with the MonetDB/SQL module v2.32.2 on a windows xp platform. I have some problems with the interpretation of the queryhistory and the callhistory. My first question is: What is the meaning of the parse and the optimize attributes in the queryhistory. Is the optimize attribute used to store the time which the front-end needs to parse the sql statement into a mal plan respectively is the optimize attribute used to store the time used for the optimization done for the mal plan. I’m asking because couldn’t find an explanation in the sql manual. My next questions concern the xtime and the rtime attributes of the callhistory. Sometimes for some queries the xtime is 0 usec. Does this indicate that the query was already executed once before and therefore there isn’t a preparation time needed for the result set? Sometimes for some queries only the rtime is 0 usec but the the xtime is > 0 usec. What does this means? There is a preparation time for a result set but there is no time needed to construct and return the result set to the user. For me this makes no sense. Sometimes there is the xtime and the rtime 0 usec. Does this mean that the query never runs in the database engine? Does somebody know the reason for this behavior? Lust but not least in the sql manual you can read that all storage is in K/M/G bytes. But where can I find out if K bytes M bytes or G bytes are used? Thank you for any help Gerrard (I apologize for any grammar and spelling mistakes in this posting but my English is not the best.) -- View this message in context: http://old.nabble.com/problems-with-the-interpretation-of-the-query-history… Sent from the monetdb-users mailing list archive at Nabble.com.

2 2

[MonetDB-users] sql execution time
by Gerrard_56 09 Jan '10

09 Jan '10

Hi. I would like to compare the performance of MonetDB with the performance of an Oracle DB by using the TPC-H benchmark. I use the MonetDB server v5.14.2 with the MonetDB/SQL module v2.32.2 on a windows xp platform. On the MonetDB website (http://monetdb.cwi.nl/SQL/Benchmark/TPCH/) I read that they used the mclient with the –t argument in order to obtain the timings for the queries. I tried the same but the displayed time for queries which return several 100 rows is very high. For example the TPC-H query nr. 3 with returns 11620 row the time was 13796.174 msec. If I execute the same query in the db visualize it just needs 0.864 sec for the result. I think the problem with the mclient is that it needs a lot of time to display the result of the queries. Is there a better alternative to the –t argument of the mclient to get a useful time result? Thank you for any help Gerrard -- View this message in context: http://old.nabble.com/sql-execution-time-tp27088775p27088775.html Sent from the monetdb-users mailing list archive at Nabble.com.

1 0