About compilation time

Stefan Manegold Stefan.Manegold at cwi.nl
Tue Dec 8 18:30:17 CET 2015

Dear Mustafa, et al.,

here some related info than you might find helpful:

The MAL code given with EXPLAIN is the output of the second tier,
i.e., the fully (statically) optimized MAL plan, ready for execution.

Without diving deep into the MonetDB source code,
possible ways to indirectly get an indication of the pure query translation
cost (excluding execution) are:

- simple, but not very "accurate":
  run the query on empty tables; if the plan is small, MAL interpretation should be small and negligible.
  Caveat: the plan on empty tables might (will) be different than the plan on full tables,
  and thus also the translation time *might* differ.

- better (and more accurate?):
  run the query via mclient with -i option for timing, and prefixed with TRACE for profiling;
  the difference between the total execution time reported by mclient's -i option and
  the total execution time reported in the TRACE profile is a reasonable indication for the
  query translation time (basically, the TRACE reports the pure execution time of the MAL plan,
  while mclient's -i reports end-to-end wall-clock.

A caveat for both options is that MonetDB tries to cache (parameterized) MAL plans to eliminate
query translation time with repeatedly executed queries.
To exclude the potential effects of potential query plan caching, you might want to restart
the server before each query execution.

In any case, any of such indirect measurements are to be taken with a grain of salt ...

Hope this helps you further.


----- On Dec 3, 2015, at 11:50 PM, Mustafa Korkmaz t-mukork at microsoft.com wrote:

> Thank you Jannie,
> By using explore I can see the mat version of the query.
> In software stack page,
> https://www.monetdb.org/Documentation/Manuals/MonetDB/Architecture/SoftwareStack
> it is written that
> Outputs of both the first and the second layer are MAL expressions. So do you
> have any idea about which layer does the EXPLORE command MAL output belongs to?
> I am especially interested in the process between query submission and actual
> execution. So can you suggest me a starting point in the source code to explore
> more?
> Thank you,
> Mustafa Korkmaz
> -----Original Message-----
> From: developers-list
> [mailto:developers-list-bounces+t-mukork=microsoft.com at monetdb.org] On Behalf
> Of Ying Zhang
> Sent: Wednesday, December 2, 2015 7:47 AM
> To: Communication channel for developers of the MonetDB suite.
> <developers-list at monetdb.org>
> Subject: Re: About compilation time
> Hai Mustafa,
> I’m not aware that monetdb directly gives you this information.  what you could
> try is to prefix your query with PLAN or EXPLAIN, which will limit a monetdb
> server to only generate the logical or physical execution plan for the query.
> Do this with client and server on the same machine, send the output to
> /dev/null, then you can measure the wall clock time.  This should give you
> something usable.
> For only the parsing time, I’m afraid you need to dig into the source code to
> add timing…
> Not sure what parsing/compilation details you’re looking for, but you can have a
> look at the output of PLAN and EXPLAIN anyway.
> Best,
> Jennie
>> On Dec 01, 2015, at 20:29 , Mustafa Korkmaz <t-mukork at microsoft.com> wrote:
>> Hello all,
>> I’m conducting an experiment to show pros/cons of certain DBs and I
>> need to investigate query compilation performance in details. For
>> this, Is there any way to measure the elapsed time in query parsing/compilation
>> time in MonetDB?
>> Also is there any document out there to show the query parsing/compilation
>> details?
>> What I find for timing is only “-i” flag which gives an overall timing stat.
>> I’d be glad for any help.
>> Thank you
>> Mustafa Korkmaz
>> _______________________________________________
>> developers-list mailing list
>> developers-list at monetdb.org
>> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fwww.m
>> onetdb.org%2fmailman%2flistinfo%2fdevelopers-list&data=01%7c01%7ct-muk
>> ork%40microsoft.com%7c959abb246ef04a3355c508d2fb2fd7b6%7c72f988bf86f14
>> 1af91ab2d7cd011db47%7c1&sdata=5sBbGDG7c74uNR18oJ1BrInBQQpTXQ7CDIJMTdwF
>> ah0%3d
> _______________________________________________
> developers-list mailing list
> developers-list at monetdb.org
> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fwww.monetdb.org%2fmailman%2flistinfo%2fdevelopers-list%0a&data=01%7c01%7ct-mukork%40microsoft.com%7c959abb246ef04a3355c508d2fb2fd7b6%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=ofgvw1tY7Xm9t3I8gcMpzz38zr7mqsg6kCJ2ym2mOxk%3d
> _______________________________________________
> developers-list mailing list
> developers-list at monetdb.org
> https://www.monetdb.org/mailman/listinfo/developers-list

| Stefan.Manegold at CWI.nl | DB Architectures   (DA) |
| www.CWI.nl/~manegold/  | Science Park 123 (L321) |
| +31 (0)20 592-4212     | 1098 XG Amsterdam  (NL) |

More information about the developers-list mailing list