MonetDB Integration Question

Stefan Manegold Stefan.Manegold at cwi.nl
Mon Nov 13 20:00:59 CET 2017


Hi David & Florian,

please find two quick examples/answers below.

Best,
Stefan

----- On Nov 6, 2017, at 10:11 PM, David Broneske david.broneske at ovgu.de wrote:

> Dear developers of MonetDB,
> 
> I hope you are the right recipients of this mail that can help us in getting
> some more insights into MonetDB’s Code.
> Currently, me and my student (Florian) want to integrate our index structure Elf
> ( www.elf.ovgu.de ) into MonetDB as a proof of concept and to show its
> competitiveness to fast main-memory scans. However, we’ve got a hard time to
> understand some part of the code — we hope that you could help us with this.
> While the kernel (gdk) is well documented, we hardly find any documentation that
> may help us in answering the following questions:
> 
> 
> 
>    * Which function takes care of filling the BATs when inserting into a table
>    (i.e., sql_table/sql_rel)?

sql>create table t (a int, b int);
operation successful (8.722ms)
sql>explain insert into t values (1,2);
+-------------------------------------------------------------------------------------------------+
| mal                                                                                             |
+=================================================================================================+
| function user.s6_1():void;                                                                      |
|     X_3:void := querylog.define("explain insert into t values (1,2);", "default_pipe", 12:int); |
|     X_6 := sql.mvc();                                                                           |
|     X_12 := sql.append(X_6, "sys", "t", "a", 1:int);                                            |
|     X_17 := sql.append(X_12, "sys", "t", "b", 2:int);                                           |
|     sql.affectedRows(X_17, 1:lng);                                                              |
| end user.s6_1;                                                                                  |
| # optimizer.mitosis()                                                                           |
| # optimizer.dataflow()                                                                          |
| #inline               actions= 0 time=1 usec                                                    |
| #remap                actions= 0 time=3 usec                                                    |
| #costmodel            actions= 1 time=2 usec                                                    |
| #coercion             actions= 1 time=5 usec                                                    |
| #evaluate             actions= 4 time=60 usec                                                   |
| #emptybind            actions= 0 time=0 usec                                                    |
| #pushselect           actions= 0 time=7 usec                                                    |
| #aliases              actions= 4 time=8 usec                                                    |
| #mergetable           actions= 0 time=35 usec                                                   |
| #deadcode             actions= 0 time=7 usec                                                    |
| #aliases              actions= 0 time=0 usec                                                    |
| #constants            actions= 2 time=6 usec                                                    |
| #commonTerms          actions= 0 time=5 usec                                                    |
| #projectionpath       actions= 0 time=4 usec                                                    |
| #deadcode             actions= 0 time=4 usec                                                    |
| #reorder              actions= 1 time=26 usec                                                   |
| #matpack              actions= 0 time=1 usec                                                    |
| #multiplex            actions= 0 time=2 usec                                                    |
| #profiler             actions=1 time=1 usec                                                     |
| #candidates           actions=1 time=1 usec                                                     |
| #garbagecollector     actions= 1 time=64 usec                                                   |
| #total                actions=23 time=397 usec                                                  |
+-------------------------------------------------------------------------------------------------+
31 tuples (1.515ms)


>    * COPY INTO creates a MAL-procedure copy_into, which is preparing the copy, but
>    does not finally execute the copy. Which function is executed for this?

sql>explain copy 1 records into t from stdin;
+--------------------------------------------------------------------------------------------------------------------------------------------+
| mal                                                                                                                                        |
+============================================================================================================================================+
| function user.s10_1():void;                                                                                                                |
|     X_1:void := querylog.define("explain copy 1 records into t from stdin;", "default_pipe", 20:int);                                      |
|     X_4 := sql.mvc();                                                                                                                      |
|     (X_24:bat[:int], X_25:bat[:int]) := sql.copy_from(nil:ptr, "|", "\\n", nil:str, "null", nil:str, 1:lng, 0:lng, 0:int, 0:int, nil:str); |
| barrier X_80 := language.dataflow();                                                                                                       |
|     X_27 := sql.append(X_4, "sys", "t", "a", X_24);                                                                                        |
|     X_32 := sql.append(X_27, "sys", "t", "b", X_25);                                                                                       |
|     X_34 := aggr.count(X_25);                                                                                                              |
| exit X_80;                                                                                                                                 |
|     sql.affectedRows(X_32, X_34);                                                                                                          |
| end user.s10_1;                                                                                                                            |
| #inline               actions= 0 time=1 usec                                                                                               |
| #remap                actions= 0 time=3 usec                                                                                               |
| #costmodel            actions= 1 time=2 usec                                                                                               |
| #coercion             actions=11 time=13 usec                                                                                              |
| #evaluate             actions= 0 time=4 usec                                                                                               |
| #emptybind            actions= 0 time=0 usec                                                                                               |
| #pushselect           actions= 0 time=6 usec                                                                                               |
| #aliases              actions= 0 time=0 usec                                                                                               |
| #mergetable           actions= 0 time=37 usec                                                                                              |
| #deadcode             actions= 0 time=7 usec                                                                                               |
| #aliases              actions= 0 time=0 usec                                                                                               |
| #constants            actions= 0 time=8 usec                                                                                               |
| #commonTerms          actions= 0 time=6 usec                                                                                               |
| #projectionpath       actions= 0 time=3 usec                                                                                               |
| #deadcode             actions= 0 time=6 usec                                                                                               |
| #reorder              actions= 1 time=31 usec                                                                                              |
| #matpack              actions= 0 time=1 usec                                                                                               |
| #dataflow             actions= 1 time=23 usec                                                                                              |
| #multiplex            actions= 0 time=3 usec                                                                                               |
| #profiler             actions=1 time=1 usec                                                                                                |
| #candidates           actions=1 time=1 usec                                                                                                |
| #garbagecollector     actions= 1 time=80 usec                                                                                              |
| #total                actions=25 time=404 usec                                                                                             |
+--------------------------------------------------------------------------------------------------------------------------------------------+
34 tuples (1.455ms)

I.e., in both cases MAL command sql.append() --- respectively it's C implementation --- takes care of inserting the data into the final BATs.

Does that answer your questions?

>    * Regarding ChangeSets: what is dset and nelm used for?

I cannot answer this one, as I don't even know what/where "ChangeSets", "dset", and "nelm" are ...

Best,
Stefan

> Any pointer or explanation for these questions would be great!
> 
> Regards,
> David and Florian
> M.Sc. David Broneske
> Otto-von-Guericke-Universität Magdeburg
> Department of Technical and Business Information Systems
> Databases and Software Engineering
> 
> P.O. Box 4120, D-39016 Magdeburg, Germany
> Office: Building 29, Room112
> Tel: +49 391 67-52994 Fax: +49 391 67-12020
> 
> 
> _______________________________________________
> developers-list mailing list
> developers-list at monetdb.org
> https://www.monetdb.org/mailman/listinfo/developers-list

-- 
| Stefan.Manegold at CWI.nl | DB Architectures   (DA) |
| www.CWI.nl/~manegold/  | Science Park 123 (L321) |
| +31 (0)20 592-4212     | 1098 XG Amsterdam  (NL) |


More information about the developers-list mailing list