Hi,

I haven't understand it completely, does it mean that it will always use a single thread while doing Group by.

Also I have noticed one behaviour as below --

1) Very Fast Execution --- 

Select product_type,sum(ord_qty)  FROM bookings where ord_entry_date between '2014-04-01' and  '2014-10-01'  group by product_type;

2) Suddenly very slow execution , the moment I add another AGGREGATE function something like below --

select product_type,sum(ord_qty) ,COUNT(DISTINCT sales_document) FROM bookings where ord_entry_date between '2014-04-01' and  '2014-10-01'  group by product_type;  

So with 2 AGGREGATE functions it behaves very slow as compared to 1 AGGREGATE function.

Also even with 2 AGGREGATE functions example above , if i add more dimentions and group by 2-3 columns then there is a slight improvement in performance even though we are adding more GROUP BYs

Is this behaviour is expected, does anybody else have also experienced such behaviour.

Can it be because of OCT2014-SP1 release ??

Thanks

On Sun, Jan 18, 2015 at 11:20 AM, Martin Kersten <martin@monetdb.org> wrote:

Hi

Each MAL instruction is executed by a single thread.
Some operators require the complete input to be available,
such as in your case the group.subgroup().

martin

On 18/01/15 01:27, shamsul hassan wrote:

Hi All,

I have recently install monetdb "OCT 2014-SP1" release on my Oracle Enterprise 6 Linux server.

Server configuration -

8 CPU Cores
64 GB RAM

I have loaded a data set with 70 columns having 15 Millions Rows.

I have started mserver5 as below --

mserver5 --debug=10 --set *gdk_nr_threads=8* --dbpath=/data/monetdb/testdb  --set mapi_open=true --set mapi_port=54321  --set merovingian_uri=mapi:monetdb://localhost:54321/testdb --set mapi_usock=/data/monetdb/testdb/.mapi.sock --set
monet_vault_key=/data/monetdb/testdb/.vaultkey  --set sql_optimizer=default_pipe  --set max_clients=64

Now I have ran a query as below --

select product_type,COUNT(DISTINCT sales_document),sum(ord_qty)  FROM bookings where ord_entry_date between '2014-04-01' and  '2014-10-01'  group by product_type;

This query took around 90 Secs ..
So I started tomograph and checked plotted graph ... which shows the parallelism usage as 39.9 % only and its utilizing only 1 thread with maximum time went in the step - group.subgroupdone almost 98%.

My Question is that even I have started the server with gdk_nr_threads=8 , then why it is just using single thread.

More info --

mserver5 --version

MonetDB 5 server v11.19.7 "Oct2014-SP1" (64-bit, 64-bit oids)

Copyright (c) 1993-July 2008 CWI

Copyright (c) August 2008-2014 MonetDB B.V., all rights reserved

Visit http://www.monetdb.org/ for further information

Found 63.0GiB available memory, 8 available cpu cores

Libraries:

   libpcre: 7.8 2008-09-05 (compiled with 7.8)

   openssl: OpenSSL 1.0.1e 11 Feb 2013 (compiled with OpenSSL 1.0.1e-fips 11 Feb 2013)

   libxml2: 2.7.6 (compiled with 2.7.6)

Compiled by: root@localhost (x86_64-unknown-linux-gnu)

Compilation: gcc -O3 -fomit-frame-pointer -pipe  -D_FORTIFY_SOURCE=2

Linking    : /usr/bin/ld -m elf_x86_64

Any pointers ??

Thanks



_______________________________________________
users-list mailing list
users-list@monetdb.org
https://www.monetdb.org/mailman/listinfo/users-list

_______________________________________________
users-list mailing list
users-list@monetdb.org
https://www.monetdb.org/mailman/listinfo/users-list