Thank you Martin. So, did monetdb bought the codes from actian code base and included in it's source code? Also kindly tell me if monetdb is following any other compression techniques apart from pfor, pfor-delta and pdict. Thanks in advance.
On Mon, Jan 19, 2015 at 4:30 PM, users-list-request@monetdb.org wrote:
Send users-list mailing list submissions to users-list@monetdb.org
To subscribe or unsubscribe via the World Wide Web, visit https://www.monetdb.org/mailman/listinfo/users-list or, via email, send a message with subject or body 'help' to users-list-request@monetdb.org
You can reach the person managing the list at users-list-owner@monetdb.org
When replying, please edit your Subject line so it is more specific than "Re: Contents of users-list digest..."
Today's Topics:
- Re: Not utilising Mutithreads (Martin Kersten)
- regarding source code (bhuvan Mitra)
- Re: regarding source code (Martin Kersten)
- information about compression/decompression techniques of monetdb (bhuvan Mitra)
Message: 1 Date: Sun, 18 Jan 2015 12:20:15 +0100 From: Martin Kersten martin@monetdb.org To: users-list@monetdb.org Subject: Re: Not utilising Mutithreads Message-ID: 54BB96EF.3030906@monetdb.org Content-Type: text/plain; charset=windows-1252; format=flowed
Hi
Each MAL instruction is executed by a single thread. Some operators require the complete input to be available, such as in your case the group.subgroup().
martin
On 18/01/15 01:27, shamsul hassan wrote:
Hi All,
I have recently install monetdb "OCT 2014-SP1" release on my Oracle
Enterprise 6 Linux server.
Server configuration -
8 CPU Cores 64 GB RAM
I have loaded a data set with 70 columns having 15 Millions Rows.
I have started mserver5 as below --
mserver5 --debug=10 --set *gdk_nr_threads=8*
--dbpath=/data/monetdb/testdb --set mapi_open=true --set mapi_port=54321 --set merovingian_uri=mapi:monetdb://localhost:54321/testdb --set mapi_usock=/data/monetdb/testdb/.mapi.sock --set
monet_vault_key=/data/monetdb/testdb/.vaultkey --set
sql_optimizer=default_pipe --set max_clients=64
Now I have ran a query as below --
select product_type,COUNT(DISTINCT sales_document),sum(ord_qty) FROM
bookings where ord_entry_date between '2014-04-01' and '2014-10-01' group by product_type;
This query took around 90 Secs .. So I started tomograph and checked plotted graph ... which shows the
parallelism usage as 39.9 % only and its utilizing only 1 thread with maximum time went in the step - group.subgroupdone almost 98%.
My Question is that even I have started the server with gdk_nr_threads=8
, then why it is just using single thread.
More info --
mserver5 --version
MonetDB 5 server v11.19.7 "Oct2014-SP1" (64-bit, 64-bit oids)
Copyright (c) 1993-July 2008 CWI
Copyright (c) August 2008-2014 MonetDB B.V., all rights reserved
Visit http://www.monetdb.org/ for further information
Found 63.0GiB available memory, 8 available cpu cores
Libraries:
libpcre: 7.8 2008-09-05 (compiled with 7.8)
openssl: OpenSSL 1.0.1e 11 Feb 2013 (compiled with OpenSSL
1.0.1e-fips 11 Feb 2013)
libxml2: 2.7.6 (compiled with 2.7.6)
Compiled by: root@localhost (x86_64-unknown-linux-gnu)
Compilation: gcc -O3 -fomit-frame-pointer -pipe -D_FORTIFY_SOURCE=2
Linking : /usr/bin/ld -m elf_x86_64
Any pointers ??
Thanks
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
Message: 2 Date: Mon, 19 Jan 2015 12:28:05 +0530 From: bhuvan Mitra bhuvanrjb@gmail.com To: users-list@monetdb.org Subject: regarding source code Message-ID: < CAMmryP6eg5AAMY1x70KJZMdc3k7P8aqjnHuS5xN-fNSJ8mN_sQ@mail.gmail.com> Content-Type: text/plain; charset="utf-8"
Hello,
I am very interested to know and learn about implementations of compression/decompression techniques that monetdb is following. I have seen the monetdb's light weight compression algorithms(pfor, pfor-delta, pdict), cache involved decompression techniques and excited to know how actually are they implemented in form of codes. Kindly tell me in which folder/file is the source code of this compression/decompression algorithms present. I have downloaded the source tarball from the website and could not find these code snippets. Plese help me to identify the path.
Regards, mg
Hi,
MonetDB and Actian/VectorWise are two different systems that do not share any code.
MonetDB does not use any compression in the way Actian/VectorWise does.
Best, Stefan
----- Original Message -----
Thank you Martin. So, did monetdb bought the codes from actian code base and included in it's source code? Also kindly tell me if monetdb is following any other compression techniques apart from pfor, pfor-delta and pdict. Thanks in advance.
On Mon, Jan 19, 2015 at 4:30 PM, < users-list-request@monetdb.org > wrote:
Send users-list mailing list submissions to users-list@monetdb.org
To subscribe or unsubscribe via the World Wide Web, visit https://www.monetdb.org/mailman/listinfo/users-list or, via email, send a message with subject or body 'help' to users-list-request@monetdb.org
You can reach the person managing the list at users-list-owner@monetdb.org
When replying, please edit your Subject line so it is more specific than "Re: Contents of users-list digest..."
Today's Topics:
- Re: Not utilising Mutithreads (Martin Kersten)
- regarding source code (bhuvan Mitra)
- Re: regarding source code (Martin Kersten)
- information about compression/decompression techniques of
monetdb (bhuvan Mitra)
Message: 1 Date: Sun, 18 Jan 2015 12:20:15 +0100 From: Martin Kersten < martin@monetdb.org > To: users-list@monetdb.org Subject: Re: Not utilising Mutithreads Message-ID: < 54BB96EF.3030906@monetdb.org > Content-Type: text/plain; charset=windows-1252; format=flowed
Hi
Each MAL instruction is executed by a single thread. Some operators require the complete input to be available, such as in your case the group.subgroup().
martin
On 18/01/15 01:27, shamsul hassan wrote:
Hi All,
I have recently install monetdb "OCT 2014-SP1" release on my Oracle Enterprise 6 Linux server.
Server configuration -
8 CPU Cores 64 GB RAM
I have loaded a data set with 70 columns having 15 Millions Rows.
I have started mserver5 as below --
mserver5 --debug=10 --set *gdk_nr_threads=8* --dbpath=/data/monetdb/testdb --set mapi_open=true --set mapi_port=54321 --set merovingian_uri=mapi:monetdb://localhost:54321/testdb --set mapi_usock=/data/monetdb/testdb/.mapi.sock --set monet_vault_key=/data/monetdb/testdb/.vaultkey --set sql_optimizer=default_pipe --set max_clients=64
Now I have ran a query as below --
select product_type,COUNT(DISTINCT sales_document),sum(ord_qty) FROM bookings where ord_entry_date between '2014-04-01' and '2014-10-01' group by product_type;
This query took around 90 Secs .. So I started tomograph and checked plotted graph ... which shows the parallelism usage as 39.9 % only and its utilizing only 1 thread with maximum time went in the step - group.subgroupdone almost 98%.
My Question is that even I have started the server with gdk_nr_threads=8 , then why it is just using single thread.
More info --
mserver5 --version
MonetDB 5 server v11.19.7 "Oct2014-SP1" (64-bit, 64-bit oids)
Copyright (c) 1993-July 2008 CWI
Copyright (c) August 2008-2014 MonetDB B.V., all rights reserved
Visit http://www.monetdb.org/ for further information
Found 63.0GiB available memory, 8 available cpu cores
Libraries:
libpcre: 7.8 2008-09-05 (compiled with 7.8)
openssl: OpenSSL 1.0.1e 11 Feb 2013 (compiled with OpenSSL 1.0.1e-fips 11 Feb 2013)
libxml2: 2.7.6 (compiled with 2.7.6)
Compiled by: root@localhost (x86_64-unknown-linux-gnu)
Compilation: gcc -O3 -fomit-frame-pointer -pipe -D_FORTIFY_SOURCE=2
Linking : /usr/bin/ld -m elf_x86_64
Any pointers ??
Thanks
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
Message: 2 Date: Mon, 19 Jan 2015 12:28:05 +0530 From: bhuvan Mitra < bhuvanrjb@gmail.com > To: users-list@monetdb.org Subject: regarding source code Message-ID: < CAMmryP6eg5AAMY1x70KJZMdc3k7P8aqjnHuS5xN-fNSJ8mN_sQ@mail.gmail.com > Content-Type: text/plain; charset="utf-8"
Hello,
I am very interested to know and learn about implementations of compression/decompression techniques that monetdb is following. I have seen the monetdb's light weight compression algorithms(pfor, pfor-delta, pdict), cache involved decompression techniques and excited to know how actually are they implemented in form of codes. Kindly tell me in which folder/file is the source code of this compression/decompression algorithms present. I have downloaded the source tarball from the website and could not find these code snippets. Plese help me to identify the path.
Regards, mg
participants (2)
-
bhuvan Mitra -
Stefan Manegold