Hi, On processors that support SMT technology if Logical Processors are enabled your software segfaults, if you disable Logical Processors it works but then you use half of the available memory. Dariusz.

Martin Kersten wrote:

Nafees Ur Rehman wrote:

...

Hi,

Does monetdb has support for parallel processing? If some can guide me on that.

What kind of parallel processing do you have in mind? It runs the queries in parallel on a multi-core engine. regards,Martin

...

Regards Nafees Ur Rehman

---

---

Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july

---

---

MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users

---

Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users

The problem is HT (hyper-threading), eg. - writing out table output to disk. When I have Logical Processor enabled (BIOS), MonetDB thinks I have 16 cores, but my machine has only *8* (with 2 threads per core). So when I try to write the file out, the second thread will try to release the memory which has been already released by the first thread and will segfault. When I turn off Logical Processor - writing out files works, but MonetDB will use 50% of available memory at most. Dariusz.

Logical Processor Disabled root@monetdb:/var/MonetDB# mserver5 --version MonetDB server v5.18.5 (64-bit), based on kernel v1.36.5 (64-bit oids) Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved Visit http://monetdb.cwi.nl/ for further information Found 47.3GiB available memory, 8 available cpu cores Configured for prefix: /usr Libraries: libpcre: 7.8 2008-09-05 (compiled with 7.8) openssl: OpenSSL 0.9.8g 19 Oct 2007 (compiled with OpenSSL 0.9.8g 19 Oct 2007) libxml2: 2.7.5 (compiled with 2.7.5) Compiled by: root@ubuntu-karmic-64 (x86_64-pc-linux-gnu) Compilation: gcc -O2 -Wall -O2 -O6 -fomit-frame-pointer -finline-functions -falign-loops=4 -falign-jumps=4 -falign-functions=4 -fexpensive-optimizations -funroll-loops -frerun-cse-after-loop -frerun-loop-opt -ftree-vectorize Linking : ld -IPA -m elf_x86_64 -Wl,-Bsymbolic-functions

Logical Processor Enabled root@monetdb:/var/MonetDB# mserver5 --version MonetDB server v5.18.5 (64-bit), based on kernel v1.36.5 (64-bit oids) Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved Visit http://monetdb.cwi.nl/ for further information Found 47.3GiB available memory, 16 available cpu cores Configured for prefix: /usr Libraries: libpcre: 7.8 2008-09-05 (compiled with 7.8) openssl: OpenSSL 0.9.8g 19 Oct 2007 (compiled with OpenSSL 0.9.8g 19 Oct 2007) libxml2: 2.7.5 (compiled with 2.7.5) Compiled by: root@ubuntu-karmic-64 (x86_64-pc-linux-gnu) Compilation: gcc -O2 -Wall -O2 -O6 -fomit-frame-pointer -finline-functions -falign-loops=4 -falign-jumps=4 -falign-functions=4 -fexpensive-optimizations -funroll-loops -frerun-cse-after-loop -frerun-loop-opt -ftree-vectorize Linking : ld -IPA -m elf_x86_64 -Wl,-Bsymbolic-functions

Dariusz.

Stefan Manegold wrote:

On Fri, Apr 30, 2010 at 01:33:08PM -0400, dariuszs wrote:

...

Hi, On processors that support SMT technology if Logical Processors are enabled your software segfaults,

Could you be more specific and verbose?

Does "it" (I suppose you mean MonetDB, in particular MonetDB5/SQL) segfault always (i.e., as soon as you start mserver5 or merovingian), or only under particular circumstances/workloads/situations?

I.e., how (if at all) could we reproduce the segfault(s)? (We cannot do much about problems without detailed description how to reproduce them ...)

In case we cannot reproduce the segfault, can you run MonetDB in a debugger and report where/how/why it segfaults?

...

if you disable Logical Processors it works but then you use half of the available memory. Dariusz.

How exactly do you come to the conclusion that MonetDB only uses half of the (physically) available memory in case Logical Processors ("Hyperthreading") are disabled? How are Logical Processors related to physical memory usage?

Stefan

...

Martin Kersten wrote:

...

Nafees Ur Rehman wrote:

...

Hi,

Does monetdb has support for parallel processing? If some can guide me on that.

What kind of parallel processing do you have in mind? It runs the queries in parallel on a multi-core engine. regards,Martin

...

Regards Nafees Ur Rehman

Hi, I've already posted a lot of info on this list, please search for subject 'copy into' or 'copy into'. Memory usage - it never goes above 50% of total, and it used to do that - I've got 3 tables with 350 million records (about 175GB of storage) which I access at the same time. Disk usage - it used to grab 3 times the size of the original input file and now it's almost the same. Dariusz.

Stefan Manegold wrote:

Darius,

thanks for reporting this problem. It indeed seems that something might be going wrong in MonetDB in your case --- without more details about what exacly you do (i.e., the DB scheman, the data and the queries you rung that trigger the segfault, I can only vaguely guess that runnign with 16 as opposed to 8 threads triggers some race condition.

I do think, though, that the conclusion that this is due to hyperthreading is pre-mature. The most we can conclude so far it that increasing parallelism from 8 threads to 16 treads seems to *trigger* the segfault. The actual *cause* is yet unknown and requires detailed debugging to be found.

We would very much apprechiate your helpt to at least share with us the details how to reproduce the segfault --- I'm not aware of any Linux machine with hyoerthreading that we have access to, but we do have acces to a Sun T1 with 6 cores and 4 hw threads per core ...

You could also test yourself, whether, e.g., the problems occurs, too, if your machine has hyperthreading enabled, but you limit MonetDb to use only 8 core/thread by starting mserver5 with command line option --set gdk_nr_threads=8. I that works fine, you could try to find the actual degree of parallelism that triggers the segfault, e.g., by bisection between 8 & 16 threads ...

Stefan

ps: Likewise, the fact that MonetDB uses at most 50% with hyperthreading disabled needs to be analysed in detail --- MonetDB uses only as much memory as required for query processing ...

On Fri, Apr 30, 2010 at 02:08:49PM -0400, dariuszs wrote:

...

The problem is HT (hyper-threading), eg. - writing out table output to disk. When I have Logical Processor enabled (BIOS), MonetDB thinks I have 16 cores, but my machine has only *8* (with 2 threads per core). So when I try to write the file out, the second thread will try to release the memory which has been already released by the first thread and will segfault. When I turn off Logical Processor - writing out files works, but MonetDB will use 50% of available memory at most. Dariusz.

Logical Processor Disabled root@monetdb:/var/MonetDB# mserver5 --version MonetDB server v5.18.5 (64-bit), based on kernel v1.36.5 (64-bit oids) Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved Visit http://monetdb.cwi.nl/ for further information Found 47.3GiB available memory, 8 available cpu cores Configured for prefix: /usr Libraries: libpcre: 7.8 2008-09-05 (compiled with 7.8) openssl: OpenSSL 0.9.8g 19 Oct 2007 (compiled with OpenSSL 0.9.8g 19 Oct 2007) libxml2: 2.7.5 (compiled with 2.7.5) Compiled by: root@ubuntu-karmic-64 (x86_64-pc-linux-gnu) Compilation: gcc -O2 -Wall -O2 -O6 -fomit-frame-pointer -finline-functions -falign-loops=4 -falign-jumps=4 -falign-functions=4 -fexpensive-optimizations -funroll-loops -frerun-cse-after-loop -frerun-loop-opt -ftree-vectorize Linking : ld -IPA -m elf_x86_64 -Wl,-Bsymbolic-functions

Logical Processor Enabled root@monetdb:/var/MonetDB# mserver5 --version MonetDB server v5.18.5 (64-bit), based on kernel v1.36.5 (64-bit oids) Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved Visit http://monetdb.cwi.nl/ for further information Found 47.3GiB available memory, 16 available cpu cores Configured for prefix: /usr Libraries: libpcre: 7.8 2008-09-05 (compiled with 7.8) openssl: OpenSSL 0.9.8g 19 Oct 2007 (compiled with OpenSSL 0.9.8g 19 Oct 2007) libxml2: 2.7.5 (compiled with 2.7.5) Compiled by: root@ubuntu-karmic-64 (x86_64-pc-linux-gnu) Compilation: gcc -O2 -Wall -O2 -O6 -fomit-frame-pointer -finline-functions -falign-loops=4 -falign-jumps=4 -falign-functions=4 -fexpensive-optimizations -funroll-loops -frerun-cse-after-loop -frerun-loop-opt -ftree-vectorize Linking : ld -IPA -m elf_x86_64 -Wl,-Bsymbolic-functions

Dariusz.

Stefan Manegold wrote:

...

On Fri, Apr 30, 2010 at 01:33:08PM -0400, dariuszs wrote:

...

Hi, On processors that support SMT technology if Logical Processors are enabled your software segfaults,

Could you be more specific and verbose?

Does "it" (I suppose you mean MonetDB, in particular MonetDB5/SQL) segfault always (i.e., as soon as you start mserver5 or merovingian), or only under particular circumstances/workloads/situations?

I.e., how (if at all) could we reproduce the segfault(s)? (We cannot do much about problems without detailed description how to reproduce them ...)

In case we cannot reproduce the segfault, can you run MonetDB in a debugger and report where/how/why it segfaults?

...

if you disable Logical Processors it works but then you use half of the available memory. Dariusz.

How exactly do you come to the conclusion that MonetDB only uses half of the (physically) available memory in case Logical Processors ("Hyperthreading") are disabled? How are Logical Processors related to physical memory usage?

Stefan

...

Martin Kersten wrote:

...

Nafees Ur Rehman wrote:

...

Hi,

Does monetdb has support for parallel processing? If some can guide me on that.

What kind of parallel processing do you have in mind? It runs the queries in parallel on a multi-core engine. regards,Martin

...

Regards Nafees Ur Rehman

---

---

MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users

Hi, Less disk space and memory usage is definitely a plus, I was just saying what I've noticed in the context of multi-threading. Speaking of which - the fastest query processing of any type I achieved was when I set gdk_nr_threads = 1, more threads slower the queries were, which I think it should be reverse. Anyway I'll stick with single threaded setup which is the fastest and it does not cause any segfaults. Dariusz.

Stefan Manegold wrote:

On Mon, May 03, 2010 at 09:33:58AM -0400, dariuszs wrote:

...

Hi, I've already posted a lot of info on this list, please search for subject 'copy into' or 'copy into'.

Ok, than I/we will have to dig through our email archives --- we'll see when our agendas leave room for that.

...

Memory usage - it never goes above 50% of total, and it used to do that - I've got 3 tables with 350 million records (about 175GB of storage) which I access at the same time.

I must confess, not having all your previous related emails in mind (sorry!), I'm a bit lost. Do you experience the "only" 50% total memory usage with loading your data ("copy into") or with running queries? If the former, do I understand it correctly, that you load the data into the three tables with three concurrent "copy into" statements? If the latter, I guess you run queries that involve all three tables (but not necessarily all columns of all three tables in a single query).

Moreover, I consider using less memory (while still working fine) an improvement rather than a problem. Am I missing anything? --- Ah, might have been in your previous mails --- I'll sreach for that once I find the time ....

...

Disk usage - it used to grab 3 times the size of the original input file and now it's almost the same.

Well, we did some changes in the Feb2010 release that reduced the storages (and memory) foot print of string attributes considerably --- I admit that I do not recall by hard the schema of your tables that you might have reported earlier, nor do I recall by hard which version of MonetDB you reported grabbing 3 times the original size, hence, I cannot tell, whether this changes apply in your case.

But again, like with the memory usage, als for disk usage, I consider "less" and improvement, not a problem ...

I'll come back, once I managed to find the time to search for the answers to my questions in your earlier posting.

Stefan

...

Dariusz.

...

Stefan Manegold wrote:

...

Darius,

thanks for reporting this problem. It indeed seems that something might be going wrong in MonetDB in your case --- without more details about what exacly you do (i.e., the DB scheman, the data and the queries you rung that trigger the segfault, I can only vaguely guess that runnign with 16 as opposed to 8 threads triggers some race condition.

I do think, though, that the conclusion that this is due to hyperthreading is pre-mature. The most we can conclude so far it that increasing parallelism from 8 threads to 16 treads seems to *trigger* the segfault. The actual *cause* is yet unknown and requires detailed debugging to be found.

We would very much apprechiate your helpt to at least share with us the details how to reproduce the segfault --- I'm not aware of any Linux machine with hyoerthreading that we have access to, but we do have acces to a Sun T1 with 6 cores and 4 hw threads per core ...

You could also test yourself, whether, e.g., the problems occurs, too, if your machine has hyperthreading enabled, but you limit MonetDb to use only 8 core/thread by starting mserver5 with command line option --set gdk_nr_threads=8. I that works fine, you could try to find the actual degree of parallelism that triggers the segfault, e.g., by bisection between 8 & 16 threads ...

Stefan

ps: Likewise, the fact that MonetDB uses at most 50% with hyperthreading disabled needs to be analysed in detail --- MonetDB uses only as much memory as required for query processing ...

On Fri, Apr 30, 2010 at 02:08:49PM -0400, dariuszs wrote:

...

The problem is HT (hyper-threading), eg. - writing out table output to disk. When I have Logical Processor enabled (BIOS), MonetDB thinks I have 16 cores, but my machine has only *8* (with 2 threads per core). So when I try to write the file out, the second thread will try to release the memory which has been already released by the first thread and will segfault. When I turn off Logical Processor - writing out files works, but MonetDB will use 50% of available memory at most. Dariusz.

Logical Processor Disabled root@monetdb:/var/MonetDB# mserver5 --version MonetDB server v5.18.5 (64-bit), based on kernel v1.36.5 (64-bit oids) Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved Visit http://monetdb.cwi.nl/ for further information Found 47.3GiB available memory, 8 available cpu cores Configured for prefix: /usr Libraries: libpcre: 7.8 2008-09-05 (compiled with 7.8) openssl: OpenSSL 0.9.8g 19 Oct 2007 (compiled with OpenSSL 0.9.8g 19 Oct 2007) libxml2: 2.7.5 (compiled with 2.7.5) Compiled by: root@ubuntu-karmic-64 (x86_64-pc-linux-gnu) Compilation: gcc -O2 -Wall -O2 -O6 -fomit-frame-pointer -finline-functions -falign-loops=4 -falign-jumps=4 -falign-functions=4 -fexpensive-optimizations -funroll-loops -frerun-cse-after-loop -frerun-loop-opt -ftree-vectorize Linking : ld -IPA -m elf_x86_64 -Wl,-Bsymbolic-functions

Logical Processor Enabled root@monetdb:/var/MonetDB# mserver5 --version MonetDB server v5.18.5 (64-bit), based on kernel v1.36.5 (64-bit oids) Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved Visit http://monetdb.cwi.nl/ for further information Found 47.3GiB available memory, 16 available cpu cores Configured for prefix: /usr Libraries: libpcre: 7.8 2008-09-05 (compiled with 7.8) openssl: OpenSSL 0.9.8g 19 Oct 2007 (compiled with OpenSSL 0.9.8g 19 Oct 2007) libxml2: 2.7.5 (compiled with 2.7.5) Compiled by: root@ubuntu-karmic-64 (x86_64-pc-linux-gnu) Compilation: gcc -O2 -Wall -O2 -O6 -fomit-frame-pointer -finline-functions -falign-loops=4 -falign-jumps=4 -falign-functions=4 -fexpensive-optimizations -funroll-loops -frerun-cse-after-loop -frerun-loop-opt -ftree-vectorize Linking : ld -IPA -m elf_x86_64 -Wl,-Bsymbolic-functions

Dariusz.

Stefan Manegold wrote:

...

On Fri, Apr 30, 2010 at 01:33:08PM -0400, dariuszs wrote:

...

Hi, On processors that support SMT technology if Logical Processors are enabled your software segfaults,

Could you be more specific and verbose?

Does "it" (I suppose you mean MonetDB, in particular MonetDB5/SQL) segfault always (i.e., as soon as you start mserver5 or merovingian), or only under particular circumstances/workloads/situations?

I.e., how (if at all) could we reproduce the segfault(s)? (We cannot do much about problems without detailed description how to reproduce them ...)

In case we cannot reproduce the segfault, can you run MonetDB in a debugger and report where/how/why it segfaults?

...

if you disable Logical Processors it works but then you use half of the available memory. Dariusz.

How exactly do you come to the conclusion that MonetDB only uses half of the (physically) available memory in case Logical Processors ("Hyperthreading") are disabled? How are Logical Processors related to physical memory usage?

Stefan

...

Martin Kersten wrote:

> Nafees Ur Rehman wrote: > > > > >> Hi, >> >> Does monetdb has support for parallel processing? If some can guide me >> on that. >> >> >> >> > What kind of parallel processing do you have in mind? > It runs the queries in parallel on a multi-core engine. > regards,Martin > > > > >> Regards >> Nafees Ur Rehman >> >> >>

---

---

MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users

---

---

MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users