From my own experience, I would suspect the time is taken for creating / optimizing the MAL plan, not to execute it.

Try running the same query with EXPLAIN in front. This only produces the MAL plan.
I expect this will give the same problems as running the query

Then try to replace EXPLAIN with PLAN. This only generates the algebra plan and no MAL plan.
If this works as expected, then the problem is either in the MAL generation itself, or in its optimization (more likely)

To investigate further the last case, you can try an EXPLAIN, but with this first:

sql> set optimizer='minimal_pipe';

If this works well, then the problem is in one of the optimizers that is in default_pipe but not in minimal_pipe.



On 29 March 2016 at 13:20, Vijay Krishna <vijayakrishna55@gmail.com> wrote:
Hi, 

Today, we encountered this serious memory issue with MonetDB. 

Data : 10 tables - each with ~600 columns - all tables are EMPTY (No rows).
 
Query : One query that involves with 25 joins across all the 10 tables. 

MonetDB version : July2015-SP3

The above query is ran on the same MonetDB version with the same table structure with all tables empty on two different machines as follows. 

Scenario 1 - Query ran on Macbook Pro 
  • RAM - 8 GB (with other processes like Chrome/Eclipse running in parallel)
  • Free disk space - ~100 GB
  • MonetDB July2015-SP3 downloaded as installable
The query is ran. 
Expected result - return an empty set in a few seconds. 
Actual result - returns an empty set in ~5 minutes
RAM - 8 GB utilised completely. 
Swap space used from disk - 10 GB
This swap space is cleared after sometime automatically (around 5 mins) 

Scenario 2 - With a CentOS virtual machine
  • RAM - 128 GB
  • Free disk space - ~650 GB
  • MonetDB July2015-SP3 compiled from source code
The query is ran. 
Expected result - return an empty set in a few seconds. 
Actual result - the query rebooted this high end CentOS machine. 
RAM - 128 GB utilised completely. 
After reboot, the swap memory was freed. 

I am puzzled with the following questions. 
  1. Why should a query that runs on 10 EMPTY tables require memory in GBs? 
  2. How did this query manage to run on a Mac with just 8 GB RAM, but crashed a machine with 128 GB RAM?
Is this a known memory leak issue? 

Any help much appreciated. Thanks in advance. 


Regards,
Vijay.


_______________________________________________
users-list mailing list
users-list@monetdb.org
https://www.monetdb.org/mailman/listinfo/users-list