Further investigation, using primarily the TRACE statement, has shown that indeed the MAX aggregation function is the cause of the performance degradation. For example, the following statement is 100x as slow as any other statement:
X_5569=<tmp_2041>[6]:bat[:str] := aggr.submax(X_5538=<tmp_1665>[6]:bat[:str],X_4753=<tmp_1672>[6]:bat[:oid],C_4754=<tmp_2046>[6]:bat[:oid],true:bit);
This statement alone takes around 9 seconds whereas on the previous version of MonetDB the whole query took approximately 20 seconds (as opposed to 160 seconds now). Additionally this statement generates a huge number of reads (1314481) and writes (1298936).
Any ideas what could be causing this? Should I create a bug in the bug tracker?
Best regards, Dennis Pallett
On Fri, Jan 27, 2017 at 11:36 AM, Dennis Pallett dennis@pallett.nl wrote:
Hi all,
We recently upgraded to the Dec2016 release but had to downgrade immediately after experiencing a severe degradation in performance. I have attached 4 graphs of our performance monitoring (of the last 3 days). It should be fairly obvious when we upgraded and when we downgraded again.
We're still looking into the root cause but it appears that it has something to do with memory leaks when using aggregations on varchar colums with a high cardinality.
Has anyone else experienced the same performance degradation? Is this a known (regression) bug?
Best regards, Dennis Pallett