High Memory Usage - Sever Not Releasing Resources?

Ying Zhang Y.Zhang at cwi.nl
Fri Jun 8 17:17:18 CEST 2018


Hello Adam,

First of all: 

- does this also happen with Mar2018?
- Is this repeatable?

If answers to both questions are Yes, can you please provide us a script/test programme/etc to reproduce the problem?

Regards,
Jennie



> On 8 Jun 2018, at 17:03, Doherty, Adam <adam.doherty at esso.ca> wrote:
> 
> Hi all:
> 
> I ran into an issue yesterday when I ran a simple group count query - 
> The query took 7 minutes, 13 seconds to run.
> 
> Query was as follows -
> 
> SELECT zone, count(*)
> FROM chronicle.events
> WHERE event_type = 'mb'
> GROUP BY zone
> 
> This table, which a merge table currently has 546 million events.
> The merge table is made up of currently two merge table each representing a year (events_2018, and events_2017).
> Those merge tables are each made up of four tables, events_year_zone_(1-4)
> 
> I have a scheduled task which inserts events into the relevant tables every 5 minutes from 5 Microsoft SQL Servers.
> Roughly 12,000 events broken across 5 inserts (one for each server).
> 
> The import process is handled with a C# Windows service that utilizes ODBC to talk to MonetDB.
> 
> The server is currently MonetDB v11.27.11 (July2017-SP3) running on VMware 6.5 in a CentOS 7 VM with 64GB of RAM, and 16 CPU cores.
> When I ran into this issue yesterday, the mserver5 process was utilizing 90% of available RAM.
> 
> This was making my insertion task take longer and longer each time, and simple queries like the one above took minutes.
> Logged into the server via SSH, the input lag as a result of the almost total resource usage caused me to reboot the server.
> 
> I restarted the VM, and the same query took 20 seconds.
> 
> I'd like to sort where this issue might be coming from, but I'm not sure where to look.
> This server is currently part of a proof-of-concept project that will soon be going to production.
> 
> Any guidance is greatly appreciated.
> 
> Adam
> 
> _______________________________________________
> users-list mailing list
> users-list at monetdb.org
> https://www.monetdb.org/mailman/listinfo/users-list



More information about the users-list mailing list