Hi:
I have a MonetDB 5 server v11.23.7 "Jun2016-SP1 instance hosting a table with over 5600 columns (for the US General Social Survey)
If we run a simple query like SELECT * FROM gss7214_v4 limit 2 we get the first two records as expected.
However, the same query with all the column names explicitly stated like select "year","id","wrkstat","hrs1","hrs2","evwork","occ","prestige","wrkslf",<...thousands of additional columns - see attached...> from "gss7214_v4" WHERE "year"=1972 LIMIT 2 OFFSET 0; seem to kill the server. The process keeps running but become non-responsive.
Now this is on a AWS EC2 t2.small instance with just ~2Gb or memory. Maybe more memory may solve this issue (?). But in any case concerned that the server hangs without warning or error message in the log. Any potential workaround? Suggestion most appreciated
best *P
On 2016-09-30 16:36, Pascal Heus wrote:
Now this is on a AWS EC2 t2.small instance with just ~2Gb or memory. Maybe more memory may solve this issue (?). But in any case concerned that the server hangs without warning or error message in the log. Any potential workaround? Suggestion most appreciated
How many rows are you talking about?
t2s are very limited on CPU and I/O bandwidth, they also reduce your CPU power once you've reached your limit. This also comes faster than you'd expect, we used to run into this on small web server from compressing the log files.
I find bigger instances (m3s) better for monetdb, especially that being analytic work, we can shut down the instances when not in use, keep the cost to something very reasonable.
On 01 Oct 2016, at 05:54, Yves Dorfsman yves@zioup.com wrote:
On 2016-09-30 16:36, Pascal Heus wrote:
Now this is on a AWS EC2 t2.small instance with just ~2Gb or memory. Maybe more memory may solve this issue (?). But in any case concerned that the server hangs without warning or error message in the log.
You mean the merovingian.log?
I’d expect mserver5 to give errors like HEAPextend failed. But I doubt if that error message goed into the merovingian.log
Probably, mserver5 doesn’t really hang, but it can’t do anything. Since the error message of mserver5 probably doesn’t show up in the merovingian log, it feels like that the server hangs.
You can wait for some time (30+ sec) for the server to clean up its temporary data, and try a ‘select 1’ query to check if the server is really not responding…
Jennie
Any potential workaround? Suggestion most appreciated
How many rows are you talking about?
t2s are very limited on CPU and I/O bandwidth, they also reduce your CPU power once you've reached your limit. This also comes faster than you'd expect, we used to run into this on small web server from compressing the log files.
I find bigger instances (m3s) better for monetdb, especially that being analytic work, we can shut down the instances when not in use, keep the cost to something very reasonable.
-- http://yves.zioup.com gpg: 4096R/32B0F416
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
participants (3)
-
Pascal Heus -
Ying Zhang -
Yves Dorfsman