April 2016 - users-list

Strange behaviour of epoch() when converting from Unix time
by Marc Ballarin 12 Apr '16

12 Apr '16

Hi, I am observing a surprising behaviour of the epoch function: The following works as expected: SELECT epoch(now()) AS "NOW", epoch(now() - INTERVAL '5' MINUTE) AS "5 MIN AGO"; +------------+------------+ | NOW | 5 MIN AGO | +============+============+ | 1460453860 | 1460453560 | +------------+------------+ The inverse does not: SELECT epoch(1460453860) AS "NOW", epoch(1460453860-300) AS "300 SECONDS AGO"; +----------------------------+----------------------------+ | NOW | 300 SECONDS AGO | +============================+============================+ | 2016-04-12 09:37:40.000000 | 1970-01-17 21:40:53.560000 | +----------------------------+----------------------------+ The "now" part is fine, the "300 seconds ago" part is obviously not what I expect. Am I missing something or is this a bug? Regards, Marc

2 1

Check reads per sec and CPU utilization
by Shmagi Kavtaradze 11 Apr '16

11 Apr '16

I was not able to find any information how to check CPU utilization and read per sec for a query. Any ideas how to get these results? Thanks in advance.

3 3

Re: How to do profiling in MAL interface?
by Constantine Feng 11 Apr '16

11 Apr '16

Hi Martin, Thanks. stethoscope does do some work. But since it is for online monitoring, it is inconvenient for offline analysis of trace profiles. After reading more docs on the web, I find Tachograph comes closest to what I want. A minor problem is that the schema of the generated files (.trace, .csv, .json) is not given. Somehow I have to guess. And seems that each file format covers a different subset of attributes. - Neo > > ------------------------------ > > Message: 2 > Date: Sat, 9 Apr 2016 08:17:15 +0200 > From: Martin Kersten <martin.kersten(a)cwi.nl> > To: users-list(a)monetdb.org > Subject: Re: How to do profiling in MAL interface? > Message-ID: <57089E6B.7050504(a)cwi.nl> > Content-Type: text/plain; charset=windows-1252; format=flowed > > You better consider using the stethoscope for doing this kind of analysis. > The documentation need some updates. > > > On 09/04/16 06:07, Constantine Feng wrote: > > Hi all, > > > > I know I can use the TRACE SQL command. But I want to do it in the > "-lmal" interface (because I massaged the MAL plans a bit to test my > research ideas). > > > > Particularly, I find the information given on this website ( > https://www.monetdb.org/Documentation/Manuals/MonetDB/Modules/profiler ) > doesn't match the source code. I can't find the profiler.activate() or > profiler.deactivate(), for example. > > > > > > mal>manual.help("profiler.*"); > > profiler.cpuload > > profiler.cpustats > > profiler.cleanup > > profiler.closeStream > > profiler.dumpTrace > > profiler.getSystemTime > > profiler.getUserTime > > profiler.getDiskWrites > > profiler.getDiskReads > > profiler.getTrace > > profiler.noop > > profiler.openStream > > profiler.reset > > profiler.setHeartbeat > > profiler.stop > > profiler.stethoscope > > profiler.tomograph > > > > > > Version: (Jul2015-SP3). Build from source. > > > > - Neo > > > > > > _______________________________________________ > > users-list mailing list > > users-list(a)monetdb.org > > https://www.monetdb.org/mailman/listinfo/users-list > > > >

1 0

Announcement: New Jul2015-SP4 Bugfix release of MonetDB suite
by Sjoerd Mullender 11 Apr '16

11 Apr '16

The MonetDB team at CWI/MonetDB BV is pleased to announce the Jul2015-SP4 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Jul 2015-SP4 bugfix release MonetDB Common * Fixed a bug that caused various instances where old data returned or where crashes occurred. The problem was that internally data wasn't always marked dirty when it was being changed, causing later processing to not deal with the changed data correctly. Bug Fixes * 3905: MonetDB doesn't handle ANY/SOME/ALL operator correctly * 3929: R aggregate not recognized when using 3 or more parameters * 3965: Not possible to quote/escape single quote character in the name of the file to load. * 3968: Missing double use of column names

1 0

Can not enable python
by Shmagi Kavtaradze 10 Apr '16

10 Apr '16

I created function in python, but when tried to call, I receive : Embedded Python has not been enabled. Start server with --set embedded_py=true I stopped farm. used mserver5 --set embedded_py=true and got in mserver5 console output: # MonetDB/SQL module loaded # MonetDB/Python module loaded Then I started farm and still getting same error. Then I did this: monetdb stop dbname monetdb set embedpy=true dbname monetdb start dbname Still getting same. Any ideas what I am doing wrong?

2 2

Fwd: Function does not complete
by Shmagi Kavtaradze 09 Apr '16

09 Apr '16

Thanks for answer. I have bits of size 50, so I need bigint/long long int. Problem is in while loop. When I use bitand(a,b) and numbers of digits of a and b both are 18-19, the system enters endless loop I think. Bigint maximum value is 19 digits long.I have mistakes in code when using unsigned int and I changed, but same problem. As I know from data types page, monetdb uses only signed integers, so I use only signed ones. Any ideas why does it hang on 18-19 digits and not on 1-17 digits long numbers?

1 0

How to do profiling in MAL interface?
by Constantine Feng 09 Apr '16

09 Apr '16

Hi all, I know I can use the TRACE SQL command. But I want to do it in the "-lmal" interface (because I massaged the MAL plans a bit to test my research ideas). Particularly, I find the information given on this website ( https://www.monetdb.org/Documentation/Manuals/MonetDB/Modules/profiler ) doesn't match the source code. I can't find the profiler.activate() or profiler.deactivate(), for example. mal>manual.help("profiler.*"); profiler.cpuload profiler.cpustats profiler.cleanup profiler.closeStream profiler.dumpTrace profiler.getSystemTime profiler.getUserTime profiler.getDiskWrites profiler.getDiskReads profiler.getTrace profiler.noop profiler.openStream profiler.reset profiler.setHeartbeat profiler.stop profiler.stethoscope profiler.tomograph Version: (Jul2015-SP3). Build from source. - Neo

2 1

Controlling --dbextras Using MonetDB(D)
by Alex Manaev 08 Apr '16

08 Apr '16

Hi, I use wrapper shell in such case. May be useful for you. $ mv /usr/bin/mserver5 /usr/bin/mserver5.bin $ vim /usr/bin/mserver5 #!/usr/bin/env bash exec /usr/bin/mserver5.bin --set gdk_vmtrim yes --set gdk_vm_maxsize 200000000000 --dbextra=/mnt/data/monetdb/extra $@ Alexey Manaev, Head of software development CJSC "RUTUBE"

3 4

monetdb crash as start : Logger_new: BBPrename to sql_snapshots_bid failed
by Pierre-Adrien Coustillas 08 Apr '16

08 Apr '16

Hello, after a server crash : # mclient comptage_cookie_telerama monetdbd: internal error while starting mserver, please refer to the logs # cat merovingian.log 2016-04-08 14:28:33 MSG merovingian[13582]: starting database 'comptage_cookie_telerama', up min/avg/max: 28m/4w/10w, crash average: 1.00 1.00 1.00 (246-6=240) 2016-04-08 14:28:34 MSG comptage_cookie_telerama[16350]: arguments: /usr/bin/mserver5 --dbpath=/var/lib/monetdb/comptage_cookie_telerama --set merovingian_uri=mapi:monetdb://groupelemonde-crm:50000/comptage_cookie_telerama --set mapi_open=false --set mapi_port=0 --set mapi_usock=/var/lib/monetdb/comptage_cookie_telerama/.mapi.sock --set monet_vault_key=/var/lib/monetdb/comptage_cookie_telerama/.vaultkey --set gdk_nr_threads=8 --set max_clients=64 --set sql_optimizer=default_pipe --set monet_daemon=yes 2016-04-08 14:28:34 MSG comptage_cookie_telerama[16350]: # MonetDB 5 server v11.21.13 "Jul2015-SP2" 2016-04-08 14:28:34 MSG comptage_cookie_telerama[16350]: # Serving database 'comptage_cookie_telerama', using 8 threads 2016-04-08 14:28:34 MSG comptage_cookie_telerama[16350]: # Compiled for x86_64-pc-linux-gnu/64bit with 64bit OIDs and 128bit integers dynamically linked 2016-04-08 14:28:34 MSG comptage_cookie_telerama[16350]: # Found 64.000 GiB available main-memory. 2016-04-08 14:28:34 MSG comptage_cookie_telerama[16350]: # Copyright (c) 1993-July 2008 CWI. 2016-04-08 14:28:34 MSG comptage_cookie_telerama[16350]: # Copyright (c) August 2008-2015 MonetDB B.V., all rights reserved 2016-04-08 14:28:34 MSG comptage_cookie_telerama[16350]: # Visit http://www.monetdb.org/ for further information 2016-04-08 14:28:34 MSG comptage_cookie_telerama[16350]: # Listening for UNIX domain connection requests on mapi:monetdb:///var/lib/monetdb/comptage_cookie_telerama/.mapi.sock 2016-04-08 14:28:34 MSG comptage_cookie_telerama[16350]: # Start processing logs sql/sql_logs version 52200 2016-04-08 14:28:34 MSG merovingian[13582]: database 'comptage_cookie_telerama' (16350) has exited with exit status 1 2016-04-08 14:28:34 ERR merovingian[13582]: client error: database 'comptage_cookie_telerama' appears to shut itself down after starting, check monetdbd's logfile for possible hints 2016-04-08 14:28:34 MSG merovingian[13582]: starting database 'comptage_cookie_telerama', up min/avg/max: 28m/4w/10w, crash average: 1.00 1.00 1.00 (247-6=241) 2016-04-08 14:28:34 MSG comptage_cookie_telerama[16354]: arguments: /usr/bin/mserver5 --dbpath=/var/lib/monetdb/comptage_cookie_telerama --set merovingian_uri=mapi:monetdb://groupelemonde-crm:50000/comptage_cookie_telerama --set mapi_open=false --set mapi_port=0 --set mapi_usock=/var/lib/monetdb/comptage_cookie_telerama/.mapi.sock --set monet_vault_key=/var/lib/monetdb/comptage_cookie_telerama/.vaultkey --set gdk_nr_threads=8 --set max_clients=64 --set sql_optimizer=default_pipe --set monet_daemon=yes 2016-04-08 14:28:34 MSG comptage_cookie_telerama[16354]: # MonetDB 5 server v11.21.13 "Jul2015-SP2" 2016-04-08 14:28:34 MSG comptage_cookie_telerama[16354]: # Serving database 'comptage_cookie_telerama', using 8 threads 2016-04-08 14:28:34 MSG comptage_cookie_telerama[16354]: # Compiled for x86_64-pc-linux-gnu/64bit with 64bit OIDs and 128bit integers dynamically linked 2016-04-08 14:28:34 MSG comptage_cookie_telerama[16354]: # Found 64.000 GiB available main-memory. 2016-04-08 14:28:34 MSG comptage_cookie_telerama[16354]: # Copyright (c) 1993-July 2008 CWI. 2016-04-08 14:28:34 MSG comptage_cookie_telerama[16354]: # Copyright (c) August 2008-2015 MonetDB B.V., all rights reserved 2016-04-08 14:28:34 MSG comptage_cookie_telerama[16354]: # Visit http://www.monetdb.org/ for further information 2016-04-08 14:28:34 MSG comptage_cookie_telerama[16354]: # Listening for UNIX domain connection requests on mapi:monetdb:///var/lib/monetdb/comptage_cookie_telerama/.mapi.sock 2016-04-08 14:28:34 MSG comptage_cookie_telerama[16354]: # Start processing logs sql/sql_logs version 52200 2016-04-08 14:28:34 MSG merovingian[13582]: database 'comptage_cookie_telerama' (16354) has exited with exit status 1 2016-04-08 14:28:39 ERR merovingian[13582]: client error: database 'comptage_cookie_telerama' appears to shut itself down after starting, check monetdbd's logfile for possible hints # cat .gdk_lock USR=107 PID=16354 TIME=Fri Apr 8 14:28:34 2016 @ LOGON USR=107 PID=16354 TIME=Fri Apr 8 14:28:34 2016 @ !FATAL: Logger_new: BBPrename to sql_snapshots_bid failed USR=107 PID=16354 TIME=Fri Apr 8 14:28:34 2016 @ LOGOFF do you have an idea ? Thank, Pierre -- 1G6 52 route de bischwiller 67300 Schiltigheim Société de Services et de Formations en Logiciels Libres http://1g6.biz Tél : 06 64 63 70 35 // 09 54 38 70 90

1 0

any ideas why CPU usage would never go over 25%?
by Anthony Damico 08 Apr '16

08 Apr '16

hi, we have a powerful windows server with 200GB RAM and 40 cores. i've been using it for a few weeks now, and mserver.exe never seems to use more than 10 of the cores. mserver does say "using 40 threads" so they're all available is it just rare for any SQL query to use more than 10 cores at once, or is it possible i've misconfigured something? thanks

1 0