August 2017 - users-list

Error after updating monetdb
by Mahmoud Mohsen 19 Aug '17

19 Aug '17

Hello, I have updated monetdb from Dec2016-SP5 to version v11.27.5 "Jul2017-SP1, but after this when trying to accessing the database via mclient -u monetdb -d tpch2 I am getting this error: internal error while starting mserver 'database 'tpch2' started up, but failed to open up a communication channel', please refer to the logs looking at the logs, 2017-08-18 22:42:02 ERR merovingian[10645]: client error: database 'voc' started up, but failed to open up a communication channel 2017-08-18 22:42:02 MSG merovingian[10645]: database 'voc' (10660) has exited with exit status 0 2017-08-18 22:42:02 MSG merovingian[10645]: starting database 'voc', up min/avg/max: 0s/0s/0s, crash average: 0.00 0.00 0.00 (1-1=0) 2017-08-18 22:42:02 MSG merovingian[10645]: sending process 10663 (database 'voc') the TERM signal 2017-08-18 22:42:02 MSG voc[10663]: arguments: /usr/bin/mserver5 --dbpath=/home/mohsen/mfarm/voc --set merovingian_uri=mapi:monetdb://u145-16:50000/voc --set mapi_open=false --set mapi_port=0 --set mapi_usock=/home/mohsen/mfarm/voc/.mapi.sock --set monet_vault_key=/home/mohsen/mfarm/voc/.vaultkey --set gdk_nr_threads=8 --set max_clients=64 --set sql_optimizer=default_pipe --set monet_daemon=yes 2017-08-18 22:42:02 MSG voc[10663]: # MonetDB 5 server v11.27.5 "Jul2017-SP1" 2017-08-18 22:42:02 MSG voc[10663]: # Serving database 'voc', using 8 threads 2017-08-18 22:42:02 MSG voc[10663]: # Compiled for x86_64-pc-linux-gnu/64bit with 128bit integers 2017-08-18 22:42:02 MSG voc[10663]: # Found 251.889 GiB available main-memory. 2017-08-18 22:42:02 MSG voc[10663]: # Copyright (c) 1993-July 2008 CWI. 2017-08-18 22:42:02 MSG voc[10663]: # Copyright (c) August 2008-2017 MonetDB B.V., all rights reserved 2017-08-18 22:42:02 MSG voc[10663]: # Visit https://www.monetdb.org/ for further information 2017-08-18 22:42:02 MSG voc[10663]: !SyntaxException:parseError:include sqlcatalog; 2017-08-18 22:42:02 MSG voc[10663]: !SyntaxException:parseError:^LoaderException:loadLibrary:Loading error failed to open library sql (from within file '/usr/lib/monetdb5/lib_sql.so'): /usr/lib/monetdb5/lib_sql.so: undefined symbol: mnstr_writeFlt 2017-08-18 22:42:02 MSG voc[10663]: !SyntaxException:parseError:include sqlcatalog; 2017-08-18 22:42:02 MSG voc[10663]: !SyntaxException:parseError: ^';' expected 2017-08-18 22:42:03 MSG merovingian[10645]: database 'voc' has shut down 2017-08-18 22:42:03 ERR control[10645]: !monetdbd: an internal error has occurred 'database 'voc' started up, but failed to open up a communication channel' 2017-08-18 22:42:07 ERR merovingian[10645]: client error: database 'voc' started up, but failed to open up a communication channel 2017-08-18 22:42:07 MSG merovingian[10645]: database 'voc' (10663) has exited with exit status 0 Thank you a lot in advance, Best regards, Mahmoud Mohsen

2 3

Re: Re: failed to import record too long: Loading 1.5 Terabyte table
by Stefan Manegold 18 Aug '17

18 Aug '17

You need to use the correct record delimiter i.e. newline instead of space. -------- Original message --------From: Mahmoud Mohsen <mmohsen35(a)gmail.com> Date: 8/18/17 19:40 (GMT+01:00) To: users-list(a)monetdb.org Subject: Re: Re: failed to import record too long: Loading 1.5 Terabyte table Dear Stefan, Thank you very much for your quick reply, so this is the first line of the file "lineitem.tbl", so no need to use an offset I think 1|310378689|15378705|1|17|29786.89|0.04|0.02|N|O|1996-03-13|1996-02-12|1996-03-22|DELIVER IN PERSON|TRUCK|egular courts above the| as for the version, I am using: MonetDB Database Server Toolkit v1.1 (Dec2016-SP5). So can this may be the cause of the error... an old version? I am looking forward hearing from you, Best regards, Mahmoud Mohsen

2 2

failed to import record too long: Loading 1.5 Terabyte table
by Mahmoud Mohsen 18 Aug '17

18 Aug '17

Hello, I'm trying to load a table with 1.5 Terabytes of data, the command I'm using COPY INTO lineitem FROM '/vol1/tpch-dbgen/2000/lineitem.tbl' USING DELIMITERS '|', ' ' LOCKED; The physical memory I'm using is 256G, the swap is 2.2 terabytes, the dbfarm is located within area has 4 Terabytes of free storage. The message I'm getting: "Failed to import table 'lineitem', record too long" So what could be a possible solution to this problem, Thank you a lot in advance, Mahmoud Mohsen

3 3

(lastest version) monetdb source compilation and installation problem
by 张馨匀 17 Aug '17

17 Aug '17

Hi All, Could you please help me to solve the following error? When I try to install monetdb I have the following error: * fatal error: *geos_c.h: No such file or directory #include <geos_c.h> *My steps used to compile monetdb src and to install monetdb are:* a. Environment setup: $ export MonetDB_rel=July2017 $ export MonetDB_ver=11.27.5 $ export MonetDB_base_dir=$HOME/MonetDB-$MonetDB_rel $ export MonetDB_source_dir=$MonetDB_base_dir/MonetDB-$MonetDB_ver $ export MonetDB_build_dir=$MonetDB_base_dir/BUILD $ export MonetDB_prefix_dir=$MonetDB_base_dir $ mkdir $MonetDB_base_dir $ cd $MonetDB_base_dir b. Download the latest version: $ wget https://dev.monetdb.org/downloads/sources/Jul2017-SP1/MonetDB-11.27.5.tar.xz $ tar xf MonetDB-$MonetDB_ver.tar.xz c. Build: $ mkdir $MonetDB_build_dir $ cd $MonetDB_build_dir $ MonetDB_source_dir/configure --prefix=$MonetDB_prefix_dir d. Install: $ make -j 20 (error mentioned above occurs here) $ make -j 20 install Thanks,

2 1

Memory bloat on concurrency - Any fix to release memory post query run
by Sharma, Sreejith 11 Aug '17

11 Aug '17

Hi, I have been using monetdb (11.25.3) in CentOS for one of application. I have been working on a PSR testing for this application and have noticed on high concurrency and on same pattern of query running again and again causing memory to bloat up and doesn’t release once the query run completes. I understand that the data is lifted into memory and the result set is also cached, however I assume this should be release once the system is idle which I don’t see it as of now. Is there any way or parameter to release the unused memory blocks? I have asked this before, but couldn't get to the root to fix this. Regards, Sreejith

3 4

Troubles with Python API
by 倪令斌 09 Aug '17

09 Aug '17

Hi, I want to set up an API to MonetDBwith Python. I have followed the instructions（https://www.monetdb.org/Documentation/Manuals/SQLreference/Pro… plan to make connection to my DB with this Python statement: import pymonetdb; connection=pymonetdb.connect(username="monetdb",password="monetdb",hostname="127.0.0.1",database="testdb") But I got the following ERROR: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/lingbin/maizegenome/maizedbfarm/testpy/local/lib/python2.7/site-packages/pymonetdb/__init__.py", line 48, in connect return Connection(*args, **kwargs) File "/home/lingbin/maizegenome/maizedbfarm/testpy/local/lib/python2.7/site-packages/pymonetdb/sql/connections.py", line 55, in __init__ unix_socket=unix_socket, connect_timeout=connect_timeout) File "/home/lingbin/maizegenome/maizedbfarm/testpy/local/lib/python2.7/site-packages/pymonetdb/mapi.py", line 139, in connect self.socket.connect((hostname, port)) File "/usr/lib/python2.7/socket.py", line 228, in meth return getattr(self._sock,name)(*args) socket.error: [Errno 111] Connection refused Do you have a hint for me how to solve this error? Many thanks, Lingbin

2 3

Troubles with Python API
by 倪令斌 08 Aug '17

08 Aug '17

Hi, I want to set up an API to MonetDBwith Python. I have followed the instructions（https://www.monetdb.org/Documentation/Manuals/SQLreference/Pro… plan to make connection to my DB with this Python statement: import pymonetdb; connection=pymonetdb.connect(username="monetdb",password="monetdb",hostname="127.0.0.1",database="testdb") But I got the following ERROR: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/lingbin/maizegenome/maizedbfarm/testpy/local/lib/python2.7/site-packages/pymonetdb/__init__.py", line 48, in connect return Connection(*args, **kwargs) File "/home/lingbin/maizegenome/maizedbfarm/testpy/local/lib/python2.7/site-packages/pymonetdb/sql/connections.py", line 55, in __init__ unix_socket=unix_socket, connect_timeout=connect_timeout) File "/home/lingbin/maizegenome/maizedbfarm/testpy/local/lib/python2.7/site-packages/pymonetdb/mapi.py", line 139, in connect self.socket.connect((hostname, port)) File "/usr/lib/python2.7/socket.py", line 228, in meth return getattr(self._sock,name)(*args) socket.error: [Errno 111] Connection refused Do you have a hint for me how to solve this error? Many thanks, Lingbin

1 0

Mdiff outputs unexpected result
by Zhouyang Jia 05 Aug '17

05 Aug '17

Hi, I'm new to MonetDB, and I find that the component Mdiff would generate unexpected results if inputted wrong files, e.g., ./Mdiff wrong_file_1 wrong_file1. Mdiff will output that these two files are equal, whereas the expected result shoud be "files not exist". Attached please find the patch against version MonetDB-11.27.5. Hopefully, it can solve this problem. Best, Zhouyang

1 0

Announcement: New Jul2017-SP1 Bugfix release of MonetDB suite
by Sjoerd Mullender 04 Aug '17

04 Aug '17

The MonetDB team at CWI/MonetDB BV is pleased to announce the Jul2017-SP1 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Jul 2017-SP1 bugfix release Build Environment * The Debian and Ubuntu installers have been fixed: there was a file missing in the Jul2017 release. * Added a new RPM called MonetDB-selinux which provides the SELinux policy required to run MonetDB under systemd, especially on Fedora 26. * The Windows installers (*.msi files) are now created using the WiX Toolset. * The Windows binaries are now built using Visual Studio 2015. Because of this, you may need to install the Visual C++ Redistributable for Visual Studio 2015 before being able to run MonetDB. Merovingian * monetdbd was leaking open file descriptors to the mserver5 process it started. This has been fixed. MonetDB Common * Many functions in GDK are now annotated with the GCC attribute __warn_unused_result__ meaning that the compiler will issue a warning if the result of the function (usually an indication of an error) is not used. Bug Fixes * 6325: Merge table unusable in other connections * 6328: Transactional/multi-connection issues with merge tables * 6336: VALUES multiple inserts error * 6339: Mserver5 crashes on nested SELECT * 6340: sample operator takes effect after the execution of the query, expected before * 6341: MERGE TABLE issue: Cannot register * 6342: MERGE TABLE issue: hang * 6344: Spurious errors and assertions (SQLsmith) * 6375: MAL profiler truncates JSON objects larger than 8192 characters

3 2