users-list September 2017

users-list@monetdb.org

13 participants
19 discussions

DEC2016-SP2 and BINARY bulk load

by Lynn Carol Johnson

Hi all – I have always used the COPY BINARY INTO … commands to load my 2.0 Billion row genetic data into a monetdb table. With 135 columns, it has been blindingly fast. Last week I moved from the June2016-SP2 release to dec2016-SP2. My binary loads are taking WAY longer. I killed one after 3 hours (via “call sys.stop(pid)” so it could clean up properly). I then started the load again, thinking perhaps the problem was related to the new columns I was adding. I have since dropped the table and remade it using the same data and scripts that worked in just over 3 minutes in February on the jun2016-SP2 load. It is really chugging along – I’m up to 30 minutes and counting. I don’t have access to the sql log files, but the Merovingian.log shows nothing. I do notice that previously the binary files, once loaded, were removed from the loading directly. This does not happen now. Were these files previously “moved” and now they are copied? Has anyone see this performance issue with Dec2016-SP2 COPY BINARY INTO …. Commands? Thanks - Lynn

4 years, 6 months

MonetDB Blob Inserts

by Brian Hood

Hi Guys, I don't seem to be able to INSERT into a blob field is there any information how to achieve this. Regards, Brian Hood

4 years, 7 months

Setting Up Tools

by Sandy

Just getting back into trying MonetDB after many years. Got the environment set up pretty well with the test DB `VOC` loaded up and able to do all expected in the command line tools. So I figured I would give a couple of the mentioned GUI tools a shot and see how they work against the server I just set up on a virtual box. I picked up the JDBC drivers and installed on my Window's 10 install of SQuirrel SQL and DBeaver. I got both of the drivers works sort of... On DBeaver I can execute and see queries in the tool but can't do much more than that. Overall looks like an issue with accessing the system table `sys.environment`, not sure if anything related to set up or rights. Using the `voc` user as created with the example db in the getting started. Any help or thought on where to start? Thanks! Sandy -------- Errors * DBeaver 4.0.8 On inspection of some of the logs I see things like - `Error loading object metadata from database` `SQL Error [22000]: SELECT: access denied for voc to table 'sys.environment' java.sql.SQLException: SELECT: access denied for voc to table 'sys.environment'` * SQuirreL SQL Client 3.7.1 java.sql.SQLException: SELECT: access denied for voc to table 'sys.environment' -------- Driver and DB versions I'm using this JAR for the drivers - C:\Users\Squidward\Downloads\monetdb-jdbc-2.25.jar, 2.25 (Liberica 20170330 based on MCL v1.15) MonetDB version - MonetDB v11.25.23 (Dec2016-SP5), 'mapi:monetdb://sandy-VirtualBox-18:50000/voc'

4 years, 9 months

Problem in windows OS

by Amit Ambekar

Hi, I am using monetdb since long time with linux OS, i just tried it with Windows 7 OS. But in windows after shut down machine, database is crashing. I am not able to use any query in mclient now for that databases. What is problem ?

4 years, 9 months

make[9]: *** No rule to make target `../../../../sql/backends/monet5/lib_sql.la', needed by `lib_myudfname.la'. Stop.

by Anton Kravchenko

Hi there, I will try to map my issue to something that you are familiar with. For example "MonetDB-11.23.3/sql/backends/monet5/vaults" module. 1) It has Makefile.ag which has (among other lines): lib__vault = { MODULE DIR = libdir/monetdb5 SOURCES = vault.c LIBS = ../../../../monetdb5/tools/libmonetdb5 \ ../../../../gdk/libbat $(curl_LIBS) } And that means that "libmonetdb.la" has to be created before "vaults" is built. 2) Let's say I have added "../../../../sql/backends/monet5/lib_sql" dependency lib__vault = { MODULE DIR = libdir/monetdb5 SOURCES = vault.c LIBS = ../../../../monetdb5/tools/libmonetdb5 \ * ../../../../sql/backends/monet5/lib_sql \* ../../../../gdk/libbat $(curl_LIBS) } As you can see "vaults" is leaving inside "MonetDB-11.23.3/sql/backends/monet5" directory. 3) Now my question is: How can one propagate into MonetDB-11.23.3/sql/backends/monet5/Makefile.am that "lib_sql.la" has to be created before "vaults" is built? p.s. On Windows, there is MonetDB-11.23.3/sql/backends/monet5/Makefile.msc used (instead Makefile.am). And I would just change order in "all-recursive all-msc" to "all-msc all-recursive": # build-all: $(BUILT_SOURCES) all-recursive all-msc build-all: $(BUILT_SOURCES) all-msc all-recursive Thank you, Anton

4 years, 9 months

build error MonetDBv11.27.5 on Linux

by Anton Kravchenko

Hi there, I am able to build and install MonetDB v11.21.19 "Jul2015-SP4" by following and example from https://www.monetdb.org/wiki/MonetDB:Building_from_sources But when I try build MonetDB v11.27.5 on same machine I get this error (for a full log info see https://drive.google.com/drive/folders/0B7dMycp0Ku-NTlRsZW95R0xCNmM?usp=sha… ): /bin/sh ../../../../../libtool --tag=CC --mode=link gcc -DLIBPYAPI -g -O2 -D_REENTRANT -module -avoid-version -o lib_pyapi.la -rpath /home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/lib/monetdb5 lib_pyapi_la-pyapi.lo lib_pyapi_la-unicode.lo lib_pyapi_la-pytypes.lo lib_pyapi_la-type_conversion.lo lib_pyapi_la-formatinput.lo lib_pyapi_la-connection.lo lib_pyapi_la-pyloader.lo lib_pyapi_la-emit.lo lib_pyapi_la-conversion.lo ../../../../../monetdb5/tools/libmonetdb5.la ../../../../../gdk/libbat.la -lpython2.7 -lpthread -ldl -lutil -lm -Xlinker -export-dynamic libtool: link: gcc -shared -fPIC -DPIC .libs/lib_pyapi_la-pyapi.o .libs/lib_pyapi_la-unicode.o .libs/lib_pyapi_la-pytypes.o .libs/lib_pyapi_la-type_conversion.o .libs/lib_pyapi_la-formatinput.o .libs/lib_pyapi_la-connection.o .libs/lib_pyapi_la-pyloader.o .libs/lib_pyapi_la-emit.o .libs/lib_pyapi_la-conversion.o -Wl,-rpath -Wl,/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/monetdb5/tools/.libs -Wl,-rpath -Wl,/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/gdk/.libs -Wl,-rpath -Wl,/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/clients/mapilib/.libs -Wl,-rpath -Wl,/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/common/stream/.libs -Wl,-rpath -Wl,/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/lib -L/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/gdk/.libs -L/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/clients/mapilib/.libs -L/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/common/stream/.libs ../../../../../monetdb5/tools/.libs/libmonetdb5.so /home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/gdk/.libs/libbat.so /home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/ clients/mapilib/.libs/libmapi.so -lpcre -lxml2 ../../../../../gdk/.libs/libbat.so /home/SECDEV.LOCAL/akravchenko /MonetDB-Jul2017-SP1/BUILD/common/stream/.libs/libstream.so -lsnappy -lssl -lcrypto -lz -lpython2.7 -lpthread -ldl -lutil -lm -g -O2 -Wl,-export-dynamic -pthread -Wl,-soname -Wl,lib_pyapi.so -o .libs/lib_pyapi.so /usr/bin/ld: cannot find -lpython2.7 collect2: ld returned 1 exit status make[11]: *** [lib_pyapi.la] Error 1 make[11]: Leaving directory `/home/SECDEV.LOCAL/akravchenk o/MonetDB-Jul2017-SP1/BUILD/sql/backends/monet5/UDF/pyapi' make[10]: *** [all] Error 2 make[10]: Leaving directory `/home/SECDEV.LOCAL/akravchenk o/MonetDB-Jul2017-SP1/BUILD/sql/backends/monet5/UDF/pyapi' make[9]: *** [all-recursive] Error 1 make[9]: Leaving directory `/home/SECDEV.LOCAL/akravchenk o/MonetDB-Jul2017-SP1/BUILD/sql/backends/monet5/UDF' make[8]: *** [all] Error 2 make[8]: Leaving directory `/home/SECDEV.LOCAL/akravchenk o/MonetDB-Jul2017-SP1/BUILD/sql/backends/monet5/UDF' make[7]: *** [all-recursive] Error 1 make[7]: Leaving directory `/home/SECDEV.LOCAL/akravchenk o/MonetDB-Jul2017-SP1/BUILD/sql/backends/monet5' make[6]: *** [all] Error 2 make[6]: Leaving directory `/home/SECDEV.LOCAL/akravchenk o/MonetDB-Jul2017-SP1/BUILD/sql/backends/monet5' make[5]: *** [all-recursive] Error 1 make[5]: Leaving directory `/home/SECDEV.LOCAL/akravchenk o/MonetDB-Jul2017-SP1/BUILD/sql/backends' make[4]: *** [all] Error 2 make[4]: Leaving directory `/home/SECDEV.LOCAL/akravchenk o/MonetDB-Jul2017-SP1/BUILD/sql/backends' make[3]: *** [all-recursive] Error 1 make[3]: Leaving directory `/home/SECDEV.LOCAL/akravchenk o/MonetDB-Jul2017-SP1/BUILD/sql' make[2]: *** [all] Error 2 make[2]: Leaving directory `/home/SECDEV.LOCAL/akravchenk o/MonetDB-Jul2017-SP1/BUILD/sql' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/home/SECDEV.LOCAL/akravchenk o/MonetDB-Jul2017-SP1/BUILD' make: *** [all] Error 2 Thank you, Anton

4 years, 10 months

Embedded Python is enabled but an error was thrown during initialization.

by Anton Kravchenko

Hi there, sql>create or replace function f1() returns table(v1 INT) more>LANGUAGE PYTHON { more> return [1,2,3] more>}; operation successful (3.658ms) sql>select * from f1(); Embedded Python is enabled but an error was thrown during initialization. Does one have to setup another python env variables (like it's done on MonetDB for Windows: PYTHONHOME, PYTHONPATH) on Linux? p.s. Here are is how I setup environment 1) installed MonetDB v11.27.5 (Jul2017-SP1) binaries on centos-release-7-4.1708.el7.centos.x86_64: sudo yum install https://dev.monetdb.org/downloads/epel/MonetDB-release-epel.noarch.rpm sudo yum install https://www.monetdb.org/downloads/epel/7/x86_64/MonetDB-stream-11.27.5-2017… sudo yum install https://www.monetdb.org/downloads/epel/7/x86_64/MonetDB-client-11.27.5-2017… sudo yum install https://www.monetdb.org/downloads/epel/7/x86_64/MonetDB5-server-11.27.5-201… sudo yum install https://www.monetdb.org/downloads/epel/7/x86_64/MonetDB-SQL-server5-11.27.5… sudo yum install https://www.monetdb.org/downloads/epel/7/x86_64/MonetDB-python2-11.27.5-201… 2) installed Anaconda2 Python 2.7.13 |Anaconda 4.4.0 (64-bit)| (default, Dec 20 2016, 23:09:15) [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2 export PATH="/home/akravchenko/anaconda2/bin:$PATH" 3) created and started Monet database: monetdbd create hnode1_dbfarm monetdbd set port=50001 hnode1_dbfarm monetdbd set control=yes hnode1_dbfarm monetdbd set passphrase=monetdb hnode1_dbfarm monetdbd start hnode1_dbfarm monetdb create hnode1_dbfarm_db1 *monetdb set embedpy=true hnode1_dbfarm_db1* monetdb start hnode1_dbfarm_db1 monetdb release hnode1_dbfarm_db1 4)monetdbd get all hnode1_dbfarm property value hostname somehostname dbfarm hnode1_dbfarm status monetdbd[31089] 1.7 (Jul2017-SP1) is serving this dbfarm mserver /usr/bin/mserver5 logfile hnode1_dbfarm/merovingian.log pidfile hnode1_dbfarm/merovingian.pid sockdir /tmp listenaddr localhost port 50001 exittimeout 60 forward proxy discovery yes discoveryttl 600 control yes passphrase {SHA512}a73f1d86383446438ac64f56e15ada38b41fbb18f029d2181723aeb2acac6a831f60e5fdbd64ac2c8c70e035dd44cbbe3b45565ef2d58feb2821a2078c7fad35 mapisock /tmp/.s.monetdb.50001 controlsock /tmp/.s.merovingian.50001 5) monetdb get all hnode1_dbfarm_db1 name prop source value hnode1_dbfarm_db1 name - hnode1_dbfarm_db1 hnode1_dbfarm_db1 type default database hnode1_dbfarm_db1 shared default yes hnode1_dbfarm_db1 nthreads default 32 hnode1_dbfarm_db1 optpipe default default_pipe hnode1_dbfarm_db1 readonly default no hnode1_dbfarm_db1 embedr default no hnode1_dbfarm_db1 embedpy local yes hnode1_dbfarm_db1 embedpy3 default no hnode1_dbfarm_db1 nclients default 64 hnode1_dbfarm_db1 dbextra default <unknown> Thank you, Anton

4 years, 10 months

SQL_CALC_FOUND_ROWS equivalent?

by Alberto Ferrari

Hi, is there any function or trick in Monet equivalent to mysql "SELECT SQL_CALC_FOUND_ROWS ..." to get de total rows when limit is present ? ((pagination) So I could get de TOTAL rows/pages number without exec another query everytime I need paginate results. Thanks!

4 years, 10 months

Re: Column rename issue

by Stefan Manegold

Hi, Add new column, copy/cast data from old to new column, drop old column. Stefan -------- Original message --------From: Amit Ambekar <amit(a)g-square.in> Date: 9/15/17 12:14 (GMT+01:00) To: users-list(a)monetdb.org Subject: Column rename issue Hi, how to update already created column name and type in monetdb ? _______________________________________________ users-list mailing list users-list(a)monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

4 years, 10 months

Column rename issue

by Amit Ambekar

Hi, how to update already created column name and type in monetdb ?

4 years, 10 months

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

users-list September 2017