Hi all –
I have always used the COPY BINARY INTO … commands to load my 2.0 Billion row genetic data into a monetdb table. With 135 columns, it has been blindingly fast.
Last week I moved from the June2016-SP2 release to dec2016-SP2. My binary loads are taking WAY longer. I killed one after 3 hours (via “call sys.stop(pid)” so it could clean up properly). I then started the load again, thinking perhaps the problem was related to the new columns I was adding.
I have since dropped the table and remade it using the same data and scripts that worked in just over 3 minutes in February on the jun2016-SP2 load. It is really chugging along – I’m up to 30 minutes and counting. I don’t have access to the sql log files, but the Merovingian.log shows nothing.
I do notice that previously the binary files, once loaded, were removed from the loading directly. This does not happen now. Were these files previously “moved” and now they are copied?
Has anyone see this performance issue with Dec2016-SP2 COPY BINARY INTO …. Commands?
Thanks - Lynn
Just getting back into trying MonetDB after many years. Got the
environment set up pretty well with the test DB `VOC` loaded up and able
to do all expected in the command line tools. So I figured I would give
a couple of the mentioned GUI tools a shot and see how they work against
the server I just set up on a virtual box.
I picked up the JDBC drivers and installed on my Window's 10 install of
SQuirrel SQL and DBeaver. I got both of the drivers works sort of... On
DBeaver I can execute and see queries in the tool but can't do much more
than that.
Overall looks like an issue with accessing the system table
`sys.environment`, not sure if anything related to set up or rights.
Using the `voc` user as created with the example db in the getting
started. Any help or thought on where to start?
Thanks!
Sandy
-------- Errors
* DBeaver 4.0.8
On inspection of some of the logs I see things like -
`Error loading object metadata from database`
`SQL Error [22000]: SELECT: access denied for voc to table 'sys.environment'
java.sql.SQLException: SELECT: access denied for voc to table
'sys.environment'`
* SQuirreL SQL Client 3.7.1
java.sql.SQLException: SELECT: access denied for voc to table
'sys.environment'
-------- Driver and DB versions
I'm using this JAR for the drivers -
C:\Users\Squidward\Downloads\monetdb-jdbc-2.25.jar, 2.25 (Liberica
20170330 based on MCL v1.15)
MonetDB version - MonetDB v11.25.23 (Dec2016-SP5),
'mapi:monetdb://sandy-VirtualBox-18:50000/voc'
Hi,
I am using monetdb since long time with linux OS, i just tried it
with Windows 7 OS. But in windows after shut down machine, database is
crashing. I am not able to use any query in mclient now for that
databases. What is problem ?
Hi there,
I will try to map my issue to something that you are familiar with.
For example "MonetDB-11.23.3/sql/backends/monet5/vaults" module.
1) It has Makefile.ag which has (among other lines):
lib__vault = {
MODULE
DIR = libdir/monetdb5
SOURCES = vault.c
LIBS = ../../../../monetdb5/tools/libmonetdb5 \
../../../../gdk/libbat $(curl_LIBS)
}
And that means that "libmonetdb.la" has to be created before "vaults" is
built.
2) Let's say I have added "../../../../sql/backends/monet5/lib_sql"
dependency
lib__vault = {
MODULE
DIR = libdir/monetdb5
SOURCES = vault.c
LIBS = ../../../../monetdb5/tools/libmonetdb5 \
* ../../../../sql/backends/monet5/lib_sql \*
../../../../gdk/libbat $(curl_LIBS)
}
As you can see "vaults" is leaving inside
"MonetDB-11.23.3/sql/backends/monet5" directory.
3) Now my question is: How can one propagate into
MonetDB-11.23.3/sql/backends/monet5/Makefile.am that "lib_sql.la" has to be
created before "vaults" is built?
p.s. On Windows, there is MonetDB-11.23.3/sql/backends/monet5/Makefile.msc
used (instead Makefile.am). And I would just change order in "all-recursive
all-msc" to "all-msc all-recursive":
# build-all: $(BUILT_SOURCES) all-recursive all-msc
build-all: $(BUILT_SOURCES) all-msc all-recursive
Thank you,
Anton
Hi there,
sql>create or replace function f1() returns table(v1 INT)
more>LANGUAGE PYTHON {
more> return [1,2,3]
more>};
operation successful (3.658ms)
sql>select * from f1();
Embedded Python is enabled but an error was thrown during initialization.
Does one have to setup another python env variables (like it's done on
MonetDB for Windows: PYTHONHOME, PYTHONPATH) on Linux?
p.s. Here are is how I setup environment
1) installed MonetDB v11.27.5 (Jul2017-SP1) binaries
on centos-release-7-4.1708.el7.centos.x86_64:
sudo yum install
https://dev.monetdb.org/downloads/epel/MonetDB-release-epel.noarch.rpm
sudo yum install
https://www.monetdb.org/downloads/epel/7/x86_64/MonetDB-stream-11.27.5-2017…
sudo yum install
https://www.monetdb.org/downloads/epel/7/x86_64/MonetDB-client-11.27.5-2017…
sudo yum install
https://www.monetdb.org/downloads/epel/7/x86_64/MonetDB5-server-11.27.5-201…
sudo yum install
https://www.monetdb.org/downloads/epel/7/x86_64/MonetDB-SQL-server5-11.27.5…
sudo yum install
https://www.monetdb.org/downloads/epel/7/x86_64/MonetDB-python2-11.27.5-201…
2) installed Anaconda2
Python 2.7.13 |Anaconda 4.4.0 (64-bit)| (default, Dec 20 2016, 23:09:15)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
export PATH="/home/akravchenko/anaconda2/bin:$PATH"
3) created and started Monet database:
monetdbd create hnode1_dbfarm
monetdbd set port=50001 hnode1_dbfarm
monetdbd set control=yes hnode1_dbfarm
monetdbd set passphrase=monetdb hnode1_dbfarm
monetdbd start hnode1_dbfarm
monetdb create hnode1_dbfarm_db1
*monetdb set embedpy=true hnode1_dbfarm_db1*
monetdb start hnode1_dbfarm_db1
monetdb release hnode1_dbfarm_db1
4)monetdbd get all hnode1_dbfarm
property value
hostname somehostname
dbfarm hnode1_dbfarm
status monetdbd[31089] 1.7 (Jul2017-SP1) is serving this dbfarm
mserver /usr/bin/mserver5
logfile hnode1_dbfarm/merovingian.log
pidfile hnode1_dbfarm/merovingian.pid
sockdir /tmp
listenaddr localhost
port 50001
exittimeout 60
forward proxy
discovery yes
discoveryttl 600
control yes
passphrase
{SHA512}a73f1d86383446438ac64f56e15ada38b41fbb18f029d2181723aeb2acac6a831f60e5fdbd64ac2c8c70e035dd44cbbe3b45565ef2d58feb2821a2078c7fad35
mapisock /tmp/.s.monetdb.50001
controlsock /tmp/.s.merovingian.50001
5) monetdb get all hnode1_dbfarm_db1
name prop source value
hnode1_dbfarm_db1 name - hnode1_dbfarm_db1
hnode1_dbfarm_db1 type default database
hnode1_dbfarm_db1 shared default yes
hnode1_dbfarm_db1 nthreads default 32
hnode1_dbfarm_db1 optpipe default default_pipe
hnode1_dbfarm_db1 readonly default no
hnode1_dbfarm_db1 embedr default no
hnode1_dbfarm_db1 embedpy local yes
hnode1_dbfarm_db1 embedpy3 default no
hnode1_dbfarm_db1 nclients default 64
hnode1_dbfarm_db1 dbextra default <unknown>
Thank you,
Anton
Hi,
is there any function or trick in Monet equivalent to mysql "SELECT
SQL_CALC_FOUND_ROWS ..." to get de total rows when limit is present ?
((pagination)
So I could get de TOTAL rows/pages number without exec another query
everytime I need paginate results.
Thanks!
Hi,
Add new column, copy/cast data from old to new column, drop old column.
Stefan
-------- Original message --------From: Amit Ambekar <amit(a)g-square.in> Date: 9/15/17 12:14 (GMT+01:00) To: users-list(a)monetdb.org Subject: Column rename issue
Hi,
how to update already created column name and type in monetdb ?
_______________________________________________
users-list mailing list
users-list(a)monetdb.org
https://www.monetdb.org/mailman/listinfo/users-list