October 2015 - users-list

Plan for feature, external REMOTE table as a plugin (hadoop, jdbc, odbc, etc).
by Aris Setyawan 17 Oct '15

17 Oct '15

Hi, The key value of monetdb is it's parallel query. That many other opensource DBMS doesn't support it. But monetdb have it in the core engine. It is mean a mature feature. Having ability load (on the fly) any other DBMS data using external REMOTE table, and then query them using monetdb parallel query, will attract big community and contribution (as a plugin). The dbms, for example: jdbc, odbc, mysql, postgresql, hadoop, hive, or any commercial dbms. Any dbms which doesn't have parallel query feature, will benefit from this. Any plan for this feature? Regards, Aris

1 0

SQL join involving remote table.
by Aris Setyawan 17 Oct '15

17 Oct '15

Hi, According to https://www.monetdb.org/Documentation/Cookbooks/SQLrecipes/DistributedQuery… It is said that, "Queries involving remote tables are automatically split into subqueries by the master database and executed on remote databases.". I think this scenario: I have 3 server, one master server and two worker servers. I have 2 tables, A (big one, a fact table) and B (small one, a dimension table). A is a merge table of 2 remote tables. The remote table exist in two worker servers. B is a dimension table, that I'm not sure, on which server, I should place it on. And I will do queries, that join A and B. On which server, I should place table B to achieve distributed query processing? Is table B should be replicated on all remote table? Thanks, -Aris

2 2

JDBC Connection to a schema
by Vijay Krishna 16 Oct '15

16 Oct '15

Hi, I am trying to connect a schema within a database directly through JDBC with the following Java code. Connection con = DriverManager.getConnection( > "jdbc:monetdb://localhost/testUser", "monetdb", "monetdb"); > con.setSchema("schema1"); But this throws the following exception. > Exception in thread "main" java.sql.SQLException: syntax error, > unexpected '=' in: "set schema =" > at nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.executeQuery( > MonetConnection.java:2525) > at nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.processQuery( > MonetConnection.java:2273) > at nl.cwi.monetdb.jdbc.MonetStatement.internalExecute( > MonetStatement.java:497) > at nl.cwi.monetdb.jdbc.MonetStatement.execute(MonetStatement.java:338) > at nl.cwi.monetdb.jdbc.MonetStatement.executeUpdate( > MonetStatement.java:533) > at nl.cwi.monetdb.jdbc.MonetConnection.setSchema(MonetConnection.java:1267 > ) > at batcave.Monet.main(Monet.java:11) Is it really possible to connect to a specific schema? Or the only option is to query as "schemaName.tableName" ??? Thanks & Regards, Vijayakrishna.P. Mobile : (+91) 9500402305.

3 3

Baseline Performance Expectation?
by Paul Ramsey 15 Oct '15

15 Oct '15

Hi all, I’ve loaded a relatively small table (I think) of 30M records, 7 columns, and I’m doing some test queries summarizing and filtering the table, like this: select vendor_id, count(*) from taxi where geohash like '90efbc6a6fc%’ group by vendor_id; On my 4-core iMac, the CPU doesn’t get very hot (20% on top, so it’s not even lighting up one of the four CPUs) , but it’s hard to tell since the return time is only 2.5 second. Is that as fast as things will get? I doubled the table size and query times more-or-less doubled. I was just hoping to get query times in the < 1s range. Thanks, P -- Paul Ramsey Solutions Engineer CartoDB—Map Your World's Data 143 Roebling St 2A, New York, 11211, USA T: +1.917.463.3232 Skype: pwramsey3 W: www.cartodb.com Live sessions to learn CartoDB

2 1

Finding the current schema
by Enzo 14 Oct '15

14 Oct '15

Is there a query that would let me know what is the current schema in my db? E.g. I would expect as a response either sys (if I haven't set any schema), or foo (if I've set the schema foo). Kindest regards Enzo enzo(a)smartinsightsfromdata.com

1 0

Support for UUIDs in R and Python
by Doug Service 09 Oct '15

09 Oct '15

Hello everyone, I discovered this morning that the UUID type is not supported in MonetDB.R package on CRAN Error in monetdbRtype(info$types[i]) : Unknown DB type UUID and UUIDs are not support in the Python API which I installed from source File "/usr/local/lib/python3.4/dist-packages/monetdb/sql/pythonize.py", line 132, in convert raise ProgrammingError("type %s is not supported" % type_code) monetdb.exceptions.ProgrammingError: type uuid is not supported Will UUIDs be supported in R and Python in a future release? We use R extensively for statistical analysis on our data and would also like to use Python in certain cases. Doug

1 0

Re UUID and new crash problem
by Doug Service 09 Oct '15

09 Oct '15

Hello Stefan, I modified the uuid.mal file in the lib directory and that all works great for a small sample test from mclient. Thanks! Now I have run into a problem where the server crashes when I try to insert into one of the tables, though inserts in other tables work fine. I do not think this problem has anything with UUIDS as the table I am inserting into does not use UUIDS and I can do inserts into tables that have uuid columns. This insert that causes the crash worked fine on the last version of our schema that did not use UUIDs. I have a small database from msqldump and an insert statement that causes the server to crash consistently. However, I do not see any way to add attachments to a bug report on the tracker. Is there a way to do this? Is there any other information I can collect that would be useful in a bug report? Here is the monetdb log which shows a sigsegv. Doug 2015-10-06 16:02:19 MSG quantbox[3433]: # MonetDB 5 server v11.21.5 "Jul2015" 2015-10-06 16:02:19 MSG quantbox[3433]: # Serving database 'quantbox', using 4 threads 2015-10-06 16:02:19 MSG quantbox[3433]: # Compiled for x86_64-pc-linux-gnu/64bit with 64bit OIDs and 128bit integers dynamically linked 2015-10-06 16:02:19 MSG quantbox[3433]: # Found 1.907 GiB available main-memory. 2015-10-06 16:02:19 MSG quantbox[3433]: # Copyright (c) 1993-July 2008 CWI. 2015-10-06 16:02:19 MSG quantbox[3433]: # Copyright (c) August 2008-2015 MonetDB B.V., all rights reserved 2015-10-06 16:02:19 MSG quantbox[3433]: # Visit http://www.monetdb.org/ for further information 2015-10-06 16:02:19 MSG quantbox[3433]: # Listening for UNIX domain connection requests on mapi:monetdb:///var/lib/monetdb/quantbox/.mapi.sock 2015-10-06 16:02:19 MSG quantbox[3433]: # Start processing logs sql/sql_logs version 52200 2015-10-06 16:02:19 MSG quantbox[3433]: # Start reading the write-ahead log 'sql_logs/sql/log.4' 2015-10-06 16:02:19 MSG quantbox[3433]: # Finished reading the write-ahead log 'sql_logs/sql/log.4' 2015-10-06 16:02:19 MSG quantbox[3433]: # Finished processing logs sql/sql_logs 2015-10-06 16:02:19 MSG quantbox[3433]: # MonetDB/SQL module loaded 2015-10-06 16:02:19 MSG merovingian[809]: proxying client 10.1.10.101:40633 for database 'quantbox' to mapi:monetdb:///var/lib/monetdb/quantbox/.mapi.sock?database=quantbox 2015-10-06 16:02:19 MSG merovingian[809]: target connection is on local UNIX domain socket, passing on filedescriptor instead of proxying 2015-10-06 16:05:19 MSG merovingian[809]: proxying client (local) for database 'quantbox' to mapi:monetdb:///var/lib/monetdb/quantbox/.mapi.sock?database=quantbox 2015-10-06 16:05:19 MSG merovingian[809]: target connection is on local UNIX domain socket, passing on filedescriptor instead of proxying 2015-10-06 16:05:27 MSG merovingian[809]: proxying client (local) for database 'quantbox' to mapi:monetdb:///var/lib/monetdb/quantbox/.mapi.sock?database=quantbox 2015-10-06 16:05:27 MSG merovingian[809]: target connection is on local UNIX domain socket, passing on filedescriptor instead of proxying 2015-10-06 16:05:48 MSG merovingian[809]: database 'quantbox' (3433) was killed by signal SIGSEGV 2015-10-06 16:05:59 MSG merovingian[809]: starting database 'quantbox', up min/avg/max: 0s/0s/0s, crash average: 1.00 0.30 0.10 (3-0=3) 2015-10-06 16:06:00 MSG quantbox[3532]: arguments: /usr/bin/mserver5 --dbpath=/var/lib/monetdb/quantbox --set merovingian_uri=mapi:monetdb://qbdb:42000/quantbox --set mapi_open=false --set mapi_port=0 --set mapi_usock=/var/lib/monetdb/quantbox/.mapi.sock --set monet_vault_key=/var/lib/monetdb/quantbox/.vaultkey --set gdk_nr_threads=4 --set max_clients=64 --set sql_optimizer=default_pipe --set monet_daemon=yes

3 4

Problems implementing java jdbc
by Federico G Agripa 09 Oct '15

09 Oct '15

Hi, i'm having some issues implementing the java jdbc with a grails-app. Grails is a framework that runs on the jvm, allowing you to write all your code in java if you want. I'm using exactly the code you provide on your web page but it isn't recognizing a proper driver to connect to monet at runtime. I also tried to search some repo that provides a plugin or extension that allows me to use but i get the same error (the only one i found was www.clojars.com/monetdb), is there any example of a driver implementation you can provide or any forum to forward this? Thanks in advice

3 3

Grails integration
by Federico G Agripa 09 Oct '15

09 Oct '15

Hello, i'm working on a grails plugin to easily integrate with monetDB. So I would like to here what functionalities will be useful to you guys because I want it to be useful to all of you to. I'm implementing it from the monet jar 2.17 and I hope to have it up in grails plugins as soon as possible. Btw any collaboration will be really welcome Regards

1 0

Regex support
by imad hajj chahine 08 Oct '15

08 Oct '15

Hi, Monetdb is using pcre library for regular expression support, and using this function we can allow SQL to use the regex library: CREATE FUNCTION "pcre_replace" (origin string, pat string, repl string,flags string) RETURNS string EXTERNAL NAME pcre."replace"; I want to use regex for date conversion to the format supported by monetdb (yyyy-MM-dd), I have the following format dd/MM/yyyy, and i am trying to run the following with no success: select pcre_replace('15/10/2015', '^(3[01]|[12][0-9]|0?[1-9])/(1[012]|0?[1-9])/([0-9]{4})$', '\\3-\\2-\\1', 'i') select pcre_replace('15/10/2015', '^(3[01]|[12][0-9]|0?[1-9])/(1[012]|0?[1-9])/([0-9]{4})$', '{3}-{2}-{1}', 'i') select pcre_replace('15/10/2015', '^(?<day>3[01]|[12][0-9]|0?[1-9])/(?<month>1[012]|0?[1-9])/(?<year>[0-9]{4})$', '\\k<year>-\\k<month>-\\k<day>', 'i') My question is, is backreference supported in the replacement string; if yes, whats is the right syntax to use to reference the detected groups, if no, is there a clever way then using substring? Thanks

3 6