September 2011 - users-list

[MonetDB-users] EPS / SVG of new MonetDB logo
by Stefan de Konink 20 Sep '11

20 Sep '11

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi, The new logo is online on the new website for a while now, is there any EPS/SVG or other vector format available? Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEAREKAAYFAk547rYACgkQYH1+F2Rqwn14VwCgioRrllvOlEUwHJke6eLxdXca rm0An2CwkBr1KoUWmgYewB64U5B49GtS =ef+A -----END PGP SIGNATURE-----

2 3

[MonetDB-users] GUID
by Michal Dvorak 20 Sep '11

20 Sep '11

Hi MonetDB-Team, are you going to implement 128b GUID? If not, any idea how to bypass it? 2x Int64 or BLOB(128)? Thank you very much for your answer J Best regards, Michal

3 2

[MonetDB-users] Maximum Parallelism and Recommended Hardware for MonetDB
by brien colwell 20 Sep '11

20 Sep '11

I set up a MonetDB server for a data mining team on a single node. The node's specs are 32 cores (4x8 AMD 2.3GHz) 128GiB RAM, with the dbfarm on a RAID0 2xSSD (I think they are Intel SSDs). MonetDB on this hardware works well for aggregates and joins on integers, but chugs considerably on inline views with strings and joins with strings. The team usually has 2-3 concurrent jobs at any given time, some materializing new tables, with lots of bulk loading ("copy into ... from file ..."). A typical table size is 2GiB, and queries may join 5-10 tables. When I look at process activity on the node, I usually see each DB's mserver5 process pegged on CPU, using just a single core. Since I rarely see more than one core used per mserver5, I'm starting to think either I haven't configured MonetDB to favor parallel plans, or I chose the wrong hardware for MonetDB. So, as questions to everyone on the list, I'm curious 1) Are there knobs in MonetDB to favor parallel execution plans? 2) What's the ideal hardware for a single-node MonetDB server to support a workload of 2-5 concurrent queries? 3) What's the ideal disk configuration for the above? Brien

2 1

[MonetDB-users] How to compress a table
by Alexis Berlemont 20 Sep '11

20 Sep '11

Hi, I am currently trying to evaluate MonetDB as a solution to hold significant amounts of data. Because of limited storage capacities, I am trying to enable compression in MonetDB. I did not manage to find any tutorial on that subject, so I did not go very far. I cannot even tell that I wholly understand the problem. According to the page "Documentation->User Guide->compression", I understood that I had to inject a compression optimizer inside a pipeline. This "thing" is supposed to be possible via the setting of a sql environment variable named "optimizer". In the (generated) source file monetdb5/optimizer/opt_pipes.c, I found a default + compression pipeline. Then, in a mclient session, I typed: sql>set optimizer='inline,remap,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,constants,commonTerms,joinPath,deadcode,reduce,dataflow,compression,dataflow,history,multiplex,garbageCollector'; After that, I tried to use a sql function which was supposed to compress (thanks to the zlib) a table: sql.gzcompress sql>call gzcompress(myschema, mytable); Unfortunately, I am stuck here with the error message 'SELECT: identifier 'myschema' unknown'. I think I located the error message in "sql/server/rel_select.c" but I did not manage to deduce anything interesting. Do you have any idea ? Thanks in advance. Alexis.

2 1

[MonetDB-users] How to change in/out date format
by Eugene Prokopiev 20 Sep '11

20 Sep '11

Hi, is it possible to use "yy-MM-dd HH:mm:ss.SSS" date format via something like SET DATESTYLE in COPY INTO ... FROM ... with mclient to reuse existing files? -- Thanks, Eugene Prokopiev

2 1

[MonetDB-users] query time
by anagnostes 20 Sep '11

20 Sep '11

Hi. I would like to know if it's possible to get the query time using the input "\<file" and redirecting the output to "\>file" in Windows. When i use the input file it nevers throws the query time. So i can't know it. Thank you. -- View this message in context: http://old.nabble.com/query-time-tp32232809p32232809.html Sent from the monetdb-users mailing list archive at Nabble.com.

2 1

[MonetDB-users] Memory leaks in the latest trunk?
by Stefan de Konink 20 Sep '11

20 Sep '11

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hello, Recently I decided it was again a good idea(tm) to 'test' the latest MonetDB trunk on an updated dataset. Sadly over the last 6 days I observed a growing memory usage, today resulting in a segmentation fault. For now nothing more than mserver5[26142]: segfault at 8 ip 00007f1e1e592a31 sp 00007f1e00ca63b8 error 4 in libc-2.13.so[7f1e1e515000+17d000] is known to me, which is development wise totally useless. But an other question did arise: is testing currently taking memory into a count? Especially longer tests might show greater usage of memory. It is obviously no problem replay all the queries that ran against the database and valgrind it... but there might have been work done from development respective that can shed light over the problems. Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEAREKAAYFAk5OiAAACgkQYH1+F2Rqwn3d+ACdH5ySymgx46EON89qdnlYof39 ZHsAn1LjL9eOHncOjM99+wAupwMC6OR3 =BsVB -----END PGP SIGNATURE-----

2 1

[MonetDB-users] count slower than expected?
by Joris Slob 20 Sep '11

20 Sep '11

Hello mailing list! This is my first attempt to ask a question here, so please excuse my ignorance. For large tables (>1M rows), I encounter strange behavior when using the COUNT function in SQL in the mclient. When I do the following query: SELECT DISTINCT(predicate) FROM facts; I get a nice resultset of 31 tuples in 20ms. When I try the following query: SELECT COUNT(DISTINCT(predicate)) FROM facts; I get my answer in 565ms. It seems weird that the database doesn't 'optimize' this count to: SELECT COUNT(*) FROM (SELECT DISTINCT(predicate) FROM facts) AS b; I get my answer in 20ms. The facts table is defined as follows: CREATE TABLE "sys"."facts" ( "subject" CHAR(2048), "predicate" CHAR(2048), "object" CHAR(2048) ); This is my first attempt to make a knowledge store and I am aware there is lots I can do to optimize how I store the data. I was just surprised that the count took significantly longer than getting all the results out and counting them. Here is the version information of MonetDB I am using: MonetDB 5 server v11.3.7 "Apr2011-SP2" (32-bit, 32-bit oids) Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2011 MonetDB B.V., all rights reserved Visit http://monetdb.cwi.nl/ for further information Found 1.0GiB available memory, 2 available cpu cores Libraries: libpcre: 7.8 2008-09-05 (compiled with 7.8) openssl: OpenSSL 0.9.8k 25 Mar 2009 (compiled with OpenSSL 0.9.8k 25 Mar 2009) libxml2: 2.7.6 (compiled with 2.7.6) Compiled by: root(a)ottar.ins.cwi.nl (i486-pc-linux-gnu) Compilation: gcc -Wall -O6 -fomit-frame-pointer -finline-functions -falign-loops=4 -falign-jumps=4 -falign-functions=4 -fexpensive-optimizations -funroll-loops -frerun-cse-after-loop -frerun-loop-opt -ftree-vectorize Linking : /usr/bin/ld -Wl,-Bsymbolic-functions Greetings, Joris

2 1

[MonetDB-users] omission in documentation?
by Sander Kruseman 20 Sep '11

20 Sep '11

Hi, I'm unable to find documentation on 'mapi_open' on the new website. I read the 5 available man pages, but found no mention of it. The man page for monetdb does however, mention monetdb.conf(5) but I cannot find that one either. Am I overlooking something? regards, Sander Kruseman

2 1

[MonetDB-users] Announcement: New Apr2011-SP2 bugfix release of MonetDB suite
by Sjoerd Mullender 16 Sep '11

16 Sep '11

The MonetDB team at CWI/MonetDB BV is pleased to announce the Aug2011 feature release of the MonetDB suite of programs. The default width for OID columns in the database is now 64 bits for all 64-bit architecture. In releases up to this release, the default width on 64-bit Windows was still 32 bits. This means that with the current release it is possible to have tables with more than about 2 million rows even on Windows (as long as you use a 64-bit build). More information (including release notes) on this release is available at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. -- Sjoerd Mullender

1 0