September 2009 - users-list

[MonetDB-users] server crashes (may2009-sp2)
by Lazaros Koromilas 11 Sep '09

11 Sep '09

the scenario is: - create a large document with xmlgen: http://www.ins.cwi.nl/projects/xmark/Assets/unix.c - load it _updatable_ into monetdb/xquery - query many times with the current setup, it always segfaults the server at the 4th attempt. re-produce with: # xmlgen -f 1 > xmlgen-f1.xml -rw-r--r-- 112M 2009-09-11 18:24 xmlgen-f1.xml # echo 'pf:add-doc("'$PWD'/xmlgen-f1.xml", "xmlgen-f1", "xmlac", 10)' \ | mclient -lx # echo 'doc("xmlgen-f1")//africa//item' | mclient -lx > /dev/null # echo 'doc("xmlgen-f1")//africa//item' | mclient -lx > /dev/null # echo 'doc("xmlgen-f1")//africa//item' | mclient -lx > /dev/null # echo 'doc("xmlgen-f1")//africa//item' | mclient -lx > /dev/null MAPI = monetdb@localhost:50000 ACTION= read_line QUERY = doc("xmlgen-f1")//africa//item ERROR = Connection terminated other info: # uname -a Linux redrawn 2.6.30-ARCH #1 SMP PREEMPT Wed Sep 9 12:37:32 UTC 2009 i686 Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz GenuineIntel GNU/Linux # Mserver --version MonetDB server v4.30.4 (32-bit), based on kernel v1.30.4 (32-bit oids) Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2009 MonetDB B.V., all rights reserved Visit http://monetdb.cwi.nl/ for further information Configured for prefix: /usr/local Libraries: libpcre: 7.9 2009-04-11 (compiled with 7.9) openssl: OpenSSL 0.9.8k 25 Mar 2009 (compiled with OpenSSL 0.9.8k 25 Mar 2009) Compiled by: root@redrawn Compilation: gcc -O2 Linking : /usr/bin/ld i'll try the latest stable also. i think there was no problem with august2008 release. any help is appreciated. thanx, lazaros. -- http://www.ics.forth.gr/~koromil

1 0

Re: [MonetDB-users] COPY...INTO from remote
by Matthew Jones 10 Sep '09

10 Sep '09

The original suggestion was to use 'mclient' with the '<' redirect operator I'm assuming like this: mclient -lsql -s "COPY INTO table from STDIN DELIMITERS '\t';" < recordsfile but this doesn't work, it wants to interpret the contents of the file as a query... is there no way to batch load from a script from a remote monet instance? 73, Matthew W. Jones (KI4ZIB) http://matburt.net

3 2

[MonetDB-users] monetdb-install.sh: com.sun.tools.javac.Main is not on the classpath.
by Ying Zhang 09 Sep '09

09 Sep '09

Hi all, I logged in on another INS1 desktop 'vili', and try to install monetdb there with $monetdb-install.sh --enable-xquery --enable-optimize --nightly=stable --prefix="/export/scratch0/zhang/MonetDB" and it ends in: BUILD FAILED /var/tmp/MonetDB-1252495100/MonetDB-java/java/build.xml:94: The following error occurred while executing this line: /var/tmp/MonetDB-1252495100/MonetDB-java/java/build.xml:289: Unable to find a javac compiler; com.sun.tools.javac.Main is not on the classpath. Perhaps JAVA_HOME does not point to the JDK. It is currently set to "/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/ jre" While javac says: $ which javac /usr/bin/javac $ javac -version javac 1.6.0_0 Manually setting JAVA_HOME to my own java installation ends in the same error: $env JAVA_HOME=/ufs/zhang/install/java/current monetdb-install.sh -- enable-xquery --enable-optimize --nightly=stable --prefix="/export/ scratch0/zhang/MonetDB" The log is attached. Thanks in advance. Jennie

3 4

[MonetDB-users] Serious concurrency issues via python api
by Matthew Jones 09 Sep '09

09 Sep '09

I have multiple processes reading and writing to the same table and I've noticed really persistent concurrency problems: File "/usr/lib/python2.5/site-packages/monetdb/sql/cursors.py", line 208, in execute self.__store_result(block) File "/usr/lib/python2.5/site-packages/monetdb/sql/cursors.py", line 520, in __store_result self.__exception_handler(InterfaceError, "Unknown state, %s" % block) File "/usr/lib/python2.5/site-packages/monetdb/sql/cursors.py", line 574, in __exception_handler raise exception_class(message) InterfaceError: Unknown state, &2 65541 -1 !ERROR: COMMIT: transaction is aborted because of concurency conflicts, will ROLLBACK instead I'm not sure what the problem is, but it exists in such a way that all operations that load or update information in MonetDB have to be put into a loop to continuously retry. I'm using autocommit, but I would expect MonetDB to handle concurrency issues in a decent way, like maybe blocking a query until the other query finished. I'm not clobbering data, most of these are just bulk load operations that insert new rows. Is there any way to address this issue without having to putting retry loops around all of the code I'm writing to write to the database? 73, Matthew W. Jones (KI4ZIB) http://matburt.net

6 10

[MonetDB-users] readline/readline.h does not contain rl_completion_func_t
by Ying Zhang 08 Sep '09

08 Sep '09

Hello, I was trying to install monet on Mac OS 10.5 with monetdb-install.sh --enable-xquery --enable-optimize --nightly=stable It has always worked, but this time, it stops due to an error while configuring MonetDB: checking readline/readline.h usability... yes checking readline/readline.h presence... yes checking for readline/readline.h... yes checking for readline in -lreadline... yes checking for rl_completion_matches in -lreadline... no checking whether rl_completion_func_t exists... configure: error: readline/readline.h does not contain rl_completion_func_t, is it GNU readline? The complete log is in the attachment. What is wrong with my readline.h? How can I find out which readline.h the install script is checking, and which readline.h is expected / should be used? Thanks in advance. Jennie

2 3

[MonetDB-users] OpenSSL and SHA512
by Alex Bo. 08 Sep '09

08 Sep '09

Hi, I have compiled the latest August build from CVS on windows. When I run mclient, I get this: server requires 'SHA512' hash, but client support was not compiled in I saw another thread regarding this before, but couldn't resolve the issue based on it. Please let me know what I might be doing wrong. Here is my server output: # MonetDB server v5.14.0, based on kernel v1.32.0 # Serving database 'demo', using 1 thread # Compiled for i686-pc-win32/32bit with 32bit OIDs dynamically linked # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2009 MonetDB B.V., all rights reserved # Visit http://monetdb.cwi.nl/ for further information #warning: please don't forget to set your vault key! #(see C:\MonetCVS_base_Aug09\MonetCVS32_cleanBuild\Install\etc\monetdb5.conf) # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ > Thanks. -- View this message in context: http://www.nabble.com/OpenSSL-and-SHA512-tp25223356p25223356.html Sent from the monetdb-users mailing list archive at Nabble.com.

2 5

[MonetDB-users] using MonetDB for INEX data
by Erik Tjong Kim Sang 07 Sep '09

07 Sep '09

Dear MonetDB users, We are trying to add load the INEX XML Wikipedia data into MonetDB but we get some strange error messages: When adding documents: $ echo 'pf:add-doc("/data/erikt/inex2009/tmp/330.xml", "330")' | /storage/erikt/MonetDB/bin/mclient -lxquery MAPI = monetdb@localhost:50000 QUERY = pf:add-doc("/data/erikt/inex2009/tmp/330.xml", "330") ERROR = !ERROR: interpret_params: insert(param 3): invalid BAT. When querying the data: $ echo -e 'let $c := collection("330")\nfor $t in $c//article\nreturn($t//title/text())' | /storage/erikt/MonetDB/bin/mclient -lxquery ... text {"The Black Stallion (film)"}, text {"Halle, Germany"}, text {"FLCL"}, write error We get only 5% of the data that was expected from the query and then it crashes with the message "write error". Does anyone know what could cause these problems? Please respond to me personally (e.f.tjong.kim.sang(at)rug.nl) since I am not a member of this mailing list. Erik TKS -------- e-mail: e.f.tjong.kim.sang(a)rug.nl WWW: http://www.let.rug.nl/erikt/

2 2

[MonetDB-users] Segmentation fault
by Sten Hjelmqvist 07 Sep '09

07 Sep '09

Using MonetDB Aug 2009 on a Ubuntu Server 9.04 with 4Gbyte memory and 60Gbyte free disk space. I import a large database, ~1.5Gbyte , using \<filename. After a while the database exits with msg: ">Segmentation fault" Can I reconfigure the database to avoid this? --------------------------------------- Sten Hjelmqvist DataKonsult MERdata AB Sveavägen 33, 111 34 Stockholm Mobil : 0708 305616 E-mejl : sten.hjelmqvist(a)merdata.se <mailto:sten.hjelmqist@merdata.se> Webb : www.merdata.se <http://www.main.se/> ---------------------------------------

3 6

[MonetDB-users] XQUF: expected insert speed
by Ying Zhang 06 Sep '09

06 Sep '09

Hi all, Does anyone have an idea of the 'normal' insert speed (e.g., #nodes per sec, or #MBs per sec) of MonetDB/XQuery? I inserted a document of ~101mb into an xmark document of 2mb, and it took more than 10 minutes: $ mclient -lx -g -t xquery>do insert doc("persons.xml")/persons into doc("xmark2mb.xml")/ site more><> Trans 12.143 msec Shred 8.429 msec Query 5.869 msec Update 619019.583 msec Timer 619058.179 msec With the algebra back-end, it ends in an error (bug report will follow), but it is also not faster: $ mclient -lx -t xquery>do insert doc("persons.xml")/persons into doc("xmark2mb.xml")/ site more><> MAPI = monetdb@localhost:50000 QUERY = do insert doc("persons.xml")/persons into doc("xmark2mb.xml")/ site ERROR = !ERROR: interpret: no matching MIL operator to '-(lng, void)'. !MAYBE YOU MEAN: ! -(dbl) : dbl ! -(lng) : lng ! -(flt) : flt ! -(wrd) : wrd ! -(int) : int ! -(sht) : sht ! -(bte) : bte ! -(chr) : chr ! -(lng, dbl) : dbl ! -(lng, flt) : dbl ! -(lng, lng) : lng ! -(lng, wrd) : lng ! -(lng, int) : lng ! -(lng, sht) : lng ! -(lng, bte) : lng ! -(lng, chr) : lng ! -(dbl, dbl) : dbl ! -(dbl, flt) : dbl ! -(dbl, lng) : dbl ! -(dbl, wrd) : dbl ! -(dbl, int) : dbl ! -(dbl, sht) : dbl ! -(dbl, bte) : dbl ! -(dbl, chr) : dbl ! -(flt, dbl) : dbl ! -(flt, flt) : flt ! -(flt, lng) : flt ! -(flt, wrd) : flt ! -(flt, int) : flt ! -(flt, sht) : flt ! -(flt, bte) : flt ! -(flt, chr) : flt ! -(wrd, dbl) : dbl ! -(wrd, flt) : flt ! -(wrd, lng) : lng ! -(wrd, wrd) : wrd ! -(wrd, int) : wrd ! -(wrd, sht) : wrd ! -(wrd, bte) : wrd ! -(wrd, chr) : wrd ! -(int, dbl) : dbl ! -(int, flt) : flt ! -(int, lng) : lng ! -(int, wrd) : wrd ! -(int, int) : int ! -(int, sht) : int ! -(int, bte) : int ! -(int, chr) : int ! -(sht, dbl) : dbl ! -(sht, flt) : flt ! -(sht, lng) : lng ! -(sht, wrd) : wrd ! -(sht, int) : int ! -(sht, sht) : sht ! -(sht, bte) : sht ! -(sht, chr) : sht ! -(bte, dbl) : dbl ! -(bte, flt) : flt ! -(bte, lng) : lng ! -(bte, wrd) : wrd ! -(bte, int) : int ! -(bte, sht) : sht ! -(bte, chr) : bte ! -(bte, bte) : bte ! -(chr, dbl) : dbl ! -(chr, flt) : flt ! -(chr, lng) : lng ! -(chr, wrd) : wrd ! -(chr, int) : int ! -(chr, sht) : sht ! -(chr, bte) : bte ! -(chr, chr) : chr Timer 632991.583 msec xquery>\q <persons> is the root node of the document 'persons.xml', which contains 234599 xmark <person> nodes. So, actually, only one large node is inserted. It looks quite slow. Is this the expected behaviour or did I do something wrong? I'm using the latest stable version of MonetDB/XQuery, compiled and installed with 'monet-install.sh --enable-xquery --enale-optimize -- nightly=stable --prefix=...'. The document 'xmark2mb.xml' was shredded as an updatable doc. with 10% holes, and the document 'persons.xml' was shredded as a read-only doc. Both documents were already loaded, by executing the query 'count(doc("[docname]")// person)'. MonetDB/XQuery is running on my CWI PC (i.e., FC10, Intel(R) Core(TM)2 Quad CPU Q6600 @2.40GHz, 8GM RAM). During the execution, one CPU is constantly busy for 100%, Mserver uses 22% ~ 27% of MEM (according to 'top'). Does anyone have an idea how to get insert faster? Regards, Jennie

1 0

[MonetDB-users] Long query response time with 'in' statement
by mkurtulus 02 Sep '09

02 Sep '09

Hi, I'm trying to create a benchmark MonetDB vs Postgres and for many types of queries, MonetDB quite faster than Postgres.But there is an interesting result when i try a query which contains an 'in' statement.The response times are MonetDB - 58 sec Postgres - 3 sec for the query : select RFOADV_10.rfoadvsup as c0, sum((case when cabact___rfountide = 'NB_E' then cabactqte else 0 end)) as m0 from RFOADV as RFOADV_10, CABACT as CABACT where (cabact___rforefide = 'FHSJ' and cabact___rteprcide = 'CPTANA' and cabact___rtestdide = '100' and cabact___rfovsnide = '200805_001') and CABACT.cabact_c2rfodstide = RFOADV_10.rfoadvinf and RFOADV_10.rfoadvsup in ('5030', '5031', '5032', '5033', '5034', '5035', '5036', '5037', '5038') group by RFOADV_10.rfoadvsup i reform the query so that it can give the same result without 'in' : select RFOADV_10.rfoadvsup as c0, sum((case when cabact___rfountide = 'NB_E' then cabactqte else 0 end)) as m0 from RFOADV as RFOADV_10 join CABACT on (CABACT.cabact_c2rfodstide = RFOADV_10.rfoadvinf) join (select '5031' as pole UNION select '5032' as pole UNION select '5033' as pole UNION select '5034' as pole UNION select '5035' as pole UNION select '5036' as pole UNION select '5037' as pole UNION select '5038' as pole ) sub ON (RFOADV_10.rfoadvsup = sub.pole) where (cabact___rforefide = 'FHSJ' and cabact___rteprcide = 'CPTANA' and cabact___rtestdide = '100' and cabact___rfovsnide = '200805_001') group by RFOADV_10.rfoadvsup the response time for MonetDB decreases 100 times (0.6 sec). Do you have any idea about it ? I run both Monetdb and Postgres on a server with 10GB Ram and the tables have CABACT : 677000 RFOADV : 140000 rows. Thanks, Mehmet -- Open WebMail Project (http://openwebmail.org)

2 5