March 2011 - users-list

[MonetDB-users] Recycler Optimizer
by Henry Addington 24 Mar '11

24 Mar '11

Hello: I understand some of the optimizers/pipelines are experimental as indicated in the monetdb5.conf file. Has anyone used the Recycle pipeline for Skyserver? Would you mind sharing your experience, in terms of performance improvement and whether it can bet used in a production environment? We are using the Oct2010-SP1 release for a db that is read-only. I would guess if there are issues, I would think the problems would likely show up in read/write systems. Any feedback would be appreciated.... It looks like the Recycle pipeline for Skyserver pipeline is based on the Nov-2009 pipeline. Has anyone tried the recycler optimizer with the Feb2010 default pipeline? I was going to add "recycle" between "deadcode" and "reduce" in the default pipeline and see if it would work. Thanks. Henry

3 4

[MonetDB-users] ask for help
by Xianmin Liu 23 Mar '11

23 Mar '11

Hi, It's me again. Recently, I was doing some experiments using MonetDB/XQuery. And I asked one question about errors when loading XML data some days ago, Peter solved my problem very kindly. Now, there is an error again, pls help me. Following is my setting, CPU:Intel Core2 Duo CPU 2.93GHz, RAM:2.00GB System: Ubuntu 10.04 64bits MonetDB: setup by deb, serverV4.40.3 GDKv1.40.3, MonetDB/XQuery module 0.40.3 loaded. I used pf:add-doc to add data, the data size is 3GB(generated by xmark generator with factor 30), error returned: ERROR: !xquery> -- Xianmin Liu

2 1

[MonetDB-users] User-defined MAL functions
by Petros Tsialiamanis 22 Mar '11

22 Mar '11

Hi, I have some questions about using user-defined MAL functions I would like to create a function in MAL. 1) Is it possible that the function can take variable number of BATs as argument like in the case of algebra.joinPath(l:bat[:any,:any]... ) ? 2) If yes, how I can get the number of BATs passed as argument and 3) how I can get the n-th element of the argument? Thank you in advance Petros

2 1

[MonetDB-users] Some questions
by Henning Grosch 22 Mar '11

22 Mar '11

Hi everyone, I have a some questions about using monetDB. First thing I noticed when using extract on a timestamp field I can only cast the timestamp as year, month, day and so on. Is it somehow possible to extract the week of the year? Week starts with monday. Another thing is, can i use if statements in a query like i could in mysql? And the last one, i already searched in the archiv and found a post about win 32 odbc driver and column names which appear as questionmarks or other symbols like when using a wrong encoding, does anybody has the same problem or knows a solution? Regards Henning

1 0

Re: [MonetDB-users] ask for help
by Peter Boncz 21 Mar '11

21 Mar '11

Hi, You cannot load the 1G of data in the 32-bits windows version on MXQ. Simply won’t fit the virtual address space. Use the 64-bits version (best: use the 64-bits Linux version). Peter From: Xianmin Liu [mailto:xianmliu@gmail.com] Sent: Monday, March 21, 2011 2:11 PM To: monetdb-users(a)lists.sourceforge.net Cc: Nan Tang; Wenfei Fan Subject: [MonetDB-users] ask for help Hi, I got some errors when using MonetDB/XQuery, could you pls give me some help? I was running experiments over large XML data using MonetDB/XQuery. My hardware: CPU:Intel Core2 Duo CPU 2.93GHz, RAM:2.00GB MonetDB: Server v4.32.0 based on GDK v1.32.0, Compiled for i686-pc-win32/32bit, MonetDB/XQuery module v0.28.3 loaded (default back-end is 'algebra') When I try to load xmark1G.xml (generated by xmark generator with factor 10) by command 'pf:add-doc("H:\lxm\1G\xmark1000M.xml","xmark1G.xml")<>' Return errors: ERROR = !ERROR: [shred_url]: 1 times inserted nil due to errors at tuples 0@0. !ERROR: [shred_url]: first error was: !ERROR: HEAPextend: failed to extend to 278712278 for 03\373theap !ERROR: shredBAT_append_str: APPEND-STR[_prop_text]( !ERROR: CMDshred_url: operation failed. -- Xianmin Liu

1 0

[MonetDB-users] ask for help
by Xianmin Liu 21 Mar '11

21 Mar '11

Hi, I got some errors when using MonetDB/XQuery, could you pls give me some help? I was running experiments over large XML data using MonetDB/XQuery. My hardware: CPU:Intel Core2 Duo CPU 2.93GHz, RAM:2.00GB MonetDB: Server v4.32.0 based on GDK v1.32.0, Compiled for i686-pc-win32/32bit, MonetDB/XQuery module v0.28.3 loaded (default back-end is 'algebra') When I try to load xmark1G.xml (generated by xmark generator with factor 10) by command 'pf:add-doc("H:\lxm\1G\xmark1000M.xml","xmark1G.xml")<>' Return errors: ERROR = !ERROR: [shred_url]: 1 times inserted nil due to errors at tuples 0@0. !ERROR: [shred_url]: first error was: !ERROR: HEAPextend: failed to extend to 278712278 for 03\373theap !ERROR: shredBAT_append_str: APPEND-STR[_prop_text]( !ERROR: CMDshred_url: operation failed. -- Xianmin Liu

1 0

[MonetDB-users] EXTRACT(SECOND FROM timestamp) gives 0 seconds
by spamme.gs＠googlemail.com 18 Mar '11

18 Mar '11

Hello, I can't understand the following behaviour : Welcome to mclient, the MonetDB/SQL interactive terminal (Oct2010-SP1) Database: MonetDB v5.22.3, 'tradedb' Type \q to quit, \? for a list of available commands auto commit mode: on sql>select extract(second from timestamp '2011-3-10 9:30:42.246'); +---------------------+ | second_single_value | +=====================+ | 0.246 | +---------------------+ Whoops. Well maybe monetdb can EXPLAIN what's goin on sql>explain select extract(second from timestamp '2011-3-10 9:30:42.246'); | function user.s2_1{autoCommit=true}():void; | | sql.mvc(); | | _4 := mtime.milliseconds(2011-03-10 09:30:42.246:timestamp); | | sql.exportValue(1,".","second_single_value","decimal",9,3,8,_4,""); | | end s2_1; | Ok ... Looks like this function is declared here: (modules/atoms/mtime.mal) 413 command milliseconds(d:daytime) :int 414 address MTIMEdaytime_extract_milliseconds And digging into the source of _that_ function shows us that it's just returning the millisecond part. So, any chance of a quick fix here? It looks like I'm not the first to ask this question: http://sourceforge.net/mailarchive/message.php?msg_id=23238837 but that poor guy was completely ignored.

2 1

[MonetDB-users] LC_ALL=C in monetdb-install.sh breaks RDoc
by Lazaros Koromilas 18 Mar '11

18 Mar '11

Hello, all! I did the following on an Arch Linux yystem: MonetDB-Oct2010-SuperBall-SP1> ./monetdb-install.sh --enable-xquery ... make[8]: Entering directory `/var/tmp/MonetDB-1299754542/MonetDB-client/clients/ src/ruby/adapter' mkdir -p /home/myuser/MonetDB/lib/ruby/gems/1.9.1 gem install --local --install-dir /home/myuser/MonetDB/lib/ruby/gems/1.9.1 --forc e --rdoc activerecord-monetdb-adapter-0.1.gem ERROR: While executing gem ... (Encoding::UndefinedConversionError) U+2019 from UTF-8 to US-ASCII Successfully installed activerecord-monetdb-adapter-0.1 1 gem installed Installing ri documentation for activerecord-monetdb-adapter-0.1... Installing RDoc documentation for activerecord-monetdb-adapter-0.1... make[8]: *** [install-exec-local-activerecord-monetdb-adapter-0_1_gemspec] Error 1 ... Setting LC_ALL to UTF-8 fixes my problem. --- monetdb-install.sh.orig 2011-03-10 13:34:58.000000000 +0200 +++ monetdb-install.sh 2011-03-10 13:35:05.000000000 +0200 @@ -108,7 +108,7 @@ : ${FETCHCOMMAND:="wget --progress=dot:mega -t 5 --passive-ftp -O \"\${FILE}\" \"\${URI}\""} # make sure we get English error messages and so on -export LC_ALL=C +export LC_ALL="en_US.UTF-8" # parse command line options ORGARG=$*

2 3

[MonetDB-users] MonetDB/XQuery order by problem
by Rob Fynes 13 Mar '11

13 Mar '11

Hello, I'm trying to produce a sorted list of results with MonetDB/XQuery using 'order by', with a query that goes something like this: for $cname in pf:collections()/text() return if (local:is-date-between($cname, **start_date_here**, **end_date_here**)) then for $s in pf:collection($cname)//Dublin/stations[time/hour=13 and time/minute>=22 and time/minute<=25] order by $s/timeOfDay return $s else () ...the 'is-date-between' is just some custom function to check if the $cname (collection names are dates) is between the start and end dates supplied to the function. However, the problem is with the nested for loop. I want each $s that satisfies the XPath expression to be returned, but ordered by $s/timeOfDay. When I submit this query, I get the following error: ERROR = !type error: [err:XPTY0004] Orderspec requires at most one atomic item (given: string*). I've tried a number of variations on this, but the error is always the same. If I remove the order by clause, the results will return correctly, but in the unsorted order that they appear in the database. I'm still a beginner with XQuery, so I'm unsure where I'm going wrong, and documentation and tutorials have been unhelpful so far. Any help is appreciated! Regards, -Rob

2 3

[MonetDB-users] Copy records into table failed
by Tibor Barna 11 Mar '11

11 Mar '11

Hello, I am trying to import data from MySQL into MonetDB usig the bulk loader, the csv has ~11GB, 163Mil Rows. When running: sql>copy 163715240 records into sys.clicks from '/data/csv_monet_dump/clicks.tbl' using delimiters '|',' more>', '"' null as ''; I always get this error message: SQLException:importTable:failed to import table ERROR: HEAPextend: failed to extend to 654860960 for 32/3231tail ERROR: TABLETcreate_bats: Failed to create bat of size 163715240 merovingian.log: 2011-03-10 14:12:31 MSG performance[6281]: #GDKmmap(2383544320) fails, try to free up space [memory in use=41411528,virtual memory in use=12310675456] 2011-03-10 14:12:31 MSG performance[6281]: #GDKmmap(2383544320) result [mem=32087208,vm=12310675456] I am using 64bit openSUSE 11.3, 12GB RAM, 4GB Swap. Does anybody know how can i overcome this error? Any help would be most welcome :) Cherers, Tibor. -- View this message in context: http://old.nabble.com/Copy-records-into-table-failed-tp31115783p31115783.ht… Sent from the monetdb-users mailing list archive at Nabble.com.

3 7