users-list August 2010

users-list@monetdb.org

14 participants
9 discussions

[MonetDB-users] Joining Across Multiple Databases

by Hering Cheng

Hi, Is it possible to join two tables residing in different databases in a query? Here, by "database" I mean the result of running "monetdb create <database>". If so, what syntax should we use for the table names? Thanks. Hering Cheng

11 years, 11 months

[MonetDB-users] Bulk Data Extraction & Embedded mserver5

by Hering Cheng

Hi, I was wondering if you can give me some ideas on how to retrieve data from MonetDB at very high throughput. The query will be very simple, issued against a single table. This table contains billions of records and I'd like to retrieve the data at a rate of at least one million records per second. From my naive tests with mclient running on the same 16-core machine as mserver5, I am only able to extract the data at about 20,000 records per second with the Feb2010 release. As a baseline case, with the same data in compressed (gzip'd) ASCII form stored in a regular file on the same (SAN) file system as MonetDB, I am able to read at the desired speed of one million records per second. I understand that there is communication overhead between mserver5 and mclient (or whatever client I use). Therefore, one possibility is to embed my application within mserver5. The embedded application basically just needs to be able to issue a SQL (or even MAL) query against the enclosing mserver5 and process the result set. If this is a viable approach, I'd like some guidance on where the hooks are. Thanks. Hering Cheng

11 years, 11 months

[MonetDB-users] Installation error

by Alexander Rutkowski

Hi, I tried to install MonetDB on a Debian Linux system with ./monetdb-install.sh --prefix=/export/local-1/public/node/MonetDB --enable-sql --enable-xquery --enable-optimize but I got the error message ... test -z "/export/local-1/public/node/MonetDB/lib" || /bin/mkdir -p "/export/local-1/public/node/MonetDB/lib" /bin/sh ../../libtool --mode=install /usr/bin/install -c libstream.la '/export/local-1/public/node/MonetDB/lib' libtool: install: error: cannot install `libstream.la' to a directory not ending in /usr/local/lib make[6]: *** [install-streamLTLIBRARIES] Error 1 ... installing MonetDB failed I have no root rights so I need the --prefix argument. Thank you very much Alexander

11 years, 11 months

[MonetDB-users] Announcement: New Jun2010-SP2 bugfix release of MonetDB suite

by Sjoerd Mullender

The MonetDB team at CWI/MonetDB BV is pleased to announce the Jun2010-SP2 bug fix release of the MonetDB suite of programs. More information (including release notes) on this release is available at <http://monetdb.cwi.nl/Development/Releases/Jun2010/>. The download location is <http://dev.monetdb.org/downloads/>. -- Sjoerd Mullender

11 years, 11 months

[MonetDB-users] Response time

by Mitja Kleider

Hi, I am experiencing a huge difference between query response times of "mserver5 --readonly" and plain "mserver5". The difference is in the order of 25 sec vs 25 min. The disks are rather slow and there is a lot of disk write going on. (Not in readonly, obviously.) How can I investigate this further? Regards, Mitja

11 years, 11 months

[MonetDB-users] Problems with huge SQL queries.

by Manuel Mayr

Hello everyone, I'm currently using Monet5 for some experiments with ActiveRecord but some SQL queries seem to be too big for the default configuration of monet. The prominent characteristic of the query is a huge IN part in the WHERE clause (nearly 13Kb). $ mclient -d tpch01 -h localhost < q1.sql MAPI = monetdb@localhost:50000 ACTION= read_line QUERY = SELECT states.* FROM states WHERE (states.country_id IN (1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141 ERROR = !Connection terminated Is there a parameter that allows me to increase the SQL query buffer? Beste regards, Manuel PS.: I'm not sure if it is really the size of the SQL buffer since the query works when I modify it to run it against sys.tables, but it doesn't seem to return any result. Even prefix the query with "plan" doesn't come back within minutes. Maybe a performance bottleneck in the compiler or planner? -- Manuel Mayr (http://www-db.informatik.uni-tuebingen.de/team/mayr) Lehrstuhl Datenbanksysteme Wilhelm-Schickard Institut für Informatik Eberhard-Karls-Universität Tübingen

11 years, 11 months

[MonetDB-users] geom performance

by Mitja Kleider

Hi, I am having trouble with the geom module (Jun2010-SP1). A query constructing a Point(x,y) from longitude (x) and latitude (y) takes much longer than expected, see below: # MonetDB server v5.20.3, based on kernel v1.38.3 # Serving database 'germany3', using 2 threads # Compiled for x86_64-pc-linux-gnu/64bit with 64bit OIDs dynamically linked # MonetDB/SQL module v2.38.3 loaded # MonetDB/GIS module v0.18.3 loaded sql>\d nodes CREATE TABLE "sys"."nodes" ( "id" int NOT NULL DEFAULT next value for "sys"."s_nodes", "long" decimal(10,7), "lat" decimal(10,7), CONSTRAINT "pk_nodes_id" PRIMARY KEY ("id") ); sql>select long,lat from nodes limit 1; +--------------+--------------+ | long | lat | +==============+==============+ | 8.9766191 | 47.7913748 | +--------------+--------------+ 1 tuple (1.0s) sql>select point(8.9766191, 47.7913748); +------------------------------------------------+ | point_single_value | +================================================+ | POINT (8.9766191000000006 47.7913747999999998) | +------------------------------------------------+ 1 tuple (0.897ms) sql>select point(long,lat) from nodes limit 1; +------------------------------------------------+ | point_long | +================================================+ | POINT (8.9766191000000006 47.7913747999999998) | +------------------------------------------------+ 1 tuple (1m 23s) And executing something similar for the whole table caused a segfault after eating a lot of virtual memory: sql>ALTER TABLE nodes ADD geom POINT DEFAULT NULL; sql>UPDATE nodes SET geom = Point(long,lat) WHERE geom IS NULL; In case you would like to try yourself, here is the import script and the parser to get such a database: http://gitorious.org/osm-poi-tools/monetdb/blobs/master/scripts/import.sh http://gitorious.org/osm-poi-tools/monetdb/blobs/master/osmparser/src/osmpa… Thanks for your help, Mitja

11 years, 11 months

[MonetDB-users] Example of XRPC/Java application

by Ryad Ben-El-Kezadri

Hello, For whom interested by XRPC and Java GUI, I have posted a video http://www.youtube.com/watch?v=3WzM0ZOJw9k showing the great things that can be done with MonetDB/XQuery Regards Ryad PS: this is a networking application

11 years, 11 months

[MonetDB-users] How to log every sql query?

by Nicolas Lemay

Hello, I've been using MonetDB for quite some weeks, but I never find the way to log every sql query my database receive. I've tried googling this, but did not find a lot of docs about that. Is it possible to do this? If yes, how can I accomplish this? Thank you!

11 years, 12 months

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

users-list August 2010