January 2013 - users-list

MonetDB Cross Partition Joins
by Wolf Rödiger 17 Jan '13

17 Jan '13

Hello, According to its man page MonetDB is able to work as a cluster. However, I found no documentation on cross partition joins. Is there a way to partition a relation in MonetDB across servers and perform a cross partition join? Regards Wolf _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

2 1

JDBC error: 'protocol violation: unexpected line in data block: .'
by Luis Neves 11 Jan '13

11 Jan '13

Hello all, Is this a known bug? Is there any workaround? * Stack Trace java.sql.SQLException: 'protocol violation: unexpected line in data block: .' at nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.executeQuery(MonetConnection.java:2535) at nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.processQuery(MonetConnection.java:2284) at nl.cwi.monetdb.jdbc.MonetStatement.internalExecute(MonetStatement.java:508) at nl.cwi.monetdb.jdbc.MonetStatement.execute(MonetStatement.java:349) at nl.cwi.monetdb.jdbc.MonetPreparedStatement.execute(MonetPreparedStatement.java:247) at nl.cwi.monetdb.jdbc.MonetPreparedStatement.executeQuery(MonetPreparedStatement.java:265) * Table definition sql>\d sys.wa_blogs_audience CREATE TABLE "sys"."wa_blogs_audience" ( "tid" CHAR(32) NOT NULL, "ts" BIGINT, "ts_hour" BIGINT, "ts_day" BIGINT, "ts_month" BIGINT, "s_site_name" VARCHAR(32), "s_hostname" VARCHAR(256), "n_pageviews" DOUBLE, "n_visits" DOUBLE, "n_visitors" DOUBLE, "h_visitors_register" VARCHAR(1881), CONSTRAINT "wa_blogs_audience_tid_pkey" PRIMARY KEY ("tid") ); * SQL SELECT ts, hllagg(h_visitors_register) FROM wa_blogs_audience WHERE ts>=1355172393000 AND ts<=1357764393000 GROUP BY ts; * Output from mclient $ mclient -p9701 kpi_v1 -f raw -s "SELECT ts, hllagg(h_visitors_register) FROM wa_blogs_audience WHERE ts>=1355172393000 AND ts<=1357764393000 GROUP BY ts;" at: <https://dl.dropbox.com/u/279023/misc/mclient_output.txt> * JDBC debug log at: <https://dl.dropbox.com/u/279023/misc/monet_1357773164928.log> Thanks in advance! -- Luis Neves _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

2 6

Perf. benchmark for monetdb
by nitin sharma 10 Jan '13

10 Jan '13

Hi there, I am evaluating monetdb and was wondering if there are performance bench mark numbers like the one published in C-store paper: http://people.csail.mit.edu/tdanford/6830papers/stonebraker-cstore.pdf I am looking for perf. number for 100 millions rows in fact table (500 GB) joined with couple of dimension tables. Does moentdb supports horizontal partitioning of data and joins across machine (distributed joins) ? Thanks!Nitin _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

2 3

Weird concurrency issue with JDBC getConnection
by Percy Wegmann 10 Jan '13

10 Jan '13

Dear Monet Team, I've been using the JDBC driver and have recently noticed that under load, calls to DriverManager.getConnection() will sometimes appear to hang. I was able to resolve this problem by serializing access to getConnection() using a synchronized block. I haven't had a chance to dig through the JDBC driver code to see what would account for this, but I figured I'd make you aware just in case someone else runs into this. Cheers, -- Percy Wegmann +1 512 637 8500 ext 148 _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

3 4

What is normally done to recover from 'crash'ed databases.
by Brandon Jackson 08 Jan '13

08 Jan '13

Hello, I have a toy database on my laptop that I've been using while testing development of a Pentaho Data Integration plugin for bulk loading. One day I shutdown my machine without stopping monetdb. When I turned on the computer and started monetdbd on the database everything seems to work fine. 'monetdb status' outputs the following: brandon-jacksons-macbook-pro:bin bjackson$ ./monetdb status name state uptime health demo crashed 0%, 0s crash 2013-01-05 00:29:20 brandon-jacksons-macbook-pro:bin bjackson$ ./monetdb start demo starting database 'demo'... done brandon-jacksons-macbook-pro:bin bjackson$ ./monetdb status name state uptime health demo running 5s 50%, 0s crash 2013-01-05 00:29:20 brandon-jacksons-macbook-pro:bin bjackson$ I cannot really tell if there is any real damage. What do people normally do to check the integrity of their databases and repair damage if it exists? Does a set of actions restore the health of the database? If I missed some documentation please feel free to refer me to some links. Thanks for the information. Brandon _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

3 3

Fwd: Re: c3p0 ConnectionTester socket read stuck after SQLException
by Hannes Mühleisen 08 Jan '13

08 Jan '13

Forwarding this to the list, please keep the discussion there... -------- Original Message -------- Subject: Re: c3p0 ConnectionTester socket read stuck after SQLException Date: Tue, 8 Jan 2013 08:17:35 -0800 (PST) From: Tapomay Dey <tapomay(a)yahoo.com> Reply-To: Tapomay Dey <tapomay(a)yahoo.com> To: Hannes.Muehleisen(a)cwi.nl <Hannes.Muehleisen(a)cwi.nl> My operations are as follows: 1. Create connections with auto-commit=false 2. Read queries from an external module. 3. Execute queries on the connection acquired in step 1 4. Update external module status as per result of queries. 5. If all is well, commit the connection. 6. If commit fails(optimistic locking is the culprit here), revert changes to the external module. All of these steps are implemented in a command chain pattern code. Thus I need to be able to pass around the connection. Thanks and Regards, Tapomay. ------------------------------------------------------------------------ *From:* Hannes Mühleisen <Hannes.Muehleisen(a)cwi.nl> *To:* Communication channel for MonetDB users <users-list(a)monetdb.org> *Sent:* Thursday, January 3, 2013 4:19 PM *Subject:* Re: c3p0 ConnectionTester socket read stuck after SQLException Hello Tapomay, after looking at your code, I was wondering why you chose the double management of connections in your Main/JdbcMonetdbTestClient classes. C3P0 provides a way to define "connection customizers" (com.mchange.v2.c3p0.ComboPooledDataSource.setConnectionCustomizerClassName(String)), which you can use to disable autocommit on new connections. This should simplify your example considerably. Nevertheless, I will try to provoke the exceptions you have mentioned and see how the JDBC driver handles them together with C3P0. Best, Hannes On 20/12/12 15:41, Tapomay Dey wrote: > As the source code has many unrelated dependencies I tried to extract > the relevant code. > Please find attached an eclipse project for the same. > However I haven't been able to reproduce the issue in this test project. > It should give you a fair idea of the sequence of calls made. Plz note > that a single insert query used in the live project is much much larger > (10K approx.) than the one in the test. > I reconfirmed that the issue is still there in the larger project. > > PS: I had sent this code long back and it got moderated just now(due to > some 40KB limit). Thus I have removed the dependencies > monetdb-jdbc-2.6.jar and c3p0-0.9.1.2.jar from lib. Plz add them to lib > folder and everything should work(after u set the correct DB names in > the code).(Anyways this code doesn't reproduce the issue I am facing so > I guess just a manual overview should suffice) > > Regards, > Tapomay. > > ------------------------------------------------------------------------ > *From:* Hannes Mühleisen <Hannes.Muehleisen(a)cwi.nl <mailto:Hannes.Muehleisen@cwi.nl>> > *To:* Communication channel for MonetDB users <users-list(a)monetdb.org <mailto:users-list@monetdb.org>> > *Sent:* Thursday, December 20, 2012 2:17 PM > *Subject:* Re: c3p0 ConnectionTester socket read stuck after SQLException > > Hello Tapomay, > > I will have a look later today, could you please also provide the source > code that you have used to provoke the issue? > > Thanks, > > Hannes > > On 20/12/12 08:26, Tapomay Dey wrote: > > Please find the thread dump below. > > Running MonetDB Database Server v1.6 (Oct2012-SP1). Compiled > > from MonetDB-11.13.5 src distribution. > > C3P0 version: 0.9.1.2. Also tried with 0.9.2. > > Occurs only after SQLException sometimes: INSERT INTO xyz : table xyz > > doesn't exist. > > Otherwise it works great. > > Also tried setting c3p0's preferredQuery for ConnectionTester to "select > > 1;". Problem persists. > > > > Name: pool-15-thread-2 > > State: RUNNABLE > > Total blocked: 0 Total waited: 1 > > > > Stack trace: > > java.net.SocketInputStream.socketRead0(Native Method) > > java.net.SocketInputStream.read(SocketInputStream.java:150) > > java.net.SocketInputStream.read(SocketInputStream.java:121) > > java.io.BufferedInputStream.fill(BufferedInputStream.java:235) > > java.io.BufferedInputStream.read1(BufferedInputStream.java:275) > > java.io.BufferedInputStream.read(BufferedInputStream.java:334) > > - locked java.io.BufferedInputStream@5cec5796 > > > nl.cwi.monetdb.mcl.net.MapiSocket$BlockInputStream._read(MapiSocket.java:846) > > > nl.cwi.monetdb.mcl.net.MapiSocket$BlockInputStream.readBlock(MapiSocket.java:895) > > > nl.cwi.monetdb.mcl.net.MapiSocket$BlockInputStream.read(MapiSocket.java:965) > > sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283) > > sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325) > > sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177) > > - locked java.io.InputStreamReader@206a6883 > > java.io.InputStreamReader.read(InputStreamReader.java:184) > > java.io.BufferedReader.fill(BufferedReader.java:154) > > java.io.BufferedReader.readLine(BufferedReader.java:317) > > - locked java.io.InputStreamReader@206a6883 > > java.io.BufferedReader.readLine(BufferedReader.java:382) > > > nl.cwi.monetdb.mcl.io.BufferedMCLReader.readLine(BufferedMCLReader.java:119) > > > nl.cwi.monetdb.mcl.io.BufferedMCLReader.waitForPrompt(BufferedMCLReader.java:196) > > - locked nl.cwi.monetdb.mcl.io.BufferedMCLReader@29f38115 > > > nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.executeQuery(MonetConnection.java:2296) > > - locked nl.cwi.monetdb.mcl.net.MapiSocket@319f9f37 > > > nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.processQuery(MonetConnection.java:2274) > > > nl.cwi.monetdb.jdbc.MonetStatement.internalExecute(MonetStatement.java:508) > > nl.cwi.monetdb.jdbc.MonetStatement.execute(MonetStatement.java:349) > > nl.cwi.monetdb.jdbc.MonetStatement.executeQuery(MonetStatement.java:525) > > > com.mchange.v2.c3p0.impl.DefaultConnectionTester.activeCheckConnection(DefaultConnectionTester.java:73) > > > com.mchange.v2.c3p0.impl.DefaultConnectionTester.statusOnException(DefaultConnectionTester.java:143) > > > com.mchange.v2.c3p0.AbstractConnectionTester.statusOnException(AbstractConnectionTester.java:82) > > > com.mchange.v2.c3p0.impl.NewPooledConnection.handleThrowable(NewPooledConnection.java:437) > > - locked com.mchange.v2.c3p0.impl.NewPooledConnection@3340302c > > > com.mchange.v2.c3p0.impl.NewProxyStatement.executeUpdate(NewProxyStatement.java:78) > > > com.sokrati.moneta.mclient.JdbcMonetdbClient.executeUpdate(JdbcMonetdbClient.java:154) > > > com.sokrati.moneta.multiplexer.MonetdbClientCallable.call(MonetdbClientCallable.java:46) > > > com.sokrati.moneta.multiplexer.MonetdbClientCallable.call(MonetdbClientCallable.java:1) > > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > > java.util.concurrent.FutureTask.run(FutureTask.java:166) > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > > java.lang.Thread.run(Thread.java:722) > > > > Regards, > > Tapomay. > > > > > > _______________________________________________ > > users-list mailing list > > users-list(a)monetdb.org <mailto:users-list@monetdb.org> <mailto:users-list@monetdb.org <mailto:users-list@monetdb.org>> > > http://mail.monetdb.org/mailman/listinfo/users-list > > > > _______________________________________________ > users-list mailing list > users-list(a)monetdb.org <mailto:users-list@monetdb.org> <mailto:users-list@monetdb.org <mailto:users-list@monetdb.org>> > http://mail.monetdb.org/mailman/listinfo/users-list > > > > > _______________________________________________ > users-list mailing list > users-list(a)monetdb.org <mailto:users-list@monetdb.org> > http://mail.monetdb.org/mailman/listinfo/users-list > _______________________________________________ users-list mailing list users-list(a)monetdb.org <mailto:users-list@monetdb.org> http://mail.monetdb.org/mailman/listinfo/users-list _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

1 0

the dbname option disappeared
by Stefan de Konink 04 Jan '13

04 Jan '13

I have justed checked out the Feb2013 release and noticed that the dbname option is gone, is there a substitute? Stefan _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

5 8

c3p0 ConnectionTester socket read stuck after SQLException
by Tapomay Dey 03 Jan '13

03 Jan '13

Please find the thread dump below. Running MonetDB Database Server v1.6 (Oct2012-SP1). Compiled from MonetDB-11.13.5 src distribution. C3P0 version: 0.9.1.2. Also tried with 0.9.2. Occurs only after SQLException sometimes: INSERT INTO xyz : table xyz doesn't exist. Otherwise it works great. Also tried setting c3p0's preferredQuery for ConnectionTester to "select 1;". Problem persists. Name: pool-15-thread-2 State: RUNNABLE Total blocked: 0 Total waited: 1 Stack trace: java.net.SocketInputStream.socketRead0(Native Method) java.net.SocketInputStream.read(SocketInputStream.java:150) java.net.SocketInputStream.read(SocketInputStream.java:121) java.io.BufferedInputStream.fill(BufferedInputStream.java:235) java.io.BufferedInputStream.read1(BufferedInputStream.java:275) java.io.BufferedInputStream.read(BufferedInputStream.java:334) - locked java.io.BufferedInputStream@5cec5796 nl.cwi.monetdb.mcl.net.MapiSocket$BlockInputStream._read(MapiSocket.java:846) nl.cwi.monetdb.mcl.net.MapiSocket$BlockInputStream.readBlock(MapiSocket.java:895) nl.cwi.monetdb.mcl.net.MapiSocket$BlockInputStream.read(MapiSocket.java:965) sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283) sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325) sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177) - locked java.io.InputStreamReader@206a6883 java.io.InputStreamReader.read(InputStreamReader.java:184) java.io.BufferedReader.fill(BufferedReader.java:154) java.io.BufferedReader.readLine(BufferedReader.java:317) - locked java.io.InputStreamReader@206a6883 java.io.BufferedReader.readLine(BufferedReader.java:382) nl.cwi.monetdb.mcl.io.BufferedMCLReader.readLine(BufferedMCLReader.java:119) nl.cwi.monetdb.mcl.io.BufferedMCLReader.waitForPrompt(BufferedMCLReader.java:196) - locked nl.cwi.monetdb.mcl.io.BufferedMCLReader@29f38115 nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.executeQuery(MonetConnection.java:2296) - locked nl.cwi.monetdb.mcl.net.MapiSocket@319f9f37 nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.processQuery(MonetConnection.java:2274) nl.cwi.monetdb.jdbc.MonetStatement.internalExecute(MonetStatement.java:508) nl.cwi.monetdb.jdbc.MonetStatement.execute(MonetStatement.java:349) nl.cwi.monetdb.jdbc.MonetStatement.executeQuery(MonetStatement.java:525) com.mchange.v2.c3p0.impl.DefaultConnectionTester.activeCheckConnection(DefaultConnectionTester.java:73) com.mchange.v2.c3p0.impl.DefaultConnectionTester.statusOnException(DefaultConnectionTester.java:143) com.mchange.v2.c3p0.AbstractConnectionTester.statusOnException(AbstractConnectionTester.java:82) com.mchange.v2.c3p0.impl.NewPooledConnection.handleThrowable(NewPooledConnection.java:437) - locked com.mchange.v2.c3p0.impl.NewPooledConnection@3340302c com.mchange.v2.c3p0.impl.NewProxyStatement.executeUpdate(NewProxyStatement.java:78) com.sokrati.moneta.mclient.JdbcMonetdbClient.executeUpdate(JdbcMonetdbClient.java:154) com.sokrati.moneta.multiplexer.MonetdbClientCallable.call(MonetdbClientCallable.java:46) com.sokrati.moneta.multiplexer.MonetdbClientCallable.call(MonetdbClientCallable.java:1) java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) java.util.concurrent.FutureTask.run(FutureTask.java:166) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) java.lang.Thread.run(Thread.java:722) Regards, Tapomay. _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

2 6

Re: users-list Digest, Vol 5, Issue 1
by swapnil joshi 03 Jan '13

03 Jan '13

Hi nitin, # Serving database 'demo', using 12 threads # Compiled for x86_64-pc-winnt/64bit with 64bit OIDs dynamically linked # Found 23.998 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2012 MonetDB B.V., all rights reserved # Visit http://www.monetdb.org/ for further information !IOException:mal_mapi.listen: operation failed: bind to stream socket port 50000 failed: solution 1 = It's an just port conflict error the port had assign to monetdb is already used by any other application please change the port of monetdb or find your another application which using same port and sopt it's service, then it will work. Solution 2 = If you are an student and if you are using your own system / personal computer then just shut down the system and restart it and first start monetdb server. If any only if you are studying monetdb on your personal computer . (solution 2 is foolish solution :) ) On Thu, Jan 3, 2013 at 4:30 PM, <users-list-request(a)monetdb.org> wrote: > nitin -- Regards, Swapnil K. Joshi _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

1 0

Error while starting monetdb on windows server 2008 R2
by nitin sharma 03 Jan '13

03 Jan '13

Hi there, I am getting following error while running monetdb server >>>># MonetDB 5 server v11.13.7 "Oct2012-SP2" # Serving database 'demo', using 12 threads # Compiled for x86_64-pc-winnt/64bit with 64bit OIDs dynamically linked # Found 23.998 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2012 MonetDB B.V., all rights reserved # Visit http://www.monetdb.org/ for further information !IOException:mal_mapi.listen:operation failed: bind to stream socket port 50000 failed: No error > Any pointer what could be the issue? thx,Nitin _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

2 1