users-list January 2013

users-list@monetdb.org

29 participants
29 discussions

Dynamic where clause in a stored procedure

by Vassilis Papakonstantinou

Hi all, Is it possible in monetdb to use a dynamic where clause in a selection query inside a stored procedure? More specifically, i want to pass the clause as parameter. Thank you, Vassilis

9 years, 5 months

[help] I have a simple question about monetdb query plan.

by 이수호

p{margin:0;padding:0;} Hello, I'm a student of Korea Advanced Institute of Science and Technology in Korea. I want to compare late materialization with join and invisible join on monetdb. But when i typed SQL query, i couldn't choose which query plan to use. There was no choice to me. I think query optimizer choose some join method. Can I choose join method? or control query plan? I installed Monetdb on my desk-top and followed tutorials. Everything is done. There is no error. In this situation, i tried to control query plan. So I read reference on official homepage. But I couldn't find the information that how to control query plan. Can I control query plan? If I can, I really really want to know how do that. Where can i find some example? Please give me your favor.

9 years, 5 months

caught SIGTERM, starting shutdown sequence

by Tapomay Dey

MSG merovingian[32005]: caught SIGTERM, starting shutdown sequence 2013-01-29 11:40:53 MSG control[32005]: control channel closed 2013-01-29 11:41:06 MSG merovingian[32005]: caught SIGTERM, starting shutdown sequence 2013-01-29 11:41:40 MSG merovingian[32005]: caught SIGTERM, starting shutdown sequence 2013-01-29 11:41:51 MSG merovingian[32005]: caught SIGTERM, starting shutdown sequence Every time I do a monetdbd stop {path_to_farm} I keep getting this in merovingian logs. But the monetdbd process never ends. This issue was not there on the same installation of monetdb initially. This problem started occurring annoyingly frequently after I renamed my last farm and created a fresh one.(with monetdbd not running) What leads me to stop the farm is that I am unable to connect to the db using mclient(There's another app accessing the DB using JDBC-c3p0). There's is no complaint in the merovingian log. The mclient command just hangs. I get back access once I force kill the monetdbd using kill -9. But it gets back into this state after some time. Running on Feb13 branch some older revision. Ubuntu 12.10 with lots of resources. Is this a known issue. Is there a workaround/config or something.

9 years, 6 months

best way to get autogenerated primary key

by Mate Birkas

Hi, this is my first post to this list, so forgive me if this is not the right place to post this. I tried to do my research (google, monetdb.org, python api help) but didn't find an answer. How can I get the auto-generated primary key (integer) after executing an insert. I want to use it in a second insert to another table that references the first. (in PostgreSQL I used the RETURNING clause for this) thanks in advance Mate

9 years, 6 months

MonetDB crash / termination issue.

by Brian Hood

Hi All, I'm not quite sure if i'm doing something wrong here but what is see if i performing a 2 table join on a primary key field is tablea and tableb work fine yet tablec which has less data crashes the daemon. SELECT a.ip_dst as ip_dst, b.tcp_dport as tcp_dport, count(*) as num from "myschema".tablea a LEFT JOIN "myschema".tableb b ON (a.guid = b.guid) WHERE b.tcp_dport IS NOT NULL GROUP BY a.ip_dst, b.tcp_dport ORDER BY num DESC LIMIT 20; This one works fine. However when i perform the join from tablea to tablec I get the following. sql>SELECT a.ip_dst as ip_dst, b.udp_dport as udp_dport, count(*) as num from "myschema".tablea a LEFT JOIN "myschema".tablec b ON (a.guid = b.guid) WHERE b.udp_dport IS NOT NULL GROUP BY a.ip_dst, b.udp_dport ORDER BY num DESC LIMIT 20 more>; Connection terminated I've added alot more detail to the pastebin link. I had the same issue with 11.13.5 after upgrading i have run into the same thing also. http://pastebin.com/m4XseSnH If you want to schema information to reproduce please just email me off list. Regards, Brian Hood

9 years, 6 months

Strange concurrency issue

by Percy Wegmann

Dear MonetDB Team, We're starting to exercise our MonetDB installation more heavily now and I've run into a strange issue where I get an occasional "COMMIT: failed" message. It seems that the conditions required to produce this are: - Concurrently inserting into two different tables - At the same time, issuing a SELECT COUNT(*) against either of those tables I've logged a bug with more detailed information, including a Java test case: http://bugs.monetdb.org/show_bug.cgi?id=3210 I'm only just starting my investigation, but I suspect that COUNT(*) may be the culprit, since other selects like SELECT * don't seem to have this problem. Does this ring any bells? Are there any special optimizations for COUNT(*) that bypass the underlying BAT and/or have side effects on some shared table somewhere that could be causing a concurrency violation? Thanks, -- Percy Wegmann +1 512 637 8500 ext 148 _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list

9 years, 6 months

Bit functions (XOR and inverse)

by Eduardo Oliveira

Hi, I'm running MonetDB v11.13.9 (Oct2012-SP3), i have some doubts about bit functions, i was trying the bit functions looking at MySQL docs in MonetDB: http://dev.mysql.com/doc/refman/5.0/en/bit-functions.html And when i used ^, the pow was calculated instead of XOR, the ~ don't work as expected too. I google and I found this http://permalink.gmane.org/gmane.comp.db.monetdb.scm/28212 This patch will be applied on MonetDb, any date of release of MonetDb with ^ already as XOR? Right now what operator should i used to make a xor? I tested xor XOR and MonetDb didn't recognize. If i want to insert in binary form how i do? b'binary' works? Thanks in advance, -- *Eduardo Oliveira * *IT* ***Email:* eduardo.oliveira(a)adclick.pt *Web: *www.adclickint.com **<http://www.adclickint.com/> * * <http://www.adclickint.com/> <http://www.adclickint.com/>

9 years, 6 months

Maximum number of redirects reached, aborting connection attempt. Sorry.

by Brandon Jackson

I am using a release version of MonetDB 11.13.9 "OCT2012-SP3" on Fedora 18, running as an unprivileged user with no firewall. The server quit allowing connections via JDBC and mclient this morning. mclient hangs indefinitely after asking for a password run from the same machine and account that started mserver5 via monetdbd. "monetdb status", reports a healthy database. JDBC connections initiated using monetdb-jdbc-2.8.jar through Pentaho Data Integration. "Maximum number of redirects reached, aborting connection attempt. Sorry." is the error message. Shutting down MonetDB seemed to work. It also seems to start up okay, but still does not allow any connections. I am happy to install MonetDB with debug symbols and provide some output from gdb if that would be helpful. Please let me know a few things I can try that may be helpful for filing a bug report and getting us back online. Thanks for your help, Brandon The stack trace that came back to java was this: Error connecting to database [monet_testing] : org.pentaho.di.core.exception.KettleDatabaseException: Error occured while trying to connect to the database Error connecting to database: (using class nl.cwi.monetdb.jdbc.MonetDriver) Maximum number of redirects reached, aborting connection attempt. Sorry. org.pentaho.di.core.exception.KettleDatabaseException: Error occured while trying to connect to the database Error connecting to database: (using class nl.cwi.monetdb.jdbc.MonetDriver) Maximum number of redirects reached, aborting connection attempt. Sorry. at org.pentaho.di.core.database.Database.normalConnect(Database.java:366) at org.pentaho.di.core.database.Database.connect(Database.java:315) at org.pentaho.di.core.database.Database.connect(Database.java:277) at org.pentaho.di.core.database.Database.connect(Database.java:267) at org.pentaho.di.core.database.DatabaseFactory.getConnectionTestReport(DatabaseFactory.java:86) at org.pentaho.di.core.database.DatabaseMeta.testConnection(DatabaseMeta.java:2464) at org.pentaho.ui.database.event.DataHandler.testDatabaseConnection(DataHandler.java:533) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.pentaho.ui.xul.impl.AbstractXulDomContainer.invoke(AbstractXulDomContainer.java:329) at org.pentaho.ui.xul.impl.AbstractXulComponent.invoke(AbstractXulComponent.java:139) at org.pentaho.ui.xul.impl.AbstractXulComponent.invoke(AbstractXulComponent.java:123) at org.pentaho.ui.xul.swt.tags.SwtButton.access$500(SwtButton.java:26) at org.pentaho.ui.xul.swt.tags.SwtButton$4.widgetSelected(SwtButton.java:119) at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source) at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source) at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source) at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source) at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source) at org.eclipse.jface.window.Window.runEventLoop(Window.java:820) at org.eclipse.jface.window.Window.open(Window.java:796) at org.pentaho.ui.xul.swt.tags.SwtDialog.show(SwtDialog.java:378) at org.pentaho.ui.xul.swt.tags.SwtDialog.show(SwtDialog.java:304) at org.pentaho.di.ui.core.database.dialog.XulDatabaseDialog.open(XulDatabaseDialog.java:115) at org.pentaho.di.ui.core.database.dialog.DatabaseDialog.open(DatabaseDialog.java:62) at org.pentaho.di.ui.trans.steps.monetdbbulkloader.MonetDBBulkLoaderDialog$20.widgetSelected(MonetDBBulkLoaderDialog.java:1350) at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source) at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source) at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source) at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source) at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source) at org.pentaho.di.ui.trans.steps.monetdbbulkloader.MonetDBBulkLoaderDialog.open(MonetDBBulkLoaderDialog.java:905) at org.pentaho.di.ui.spoon.delegates.SpoonStepsDelegate.editStep(SpoonStepsDelegate.java:136) at org.pentaho.di.ui.spoon.Spoon.editStep(Spoon.java:7835) at org.pentaho.di.ui.spoon.trans.TransGraph.editStep(TransGraph.java:2749) at org.pentaho.di.ui.spoon.trans.TransGraph.mouseDoubleClick(TransGraph.java:705) at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source) at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source) at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source) at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source) at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source) at org.pentaho.di.ui.spoon.Spoon.readAndDispatch(Spoon.java:1221) at org.pentaho.di.ui.spoon.Spoon.waitForDispose(Spoon.java:7044) at org.pentaho.di.ui.spoon.Spoon.start(Spoon.java:8304) at org.pentaho.di.ui.spoon.Spoon.main(Spoon.java:580) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.pentaho.commons.launcher.Launcher.main(Launcher.java:134) Caused by: org.pentaho.di.core.exception.KettleDatabaseException: Error connecting to database: (using class nl.cwi.monetdb.jdbc.MonetDriver) Maximum number of redirects reached, aborting connection attempt. Sorry. at org.pentaho.di.core.database.Database.connectUsingClass(Database.java:502) at org.pentaho.di.core.database.Database.normalConnect(Database.java:350) ... 51 more Caused by: java.sql.SQLException: Maximum number of redirects reached, aborting connection attempt. Sorry. at nl.cwi.monetdb.jdbc.MonetConnection.<init>(MonetConnection.java:254) at nl.cwi.monetdb.jdbc.MonetDriver.connect(MonetDriver.java:171) at java.sql.DriverManager.getConnection(DriverManager.java:582) at java.sql.DriverManager.getConnection(DriverManager.java:185) at org.pentaho.di.core.database.Database.connectUsingClass(Database.java:482) ... 52 more Caused by: nl.cwi.monetdb.mcl.MCLException: Maximum number of redirects reached, aborting connection attempt. Sorry. at nl.cwi.monetdb.mcl.net.MapiSocket.connect(MapiSocket.java:255) at nl.cwi.monetdb.mcl.net.MapiSocket.connect(MapiSocket.java:395) at nl.cwi.monetdb.mcl.net.MapiSocket.connect(MapiSocket.java:395) at nl.cwi.monetdb.mcl.net.MapiSocket.connect(MapiSocket.java:395) at nl.cwi.monetdb.mcl.net.MapiSocket.connect(MapiSocket.java:395) at nl.cwi.monetdb.mcl.net.MapiSocket.connect(MapiSocket.java:395) at nl.cwi.monetdb.mcl.net.MapiSocket.connect(MapiSocket.java:395) at nl.cwi.monetdb.mcl.net.MapiSocket.connect(MapiSocket.java:395) at nl.cwi.monetdb.mcl.net.MapiSocket.connect(MapiSocket.java:395) at nl.cwi.monetdb.mcl.net.MapiSocket.connect(MapiSocket.java:395) at nl.cwi.monetdb.mcl.net.MapiSocket.connect(MapiSocket.java:395) at nl.cwi.monetdb.mcl.net.MapiSocket.connect(MapiSocket.java:248) at nl.cwi.monetdb.jdbc.MonetConnection.<init>(MonetConnection.java:232) ... 56 more Hostname : 172.27.0.209 Port : 50000 Database name : testing

9 years, 6 months

How to handle too much data ?

by mail_server＠lavabit.com

Hi, i am working on monetdb so as it can be used for handling huge streaming data, in the most efficient way. i am having a requirement of sustained inserts of rate around 50k-100k inserts per sec(maybe more).i have tested up the results on big enough data (upto 100Gb) and it's the fastest one, to be frank.Most data is 8 byte longs,1 column is time in secs and 1 column is string based.FYI,Data writer in single threaded. However,the issue is too much data and it will be going on for months( maybe years) and all queries have a range part.Most queries are on current data, while some will be for ages old data.Horizontal partitioning is the way to go as it seems. so what i thought was for efficiency, dump the arrays as per time in secs and then when a query comes, somehow form a new BAT (maybe using FUSE, since copying is unusable), join the data in temporary way and use it and discard.Few secs delay is okay even for the range query of 1 sec. as i have been digging around, http://www.monetdb.org/Documentation/Cookbooks/SQLrecipies/BinaryBulkLoad it's possible for full size columns. but since this is going to be a copy,it's inefficient. is there a way to get it done ( already exists) ?? any pointers might help such as BAT Partition Manager and so on. if NOT, i may try to get down and get it done but some pointers might be useful from the experts. Thanks for help. mike

9 years, 6 months

Unsigned integers

by Eduardo Oliveira

Hello, I would like to know if MonetDb supports unsigned integer, i couldn't find anything on documentation. I tried to created a table with unsigned int and i get an error. Thank you, -- *Eduardo Oliveira * *IT* ***Email:* eduardo.oliveira(a)adclick.pt *Web: *www.adclickint.com **<http://www.adclickint.com/> * * <http://www.adclickint.com/> <http://www.adclickint.com/>

9 years, 6 months

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

users-list January 2013