Hi Frédéric,

We use MonetDB as the backend for all our search solutions, for many years already. Like you, we put MonetDB under quite some stress, we find many bugs, and we often feel the need to customize it to better serve our own purposes. We also experience stability issues, but most of the times it's because we can't resist the temptation to work on the latest release.

My own opinion, in short: we do need to put some effort in it ("expect problems and eat them for breakfast"), but it still pays off. There aren't that many alternatives for us - our products would simply be ridiculous if they relied on PostgreSQL or alike.

Roberto spinque.com

PS. related discussion on StackOverflow: http://stackoverflow.com/questions/10087647/monetdb-anyone-uses-it-in-produc...

On 18 June 2015 at 12:44, Anthony Damico ajdamico@gmail.com wrote:

yo frederic, answer to your question depends on whether you want a rocket ship that occasionally needs tlc or a horse. i've submitted 20 bugs, the folks at monetdb work hard, fix things fast, just keep reporting and then re-write the query with a workaround. or gallop away on some trusty steed. ps psa if you make it to amsterdam ask them for a keychain w cute wooden shoes peace

https://www.monetdb.org/bugzilla/buglist.cgi?bug_status=UNCONFIRMED&bug_...

On Thu, Jun 18, 2015 at 6:26 AM, Martin Kersten martin@monetdb.org wrote:

...

Hi Frederic

Thank you for your interest and (private) enhancements of the MonetDB code base. Would be nice to share some of those to the community if relevant for a larger audience. Pushing a piece of technology that large and complex is a challenge for all involved.

Your bugreport will be taken up in the next round of software improvements. It seems you retract the creation of 'foo' by releasing the savepoint record, dropping it from further consideration. Clearly, you won't find it anymore. Enhanced savepoint management is on our list of 'unsupported' features.

https://www.monetdb.org/Documentation/Manuals/SQLreference/Features/unsuppor...

Your request puts it into view for re-consideration.

You can have a look at the extensive regression testing that takes place on a daily basis and covers both functional and user supplied bug reports. https://www.monetdb.org/Developers/NightlyTesting

For business urgent issues you can always contact MonetDB Solutions to find a solution for your problem.

regards, Martin Kersten

On 18/06/15 10:44, Frédéric Jolliton wrote:

...

Greetings,

We filled a bug few days ago about simple SAVEPOINT usage corrupting the database (both in Oct2014-SP4 and tip of hg). This is worrying us a lot, because that's yet another bug easy to trigger.

We're studying MonetDB extensively for our future needs. We've written optimizers, custom aggregate functions in C, and we've studied many internal aspects.

But yet, from time to time, we are hit by bugs when performing simple, trivial queries. You can see the history of bugs we filled on the bugtracker (regarding UNION, SELECT, ..). This become time consuming.

This make us nervous because we might have hard time trusting our complexes queries run by MonetDB.

The performance are stunning, but the quality seems not high enough for production. Projects like PostgreSQL shows that even Open Source projects can attains incredible stability and robustness. (Admittedly there are many more contributors to PostgreSQL.)

Note that we understand that it is a complex software, and that it is hard to cover all possible corner cases in tests.

But we're wondering how stable MonetDB is expected to be. Is this really used in production?

Regarding the MonetDB website, the primary source of information for the project, we've not found any reference documentation regarding all the supported SQL features. Sure, there is a page listing what is *not* supported, there are pages on various topics, but it is unclear if SAVEPOINT is really supported. (If I can say, I think that the website probably needs some refresh and better navigation.)

Here is the latest bug I was talking about:

 $ bin/mclient -p 50005 test
 Welcome to mclient, the MonetDB/SQL interactive terminal

(Oct2014-SP4) Database: MonetDB v11.19.13 (Oct2014-SP4), 'mapi:monetdb://fjolliton:50005/test' Type \q to quit, ? for a list of available commands auto commit mode: on sql>start transaction; auto commit mode: off sql>create table foo (a int); operation successful (0.528ms) sql>savepoint sp1; auto commit mode: off sql>release savepoint sp1; auto commit mode: off sql>commit; auto commit mode: on sql>select * from foo; +---+ | a | +===+ +---+ 0 tuples (0.9s)

So far, so good. Now try to connect again:

 $ bin/mclient -p 50005 test
 Welcome to mclient, the MonetDB/SQL interactive terminal

(Oct2014-SP4) Database: MonetDB v11.19.13 (Oct2014-SP4), 'mapi:monetdb://fjolliton:50005/test' Type \q to quit, ? for a list of available commands auto commit mode: on sql>select * from foo; SELECT: no such table 'foo'

Whoops !

Regards,

---

users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

---

users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

Probably not, but a simple workaround would be to calculate your aggregations in a subquery, something like this:

SELECT count / nr / 60 FROM ( SELECT count(*) AS count, (max("end")-min(start) AS nr FROM table GROUP BY s HAVING nr <> 0 );

Cheers, Robin

On 19-06-15 03:00, Raffael Marty wrote:

I might be doing something wrong, but should this really throw a division by zero?

SELECT count(*) / ((max("end")-min(start)) / 60) FROM table GROUP BY s HAVING max("end")-min(start)<>0 ;

Strangely enough, this works:

SELECT count(*) / ((max("end")-min(start))) FROM table GROUP BY s HAVING max("end")-min(start)<>0;

Am I doing something wrong?

Thanks

Raffael

—

Visual Analytics Workshop at #BlackHat US in August: http://bit.ly/1FN1W5e Raffael Marty ceo @ pixlcloud http://pixlcloud.com @raffaelmarty http://raffy.ch _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

On Thu, Jun 18, 2015 at 06:00:05PM -0700, Raffael Marty wrote:

I might be doing something wrong, but should this really throw a division by zero?

SELECT count(*) / ((max("end")-min(start)) / 60) FROM table GROUP BY s HAVING max("end")-min(start)<>0 ;

Strangely enough, this works:

SELECT count(*) / ((max("end")-min(start))) FROM table GROUP BY s HAVING max("end")-min(start)<>0;

Am I doing something wrong?

Raffael

Seems a bug, ie could you report it on the bugtracker. Including an example of the problem (with ddl statements)?

Niels

Thanks

Raffael

—

Visual Analytics Workshop at #BlackHat US in August: http://bit.ly/1FN1W5e Raffael Marty ceo @ pixlcloud http://pixlcloud.com @raffaelmarty http://raffy.ch _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

Hi,

We have been also analysing MonetDB and quite impressed with its performance. We were very much excited to build our application on it and considered testing it with our app. App communicates with mserver through JDBC.

While we ran some basic automation scripts on our app with MonetDB as its backend, we were surprised to meet a few downtimes of the MonetDB database and at time the mserver even.. The automation was ran on an 8GB RAM machine which served 11 threads at a time.

We received error messages indicated the mserver down as follows during the automation.

1. java.sql.SQLException: Unable to connect (localhost:50000): Connection to server lost! 2. Connection terminated 3. java.sql.SQLException: no supported password hashes (would like to know what this exception means..)

What is the maximum requests limit for mserver? Is it configurable? What are all the other factors that I have to look for to avoid this?

Earlier, while I was load testing the database with a few tables with a few million rows (on a CentOS machine with 128GB RAM), I had received the following exception at times while connecting the mclient. There were no well set patterns on which this was reproduced.

$ ./bin/mclient -d dbname unsupported hash algorithms: gorithms: nect

and the client doesn't open up. In order to get things back, I will have to restart the mserver. And during this process, a single monetdbd stop command doesn't kill the server. Instead, it takes 2 or 3 minutes to kill the mserver. What is this error and how shall it be avoided?

Is there any best practices or FAQs to consider around while using MonetDB in production?

Being totally impressed with the DB's performance, we wish to move our application on MonetDB. But we consider this as a serious show stopping concern. A little bit stability with the pinnacle of performance would definitely take MonetDB to greater heights.

Any tips or help on my exceptions would be much appreaciated. Thanks in advance.

Thanks & Regards,

Vijayakrishna.P. Mobile : (+91) 9500402305.

On Mon, Jun 22, 2015 at 4:01 PM, Frédéric Jolliton < frederic.jolliton+monetdb@securactive.net> wrote:

Thanks for all your answers. I'm grouping the response here.

Regarding SAVEPOINT, I think that if they are not supported at all, they should be disabled!

Maybe add a flag to give access to all the unstable/uncomplete features, but disable them by default. That could be either a global setting or a per-connection setting. This way, no risk to use a feature that may look like it is supported.

You said that it was documented that the feature was not supported, but from what I understand there are two SQL:2003 features involved:

• T271 (Savepoints)
• T272 (Enhanced savepoint management)

But in this page:

https://www.monetdb.org/Documentation/Manuals/SQLreference/Features/unsuppor...

only T272 is listed as not supported.

Should this imply that T271 is supported? Isn't that what we were using in our bug report?

Regarding our internals patches, we discussed that a bit when we meet some of you in Paris. We plan to release our patches if they can be useful to the community. We're a shop that make heavy use of Open Source, and in turn, we're releasing various components to the community (including our core business engine.)

Overall, we're pretty happy with reactivity when we fill a bug: it is fixed in few days! And that's awesome. Part on our interest for MonetDB come from that. The projet is well alive and actively maintained.

Nonetheless, the nature of the bugs, which seems to touch "basic" stuff each time (query that are not uncommon) make us doubt about the targeted audience of MonetDB.

Sure, many of you are using MonetDB in production. My article title was a bit harsh. But since we're building an application around it, and that we want a certain assurance regarding the result, we grow some concerns. Especially because we're building the query on the fly from a abstract, dynamic, definition, thus we can't anticipate all the sort of combination we will get (UNION, GROUP BY, aggregates, lot of subselect, ..) and thus might trigger a silent bug returning incorrect data for example.

But don't get me wrong: we love the MonetDB for its design, its speed and its ability to be extended.

Regards,

-- Frédéric Jolliton Sécuractive _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list