Hi Karl,
A quick (breakfast) reply.
On 01/08/2020 09:26, Karl Seguin wrote:
I've been working on a driver for Elixir (https://github.com/karlseguin/monet). It's immature, but in a decent enough state for our prototyping.
Nice you took up this task, looking forward to a mature version.
Overall, I've been very pleased and excited about MonetDB. I wanted to share some observations from the perspective of both a newcomer and having developed a driver.
First though, a couple questions:
I understand what a Q_BLOCK is, but I don't understand under what circumstances it happens.
Aside from the initial challenge response, it seems like all messages from the server are _always_ in response to a query. This is different than, say, PostgreSQL which can push data to signal some change, outside of the typical request->response flow. Correct?
Yes, there are no out-of-bound messages.
As for my feedback:
- The first and, by far, single biggest issue is with the text-base protocol. I'm sure this has been discussed before, and I doubt that I can add any new insight. But it was tedious to implement and error prone. We were initially considering storing large blobs in MonetDB (for a secondary purpose), but I'm now rethinking that idea because of the performance for serializing and deserializing the data.
The text based prototyp serves (served) its purpose. The weakness of all such prototols is described in Mark Raasveldt, Hannes Mühleisen: Don't Hold My Data Hostage - A Case For Client Protocol Redesign. Proc. VLDB Endow. 10(10): 1022-1033 (2017)
Given the simplicity of the underlying storage, you might consider ingest using the binary attach facility https://www.monetdb.org/Documentation/ServerAdministration/LoadingBulkData/B...
The rest of these issues are in no particular order.
- I'm pretty sure if you send an invalid x command, the server goes haywire. The simplest way I found to reproduce this is to change the 's' to 'x' in the official Node client (https://github.com/MonetDB/monetdb-nodejs/blob/19f24ec16db70a26d0b11e51d77be...) and connect.
The CPU load spikes and the logs are filled with: "createExceptionInternal: !ERROR: SQLException:SQLparser:42000!Unrecognized X command: SET TIME ZONE INTERVAL '-480' MINUTE;""
(I'm running: 17ab9b680ec55b342c08d034f2c5659a073cc74a)
- Maybe this is naive, but I don't understand why this works: insert into x (time_col) values ('10:30:50')
but this doesn't: exec 7('10:30:50')
I was hoping I could ignore the response from a "prepare" (except for getting the id from the first line) and just encode based on the argument types, but no. I need the type AND precision so that I can do "time(3) '...'" or "blob '....'" etc.
- Point 3 is made marginally worse because: prepare select 1 - ?
returns 2 rows. I look for the suffix "NULL,\tNULL,\tNULL\t]" to tell which are "real" placeholders. I'm not sure if that's correct.
I was surprised to see dates with 1, 2 and 3 digit years. Making it 4 digits would make it ISO8601 compliant.
I was caught off guard by the fact that ab error can show up as part of what seems like a healthy response. Initially, I was just looking for a response that stars with !. But while testing, a concurrent create table failed and I got "&3 X Y\n!CODE!ERR\n". Is there a rule that can help me parse these efficiently? (like they'll always be on the first or 2nd line), or can they really show up anywhere?
Some of the drivers appear to be in bad shape. The node driver gets confused by my point 4 above and fails to handle it. It also doesn't support time or blob values, and tries to JSON the built-in Date object.
I couldn't get the go driver to work at all (nil referencse trying to read rows). The ruby driver has no support for prepared statements and I'm pretty sure it doesn't return the correct value for: db.query("select '\\'")
- While I'm probably wrong, I can't help but think that merging the functionalities of the monetdbd and monetdb binaries would streamline the experience for newcomers.
Please don't interpret these as unkind criticisms of (or unhappiness with) MonetDB. Again, overall, very happy.
We'll take the comments for further scrutiny and resolution the issues.
regards, Martin Kersten
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list