Suggested correction for the Java SQLcopyinto example
Dear Monet team,
I've been getting quite a bit of use out of your example code here:
http://dev.monetdb.org/hg/MonetDB/file/tip/java/example/SQLcopyinto.java
Recently, though, I ran into a problem. If the "COPY INTO" statement that I issue is wrong, for example the referenced table doesn't exist, Monet comes back with an error. The example isn't checking for this error condition and proceeds to send data anyway. With a really large batch of data, this becomes a problem because the SQL parser ends up trying to parse all that data as if it were SQL and basically chokes on it.
The solution is to add something like the following after the out.newLine() on line 86:
error = in.waitForPrompt(); if (error != null) { throw new Exception(error); }
Cheers,
Hello Percy,
On 08.02.2013, at 19:07, Percy Wegmann percy@clicksecurity.com wrote:
Recently, though, I ran into a problem. If the "COPY INTO" statement that I issue is wrong, for example the referenced table doesn't exist, Monet comes back with an error. The example isn't checking for this error condition and proceeds to send data anyway. With a really large batch of data, this becomes a problem because the SQL parser ends up trying to parse all that data as if it were SQL and basically chokes on it.
The solution is to add something like the following after the out.newLine() on line 86:
error = in.waitForPrompt(); if (error != null) { throw new Exception(error); }
You are right, that can happen. We will update this to the mentioned file soon. Thanks for providing the solution!
Best,
Hannes
On 09-02-2013 10:25:22 +0100, Hannes Mühleisen wrote:
The solution is to add something like the following after the out.newLine() on line 86:
error = in.waitForPrompt();
if (error != null) {
throw new Exception(error);
}
You are right, that can happen. We will update this to the mentioned file soon. Thanks for providing the solution!
I think it's a good suggestion and I was about to commit it soon, however, since I didn't start it yet, did you start on it? Then I'll just do nothing.
On 09.02.2013, at 10:29, Fabian Groffen fabian@monetdb.org wrote:
On 09-02-2013 10:25:22 +0100, Hannes Mühleisen wrote:
The solution is to add something like the following after the out.newLine() on line 86:
error = in.waitForPrompt();
if (error != null) {
throw new Exception(error);}
You are right, that can happen. We will update this to the mentioned file soon. Thanks for providing the solution!
I think it's a good suggestion and I was about to commit it soon, however, since I didn't start it yet, did you start on it? Then I'll just do nothing.
Please go ahead, I have no dev environment on my private system here.
Best,
Hannes
Hi Percy,
On 08-02-2013 12:07:30 -0600, Percy Wegmann wrote:
The solution is to add something like the following after the out.newLine() on line 86:
error = in.waitForPrompt();
if (error != null) {
throw new Exception(error);
}
I'm surprised this works for you. It shouldn't as a matter of fact. Since we don't flush, the server /can/ tell us it read something it is certain of it was a complete statement, but isn't happy with, but we didn't say we finished our command yet, so it could /also/ say it only once we finish sending all we got. Therefore we can't wait for a prompt, there shouldn't be one to come here, as we didn't flush (= indefinite hang).
In any case, waiting for a prompt there is a wrong thing to do, even if it accidentially seems to work. A bug in the server (due to streams being taken over) causes this behaviour, but you should never rely on it.
That said, feel free to use the hack, but I'm not going to add it to the example.
I'm not sure I follow you. Here's an MCLIENT session to demonstrate what I observe:
sql>COPY INTO alert_tbl FROM STDIN USING DELIMITERS ',','\n','"'; COPY INTO: no such table 'alert_tbl' sql>rollback; auto commit mode: off sql>COPY INTO clk.alert_tbl FROM STDIN USING DELIMITERS ',','\n','"'; more>
Note how when I issue a bad COPY INTO command, I immediately get a prompt telling me it was no good.
Note how when I issue a good COPY INTO command, I immediately get a prompt asking for more.
That 2nd prompt manifests in my code as a null response to waitForPrompt(), and it does so consistently.
I just tested my proposed change to the example and it works consistently. Have you tested the example with and without a bad COPY INTO?
Cheers, Percy
P.S. To be safe, I've now switched to using COPY INTO ... FROM file and just write my files to /dev/shm
On Sat, Feb 9, 2013 at 4:31 AM, Fabian Groffen fabian@monetdb.org wrote:
Hi Percy,
On 08-02-2013 12:07:30 -0600, Percy Wegmann wrote:
The solution is to add something like the following after the
out.newLine()
on line 86:
error = in.waitForPrompt();
if (error != null) {
throw new Exception(error);}
I'm surprised this works for you. It shouldn't as a matter of fact. Since we don't flush, the server /can/ tell us it read something it is certain of it was a complete statement, but isn't happy with, but we didn't say we finished our command yet, so it could /also/ say it only once we finish sending all we got. Therefore we can't wait for a prompt, there shouldn't be one to come here, as we didn't flush (= indefinite hang).
In any case, waiting for a prompt there is a wrong thing to do, even if it accidentially seems to work. A bug in the server (due to streams being taken over) causes this behaviour, but you should never rely on it.
That said, feel free to use the hack, but I'm not going to add it to the example.
-- Fabian Groffen fabian@monetdb.org column-store pioneer http://www.monetdb.org/Home
users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list
participants (3)
-
Fabian Groffen -
Hannes Mühleisen -
Percy Wegmann