[MonetDB-users] Issues with concurrent connections using JDBC
Hi!
I'm working on a research about database performance and I'm running a TPC-W implementation (based on http://www.ece.wisc.edu/~pharm/tpcw.shtml) on some databases (Monet, MySQL, Postgresql).
Basically, what is being done is having some emulated browsers generating load on the servlet container (tomcat in my case) that access the database through JDBC.
-------------- -------------- ---------------- | Emulated | | Tomcat | | DB | | Browser | -----> | | ------> | | -------------- -------------- ----------------
When running this benchmark with concurrent accesses I'm getting problems. If I try 5 concurrent Emulated Browsers running (and 5 concurrent database connections) everything goes well. The problem starts when I increase the concurrency. With 30 concurrent connections everything goes well for a few minutes and then it stops working. The servlet container stops responding and monet uses low percentage of CPU on this state.
When I try to kill tomcat and start again (without killing Monet) it starts working again and stops after a while.
Based on this experiment it seems that Monet is still responsive, but the instance of servlet container that was accessing Monet just freezes.
I'm trying to investigate it a little more, but since I'm not an expert on MonetDB internals, I'd like to know if there is some known issue like this one or if anyone can have a tip on how to solve this.
Since Monet is responsive and the servlet container is freezing, it looks like JDBC driver is doing something that is freezing the container (I used to suspect on application, but since it is working very well with MySQL and Postgresql, it seems to not be the problem).
I'll appreciate any help on this subject.
Thanks very much for your attention.
Best regards, Anderson Supriano
Hi,
Thank you for your interest in MonetDB/SQL.
On 04-04-2006 23:03:13 -0300, Anderson Supriano wrote: [snip]
When running this benchmark with concurrent accesses I'm getting problems. If I try 5 concurrent Emulated Browsers running (and 5 concurrent database connections) everything goes well. The problem starts when I increase the concurrency. With 30 concurrent connections everything goes well for a few minutes and then it stops working. The servlet container stops responding and monet uses low percentage of CPU on this state.
[snip]
You most probably encounter some sort of deadlock caused by a JDBC driver that is trying to connect, but gets in an IO/wait lock because the server accepted the TCP connect, but didn't handle it; hence JDBC is waiting for the server to act.
I'm trying to investigate it a little more, but since I'm not an expert on MonetDB internals, I'd like to know if there is some known issue like this one or if anyone can have a tip on how to solve this.
One of the things relevant here is the setting of mapi_clients in the MonetDB.conf file. It probably is set to 2, which means 2 worker threads to handle the connections are started (rough explanation). Increasing this value does what you expect, but setting it to high crashes the server, and still then weird things start to happen. (Fun, right?)
Since Monet is responsive and the servlet container is freezing, it looks like JDBC driver is doing something that is freezing the container (I used to suspect on application, but since it is working very well with MySQL and Postgresql, it seems to not be the problem).
If the servlet container uses ConnectionPooling it might not release a Connection (thus close it), or a Connection might not have been closed and not (yet) garbage collected, hence not closed, which keeps one of the connections open and the worker thread busy (with waiting).
I'll appreciate any help on this subject.
For concurrency jobs, MonetDB/SQL is known not to be your partner at the moment, sorry. MonetDB/SQL performs more reliable when using one Connection, and having the queries go in serial order, instead of parallel. This is indeed a shortcoming of the current setup.
Best regards, Fabian
Hi Fabian!
I changed the implementation to use one connection at a time and it seems to be working now.
Thanks very much for your support!
Regards, Anderson
participants (2)
-
Anderson Supriano -
Fabian Groffen