[MonetDB-users] Clustering, Replication, MultiThreading, Restore
Hi,
I've just discover MonetDB. I've found some documentation but i've still some questions.
1) About replication, i'm unable to find a documentation about replication support inside MonetDB, if it is not supported do you plan to support it natively, or is there some special script to launch in order to have database in sync.
2) About clustering, my understanding about clustering is to allow one client to launch a sql query on a database through merovingian (if i understood). The query is analysed by all nodes of the cluster and then it return the answer. However, it seems it is not the case, if I follow the documentation on the website about clustering, it seems that I will be connected through a round robin fashion to one of the node. Once connected my query will be executed only on that node.
3) About multithreading, I mean multi query execution per user, I think MonetDb support the fact that multi user can connect to the database and launch different query on it. But is it possible to configure the number of concurrent connection, to setup the number of thread that handle queries ?
4) If MonetDb crash, is it possible that db files become corrupted ? In this case, is there a script to fix it ?
Thank you for your help
Joel
Hi Joel,
On 7/9/11 12:34 AM, joel vennin wrote:
Hi,
I've just discover MonetDB. I've found some documentation but i've still some questions.
- About replication, i'm unable to find a documentation about
replication support inside MonetDB, if it is not supported do you plan to support it natively, or is there some special script to launch in order to have database in sync.
MonetDB is geared at datawarehousing and has not invested a lot in maintaining replicas under OLTP loads. In this context there are several options to maintain replicas, which heavily rely on the application requirement.
- About clustering, my understanding about clustering is to allow one
client to launch a sql query on a database through merovingian (if i understood). The query is analysed by all nodes of the cluster and then it return the answer. However, it seems it is not the case, if I follow the documentation on the website about clustering, it seems that I will be connected through a round robin fashion to one of the node. Once connected my query will be executed only on that node.
That depends on how you have partitioned the database. In the normal sharded case, the query is executed at all nodes and the result sets glued together.
- About multithreading, I mean multi query execution per user, I
think MonetDb support the fact that multi user can connect to the database and launch different query on it. But is it possible to configure the number of concurrent connection, to setup the number of thread that handle queries ?
The max number of concurrent users in the kernel is set in the source code. Merovingian can act as a funnel for a large number of connections.
All queries are ran multi-threaded using all the cores available. The user can limit the number of threads through a system variable.
- If MonetDb crash, is it possible that db files become corrupted ?
In this case, is there a script to fix it ?
Depends on the crash. A hardware failuer may not be recoverable. Aside from that the SQL WAL recovers the database as you would expect from a DBMS.
Thank you for your help
Your welcome,
Martin
Joel
All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2d-c2 _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
On 09-07-2011 06:37:55 +0200, Martin Kersten wrote:
- About clustering, my understanding about clustering is to allow one
client to launch a sql query on a database through merovingian (if i understood). The query is analysed by all nodes of the cluster and then it return the answer. However, it seems it is not the case, if I follow the documentation on the website about clustering, it seems that I will be connected through a round robin fashion to one of the node. Once connected my query will be executed only on that node.
That depends on how you have partitioned the database. In the normal sharded case, the query is executed at all nodes and the result sets glued together.
They are not, this is only in a special case that is not documented on the website, yet.
Thank you for yours replies.
Ok for replication I've to make the things my self, so why not, but I'm still a bit surprise that monetdb is not able to exchange its update log (if they exist) between node.
About clustering, can you provide an example showing the different "shared" configuration ? Or can you tell me in which source file I should look.
Thank you guys !
On 7/9/11 10:35 PM, joel vennin wrote:
Thank you for yours replies.
Ok for replication I've to make the things my self, so why not, but I'm still a bit surprise that monetdb is not able to exchange its update log (if they exist) between node.
There is such code, but it has not been tested to the extreme that we put it in the front line of a distribution. Mostly because internally we lack a strong need for it and therefore priority is low.
Moreover, it directly works on the MAL level.
See, e.g. monetdb5/src/modules/mal/replication.mx and monetdb5/src/optimizer/opt_replication.mx
for more details
regards, Martin
About clustering, can you provide an example showing the different "shared" configuration ? Or can you tell me in which source file I should look.
Thank you guys !
All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2d-c2 _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
On Sat, Jul 9, 2011 at 10:49 PM, Martin Kersten Martin.Kersten@cwi.nl wrote:
On 7/9/11 10:35 PM, joel vennin wrote:
Thank you for yours replies.
Ok for replication I've to make the things my self, so why not, but I'm still a bit surprise that monetdb is not able to exchange its update log (if they exist) between node.
There is such code, but it has not been tested to the extreme that we put it in the front line of a distribution. Mostly because internally we lack a strong need for it and therefore priority is low.
Moreover, it directly works on the MAL level.
See, e.g. monetdb5/src/modules/mal/replication.mx and monetdb5/src/optimizer/opt_replication.mx
Ok I will take a look, just read the documentation about MAL, and after I will try to dive into replication.mx
About clustering, can you provide an example showing the different "shared" configuration ? Or can you tell me in which source file I should look.
About this point, where I can find information ?
Thank you again.
Thank you guys !
All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2d-c2 _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2d-c2 _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
participants (3)
-
Fabian Groffen -
joel vennin -
Martin Kersten