Has anyone configured an scalable MonetDB using Docker cluster on Amazon Web Services or other cloud environment? I'm looking into setting this up for one of our project around COBID-19 and essentially need to: - have my number of MonetDB instances to go up and down with user demand - use a load balancer to spread user queries across multiple instances
I had in mind that all the instances would just point to a single dbfarm directory (all the user queries are read only), but is this an option or will cause problems (locks, caching, etc.)?
Based on what I understand from Cluster Management (https://www.monetdb.org/Documentation/Cookbooks/SQLrecipes/ClusterManagement) or Lazy Replication (https://www.monetdb.org/Documentation/Cookbooks/SQLrecipes/WorkloadCaptureRe...), I need to replicate the database for each instance, which is not really practical.
This document seems to hint that using the "Replication Services" model would be an option, but then I end up with any copies of the same database, and this will require a significant amount of extra disk space (the transaction logs + one copy per instance). http://www.exanest.eu/pub/2016_RTPBD_Monetdb.pdf
Any guidance, suggestion, example would be greatly appreciated.
best *P
Please check following links, as these seems to work for your scenarios.
https://www.monetdb.org/blog/remote-tables-auth https://www.monetdb.org/blog/updatable-merge-tables
Regards, Mubin Abbas
On Sun, May 17, 2020 at 2:46 AM Pascal Heus pascal.heus@gmail.com wrote:
Has anyone configured an scalable MonetDB using Docker cluster on Amazon Web Services or other cloud environment? I'm looking into setting this up for one of our project around COBID-19 and essentially need to:
- have my number of MonetDB instances to go up and down with user demand
- use a load balancer to spread user queries across multiple instances
I had in mind that all the instances would just point to a single dbfarm directory (all the user queries are read only), but is this an option or will cause problems (locks, caching, etc.)?
Based on what I understand from Cluster Management ( https://www.monetdb.org/Documentation/Cookbooks/SQLrecipes/ClusterManagement)
or Lazy Replication ( https://www.monetdb.org/Documentation/Cookbooks/SQLrecipes/WorkloadCaptureRe...),
I need to replicate the database for each instance, which is not really practical.
This document seems to hint that using the "Replication Services" model would be an option, but then I end up with any copies of the same database, and this will require a significant amount of extra disk space (the transaction logs + one copy per instance). http://www.exanest.eu/pub/2016_RTPBD_Monetdb.pdf
Any guidance, suggestion, example would be greatly appreciated.
best *P _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
participants (2)
-
Mubin Abbas -
Pascal Heus