Request Buffer pools
Hi,
Does MonetDB have any built in buffer pools to handle parallel requests?
I see the default number of parallel connections accepted and processed at a time by MonetDB is 64.. While I see that there are ways to increase it, is there anyway built in to pool incoming requests until the live 64 requests are processed?
For example, our application would receive 100 requests at a time, while we wish to limit the number of queries handled at a time by MonetDB to 50. Is there any way to automatically handle the remaining 50 (something like innodb buffer pool) ??
Thanks & Regards,
Vijayakrishna.P. Mobile : (+91) 9500402305.
Hi Vijay,
I think the words your looking for a connection pooling.
InnoDB Buffer pool in MySQL is the amount of data stored in memory which you set to a certain size and has nothing to do with network traffic.
Regards,
Brian Hood
On Mon, Aug 10, 2015 at 8:45 AM, Vijay Krishna vijayakrishna55@gmail.com wrote:
Hi,
Does MonetDB have any built in buffer pools to handle parallel requests?
I see the default number of parallel connections accepted and processed at a time by MonetDB is 64.. While I see that there are ways to increase it, is there anyway built in to pool incoming requests until the live 64 requests are processed?
For example, our application would receive 100 requests at a time, while we wish to limit the number of queries handled at a time by MonetDB to 50. Is there any way to automatically handle the remaining 50 (something like innodb buffer pool) ??
Thanks & Regards,
Vijayakrishna.P. Mobile : (+91) 9500402305.
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
Oops.. My bad.. Thanks Brian..
Yes I was indeed looking for connection pools... Something like maximum thread concurrency option in innodb...
Is there any such available with MonetDB?
I had gone through this option... mserver5 --set max_clients=<num> Is this the only way to control parallel threads??
Thanks & Regards,
Vijayakrishna.P. Mobile : (+91) 9500402305.
On Mon, Aug 10, 2015 at 7:48 PM, Brian Hood brianh6854@googlemail.com wrote:
Hi Vijay,
I think the words your looking for a connection pooling.
InnoDB Buffer pool in MySQL is the amount of data stored in memory which you set to a certain size and has nothing to do with network traffic.
Regards,
Brian Hood
On Mon, Aug 10, 2015 at 8:45 AM, Vijay Krishna vijayakrishna55@gmail.com wrote:
Hi,
Does MonetDB have any built in buffer pools to handle parallel requests?
I see the default number of parallel connections accepted and processed at a time by MonetDB is 64.. While I see that there are ways to increase it, is there anyway built in to pool incoming requests until the live 64 requests are processed?
For example, our application would receive 100 requests at a time, while we wish to limit the number of queries handled at a time by MonetDB to 50. Is there any way to automatically handle the remaining 50 (something like innodb buffer pool) ??
Thanks & Regards,
Vijayakrishna.P. Mobile : (+91) 9500402305.
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
Hi Vijay,
Well if you take a look at your merovingian.log you can see the command that starts the database.
/usr/bin/mserver5 --dbpath=/data2/threatmonitor/threatmonitor --set merovingian_uri=mapi:monetdb://orville.lan:50000/threatmonitor --set mapi_open=false --set mapi_port=0 --set mapi_usock=/data2/threatmonitor/threatmonitor/.mapi.sock --set monet_vault_key=/data2/threatmonitor/threatmonitor/.vaultkey --set gdk_nr_threads=2 --set max_clients=64 --set sql_optimizer=default_pipe --set monet_daemon=yes
These 2 i think are the ones your looking for.
--set gdk_nr_threads=2 --set max_clients=64
Regards,
Brian Hood
On Tue, Aug 11, 2015 at 7:14 AM, Vijay Krishna vijayakrishna55@gmail.com wrote:
Oops.. My bad.. Thanks Brian..
Yes I was indeed looking for connection pools... Something like maximum thread concurrency option in innodb...
Is there any such available with MonetDB?
I had gone through this option... mserver5 --set max_clients=<num> Is this the only way to control parallel threads??
Thanks & Regards,
Vijayakrishna.P. Mobile : (+91) 9500402305.
On Mon, Aug 10, 2015 at 7:48 PM, Brian Hood brianh6854@googlemail.com wrote:
Hi Vijay,
I think the words your looking for a connection pooling.
InnoDB Buffer pool in MySQL is the amount of data stored in memory which you set to a certain size and has nothing to do with network traffic.
Regards,
Brian Hood
On Mon, Aug 10, 2015 at 8:45 AM, Vijay Krishna <vijayakrishna55@gmail.com
wrote:
Hi,
Does MonetDB have any built in buffer pools to handle parallel requests?
I see the default number of parallel connections accepted and processed at a time by MonetDB is 64.. While I see that there are ways to increase it, is there anyway built in to pool incoming requests until the live 64 requests are processed?
For example, our application would receive 100 requests at a time, while we wish to limit the number of queries handled at a time by MonetDB to 50. Is there any way to automatically handle the remaining 50 (something like innodb buffer pool) ??
Thanks & Regards,
Vijayakrishna.P. Mobile : (+91) 9500402305.
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
Hi,
indeed;
see also `man mserver5` and `man monetdb` or https://www.monetdb.org/Documentation/mserver5-man-page and https://www.monetdb.org/Documentation/monetdb-man-page
Stefan
----- On Aug 11, 2015, at 3:11 PM, Brian Hood brianh6854@googlemail.com wrote:
Hi Vijay,
Well if you take a look at your merovingian.log you can see the command that starts the database.
/usr/bin/mserver5 --dbpath=/data2/threatmonitor/threatmonitor --set merovingian_uri=mapi:monetdb://orville.lan:50000/threatmonitor --set mapi_open=false --set mapi_port=0 --set mapi_usock=/data2/threatmonitor/threatmonitor/.mapi.sock --set monet_vault_key=/data2/threatmonitor/threatmonitor/.vaultkey --set gdk_nr_threads=2 --set max_clients=64 --set sql_optimizer=default_pipe --set monet_daemon=yes
These 2 i think are the ones your looking for.
--set gdk_nr_threads=2 --set max_clients=64
Regards,
Brian Hood
On Tue, Aug 11, 2015 at 7:14 AM, Vijay Krishna < vijayakrishna55@gmail.com > wrote:
Oops.. My bad.. Thanks Brian..
Yes I was indeed looking for connection pools... Something like maximum thread concurrency option in innodb...
Is there any such available with MonetDB?
I had gone through this option... mserver5 --set max_clients=<num> Is this the only way to control parallel threads??
Thanks & Regards,
Vijayakrishna.P. Mobile : (+91) 9500402305 .
On Mon, Aug 10, 2015 at 7:48 PM, Brian Hood < brianh6854@googlemail.com > wrote:
Hi Vijay,
I think the words your looking for a connection pooling.
InnoDB Buffer pool in MySQL is the amount of data stored in memory which you set to a certain size and has nothing to do with network traffic.
Regards,
Brian Hood
On Mon, Aug 10, 2015 at 8:45 AM, Vijay Krishna < vijayakrishna55@gmail.com > wrote:
Hi,
Does MonetDB have any built in buffer pools to handle parallel requests?
I see the default number of parallel connections accepted and processed at a time by MonetDB is 64.. While I see that there are ways to increase it, is there anyway built in to pool incoming requests until the live 64 requests are processed?
For example, our application would receive 100 requests at a time, while we wish to limit the number of queries handled at a time by MonetDB to 50. Is there any way to automatically handle the remaining 50 (something like innodb buffer pool) ??
Thanks & Regards,
Vijayakrishna.P. Mobile : (+91) 9500402305 .
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
Thanks Stefan and Brian.
When I went through monetdb man page, the usage of nthreads (set as gdk_nr_threads) environmental variable was defined as
Defines how many worker threads the server should use to perform main
processing. Normally, this number equals the number of available CPU cores in the system. Reducing this number forces the server to use less parallelism when executing queries, or none at all if set to *1*. I am a bit confused in understanding this statement. The variable "nthreads" controls
1. the number of threads processing a single query using multiple cores. 2. the number of queries currently being executed over all databases by the mserver.
Please help me understand which of these two is correct.. This doubt arises because I have read from MonetDB home page that "MonetDB is designed for multi-core parallel execution on desktops to reduce response time for complex query processing".
If the variable "nthreads" controls the latter, then does monetdb have an in built queue to store the queries from remaining connections (controlled by the variable nclients) to the mserver?? I have read about multiplex funnel and a bit afraid over the transactional disadvantages mentioned in there.. Is there any other solution?
Thanks & Regards,
Vijayakrishna.P. Mobile : (+91) 9500402305.
On Tue, Aug 11, 2015 at 6:56 PM, Stefan Manegold Stefan.Manegold@cwi.nl wrote:
Hi,
indeed;
see also `man mserver5` and `man monetdb` or https://www.monetdb.org/Documentation/mserver5-man-page and https://www.monetdb.org/Documentation/monetdb-man-page
Stefan
----- On Aug 11, 2015, at 3:11 PM, Brian Hood brianh6854@googlemail.com wrote:
Hi Vijay,
Well if you take a look at your merovingian.log you can see the command
that
starts the database.
/usr/bin/mserver5 --dbpath=/data2/threatmonitor/threatmonitor --set merovingian_uri=mapi:monetdb://orville.lan:50000/threatmonitor --set mapi_open=false --set mapi_port=0 --set mapi_usock=/data2/threatmonitor/threatmonitor/.mapi.sock --set monet_vault_key=/data2/threatmonitor/threatmonitor/.vaultkey --set gdk_nr_threads=2 --set max_clients=64 --set sql_optimizer=default_pipe
--set
monet_daemon=yes
These 2 i think are the ones your looking for.
--set gdk_nr_threads=2 --set max_clients=64
Regards,
Brian Hood
On Tue, Aug 11, 2015 at 7:14 AM, Vijay Krishna <
vijayakrishna55@gmail.com >
wrote:
Oops.. My bad.. Thanks Brian..
Yes I was indeed looking for connection pools... Something like maximum
thread
concurrency option in innodb...
Is there any such available with MonetDB?
I had gone through this option... mserver5 --set max_clients=<num> Is this the only way to control parallel threads??
Thanks & Regards,
Vijayakrishna.P. Mobile : (+91) 9500402305 .
On Mon, Aug 10, 2015 at 7:48 PM, Brian Hood < brianh6854@googlemail.com wrote:
Hi Vijay,
I think the words your looking for a connection pooling.
InnoDB Buffer pool in MySQL is the amount of data stored in memory which
you set
to a certain size and has nothing to do with network traffic.
Regards,
Brian Hood
On Mon, Aug 10, 2015 at 8:45 AM, Vijay Krishna <
vijayakrishna55@gmail.com >
wrote:
Hi,
Does MonetDB have any built in buffer pools to handle parallel requests?
I see the default number of parallel connections accepted and processed
at a
time by MonetDB is 64.. While I see that there are ways to increase it, is there anyway built in
to pool
incoming requests until the live 64 requests are processed?
For example, our application would receive 100 requests at a time, while
we wish
to limit the number of queries handled at a time by MonetDB to 50. Is
there any
way to automatically handle the remaining 50 (something like innodb
buffer
pool) ??
Thanks & Regards,
Vijayakrishna.P. Mobile : (+91) 9500402305 .
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
-- | Stefan.Manegold@CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold/ | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
Hi Vijay,
I have found some interesting info in the monetdb man page actually.
--- This is interests me as it alllows you to limit the amount of clients that can connect to a database not just an mserver
*nclients=**number*
Sets the maximum amount of clients that can connect to this database at the same time. Setting this to a high value is discouraged. A multiplex-funnel may be more performant, see *MULTIPLEX-FUNNEL* below.
Regards,
Brian Hood
On Wed, Aug 12, 2015 at 7:48 AM, Vijay Krishna vijayakrishna55@gmail.com wrote:
Thanks Stefan and Brian.
When I went through monetdb man page, the usage of nthreads (set as gdk_nr_threads) environmental variable was defined as
Defines how many worker threads the server should use to perform main
processing. Normally, this number equals the number of available CPU cores in the system. Reducing this number forces the server to use less parallelism when executing queries, or none at all if set to *1*. I am a bit confused in understanding this statement. The variable "nthreads" controls
- the number of threads processing a single query using multiple
cores. 2. the number of queries currently being executed over all databases by the mserver.
Please help me understand which of these two is correct.. This doubt arises because I have read from MonetDB home page that "MonetDB is designed for multi-core parallel execution on desktops to reduce response time for complex query processing".
If the variable "nthreads" controls the latter, then does monetdb have an in built queue to store the queries from remaining connections (controlled by the variable nclients) to the mserver?? I have read about multiplex funnel and a bit afraid over the transactional disadvantages mentioned in there.. Is there any other solution?
Thanks & Regards,
Vijayakrishna.P. Mobile : (+91) 9500402305.
On Tue, Aug 11, 2015 at 6:56 PM, Stefan Manegold Stefan.Manegold@cwi.nl wrote:
Hi,
indeed;
see also `man mserver5` and `man monetdb` or https://www.monetdb.org/Documentation/mserver5-man-page and https://www.monetdb.org/Documentation/monetdb-man-page
Stefan
----- On Aug 11, 2015, at 3:11 PM, Brian Hood brianh6854@googlemail.com wrote:
Hi Vijay,
Well if you take a look at your merovingian.log you can see the command
that
starts the database.
/usr/bin/mserver5 --dbpath=/data2/threatmonitor/threatmonitor --set merovingian_uri=mapi:monetdb://orville.lan:50000/threatmonitor --set mapi_open=false --set mapi_port=0 --set mapi_usock=/data2/threatmonitor/threatmonitor/.mapi.sock --set monet_vault_key=/data2/threatmonitor/threatmonitor/.vaultkey --set gdk_nr_threads=2 --set max_clients=64 --set sql_optimizer=default_pipe
--set
monet_daemon=yes
These 2 i think are the ones your looking for.
--set gdk_nr_threads=2 --set max_clients=64
Regards,
Brian Hood
On Tue, Aug 11, 2015 at 7:14 AM, Vijay Krishna <
vijayakrishna55@gmail.com >
wrote:
Oops.. My bad.. Thanks Brian..
Yes I was indeed looking for connection pools... Something like maximum
thread
concurrency option in innodb...
Is there any such available with MonetDB?
I had gone through this option... mserver5 --set max_clients=<num> Is this the only way to control parallel threads??
Thanks & Regards,
Vijayakrishna.P. Mobile : (+91) 9500402305 .
On Mon, Aug 10, 2015 at 7:48 PM, Brian Hood < brianh6854@googlemail.com wrote:
Hi Vijay,
I think the words your looking for a connection pooling.
InnoDB Buffer pool in MySQL is the amount of data stored in memory
which you set
to a certain size and has nothing to do with network traffic.
Regards,
Brian Hood
On Mon, Aug 10, 2015 at 8:45 AM, Vijay Krishna <
vijayakrishna55@gmail.com >
wrote:
Hi,
Does MonetDB have any built in buffer pools to handle parallel requests?
I see the default number of parallel connections accepted and processed
at a
time by MonetDB is 64.. While I see that there are ways to increase it, is there anyway built
in to pool
incoming requests until the live 64 requests are processed?
For example, our application would receive 100 requests at a time,
while we wish
to limit the number of queries handled at a time by MonetDB to 50. Is
there any
way to automatically handle the remaining 50 (something like innodb
buffer
pool) ??
Thanks & Regards,
Vijayakrishna.P. Mobile : (+91) 9500402305 .
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
-- | Stefan.Manegold@CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold/ | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
Thanks for the links Brian..
Dear all,
From this, I see that a database can be set to any number of clients, which
gives me a question on number of queries processed at a time.
If I am using an 8 core machine and connecting 50 client threads (each executing many queries one by one) at a time to a database, will the mserver be processing queries from ALL the 50 clients at any given instance?? Or will that be restricted by the number of cores?? If this is restricted, then is there a default queue maintained by the server??
Put in simpler words, is the maximum number of queries processed by MonetDB at a given time equal to the number of client connections?? If not, then how is this handled (with a queue) ??
Thanks & Regards,
Vijayakrishna.P. Mobile : (+91) 9500402305.
On Wed, Aug 12, 2015 at 6:11 PM, Brian Hood brianh6854@googlemail.com wrote:
Hi Vijay,
I have found some interesting info in the monetdb man page actually.
--- This is interests me as it alllows you to limit the amount of clients that can connect to a database not just an mserver
*nclients=**number*
Sets the maximum amount of clients that can connect to this database at the same time. Setting this to a high value is discouraged. A multiplex-funnel may be more performant, see *MULTIPLEX-FUNNEL* below.
Regards,
Brian Hood
On Wed, Aug 12, 2015 at 7:48 AM, Vijay Krishna vijayakrishna55@gmail.com wrote:
Thanks Stefan and Brian.
When I went through monetdb man page, the usage of nthreads (set as gdk_nr_threads) environmental variable was defined as
Defines how many worker threads the server should use to perform main
processing. Normally, this number equals the number of available CPU cores in the system. Reducing this number forces the server to use less parallelism when executing queries, or none at all if set to *1*. I am a bit confused in understanding this statement. The variable "nthreads" controls
- the number of threads processing a single query using multiple
cores. 2. the number of queries currently being executed over all databases by the mserver.
Please help me understand which of these two is correct.. This doubt arises because I have read from MonetDB home page that "MonetDB is designed for multi-core parallel execution on desktops to reduce response time for complex query processing".
If the variable "nthreads" controls the latter, then does monetdb have an in built queue to store the queries from remaining connections (controlled by the variable nclients) to the mserver?? I have read about multiplex funnel and a bit afraid over the transactional disadvantages mentioned in there.. Is there any other solution?
Thanks & Regards,
Vijayakrishna.P. Mobile : (+91) 9500402305.
On Tue, Aug 11, 2015 at 6:56 PM, Stefan Manegold Stefan.Manegold@cwi.nl wrote:
Hi,
indeed;
see also `man mserver5` and `man monetdb` or https://www.monetdb.org/Documentation/mserver5-man-page and https://www.monetdb.org/Documentation/monetdb-man-page
Stefan
----- On Aug 11, 2015, at 3:11 PM, Brian Hood brianh6854@googlemail.com wrote:
Hi Vijay,
Well if you take a look at your merovingian.log you can see the
command that
starts the database.
/usr/bin/mserver5 --dbpath=/data2/threatmonitor/threatmonitor --set merovingian_uri=mapi:monetdb://orville.lan:50000/threatmonitor --set mapi_open=false --set mapi_port=0 --set mapi_usock=/data2/threatmonitor/threatmonitor/.mapi.sock --set monet_vault_key=/data2/threatmonitor/threatmonitor/.vaultkey --set gdk_nr_threads=2 --set max_clients=64 --set sql_optimizer=default_pipe
--set
monet_daemon=yes
These 2 i think are the ones your looking for.
--set gdk_nr_threads=2 --set max_clients=64
Regards,
Brian Hood
On Tue, Aug 11, 2015 at 7:14 AM, Vijay Krishna <
vijayakrishna55@gmail.com >
wrote:
Oops.. My bad.. Thanks Brian..
Yes I was indeed looking for connection pools... Something like
maximum thread
concurrency option in innodb...
Is there any such available with MonetDB?
I had gone through this option... mserver5 --set max_clients=<num> Is this the only way to control parallel threads??
Thanks & Regards,
Vijayakrishna.P. Mobile : (+91) 9500402305 .
On Mon, Aug 10, 2015 at 7:48 PM, Brian Hood <
brianh6854@googlemail.com > wrote:
Hi Vijay,
I think the words your looking for a connection pooling.
InnoDB Buffer pool in MySQL is the amount of data stored in memory
which you set
to a certain size and has nothing to do with network traffic.
Regards,
Brian Hood
On Mon, Aug 10, 2015 at 8:45 AM, Vijay Krishna <
vijayakrishna55@gmail.com >
wrote:
Hi,
Does MonetDB have any built in buffer pools to handle parallel
requests?
I see the default number of parallel connections accepted and
processed at a
time by MonetDB is 64.. While I see that there are ways to increase it, is there anyway built
in to pool
incoming requests until the live 64 requests are processed?
For example, our application would receive 100 requests at a time,
while we wish
to limit the number of queries handled at a time by MonetDB to 50. Is
there any
way to automatically handle the remaining 50 (something like innodb
buffer
pool) ??
Thanks & Regards,
Vijayakrishna.P. Mobile : (+91) 9500402305 .
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
-- | Stefan.Manegold@CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold/ | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
Hi
All queries are taken into execution concurrently (and possibly will compete for resources!!). Internally, queues are used to handle the eligible MAL instructions ready for execution.
regards, Martin
On 13/08/15 08:44, Vijay Krishna wrote:
Thanks for the links Brian..
Dear all,
From this, I see that a database can be set to any number of clients, which gives me a question on number of queries processed at a time.
If I am using an 8 core machine and connecting 50 client threads (each executing many queries one by one) at a time to a database, will the mserver be processing queries from ALL the 50 clients at any given instance?? Or will that be restricted by the number of cores?? If this is restricted, then is there a default queue maintained by the server??
Put in simpler words, is the maximum number of queries processed by MonetDB at a given time equal to the number of client connections?? If not, then how is this handled (with a queue) ??
Thanks & Regards,
Vijayakrishna.P. Mobile : (+91) 9500402305.
On Wed, Aug 12, 2015 at 6:11 PM, Brian Hood <brianh6854@googlemail.com mailto:brianh6854@googlemail.com> wrote:
Hi Vijay, I have found some interesting info in the monetdb man page actually. --- This is interests me as it alllows you to limit the amount of clients that can connect to a database not just an mserver *nclients=*/number/ Sets the maximum amount of clients that can connect to this database at the same time. Setting this to a high value is discouraged. A multiplex-funnel may be more performant, see /MULTIPLEX-FUNNEL/ below. Regards, Brian Hood On Wed, Aug 12, 2015 at 7:48 AM, Vijay Krishna <vijayakrishna55@gmail.com <mailto:vijayakrishna55@gmail.com>> wrote: Thanks Stefan and Brian. When I went through monetdb man page, the usage of nthreads (set as gdk_nr_threads) environmental variable was defined as >> Defines how many worker threads the server should use to perform main processing. Normally, this number equals the number of available CPU cores in the system. Reducing this number forces the server to use less parallelism when executing queries, or none at all if set to *1*. I am a bit confused in understanding this statement. The variable "nthreads" controls 1. the number of threads processing a single query using multiple cores. 2. the number of queries currently being executed over all databases by the mserver. Please help me understand which of these two is correct.. This doubt arises because I have read from MonetDB home page that "MonetDB is designed for multi-core parallel execution on desktops to reduce response time for complex query processing". If the variable "nthreads" controls the latter, then does monetdb have an in built queue to store the queries from remaining connections (controlled by the variable nclients) to the mserver?? I have read about multiplex funnel and a bit afraid over the transactional disadvantages mentioned in there.. Is there any other solution? Thanks & Regards, Vijayakrishna.P. Mobile : (+91) 9500402305 <tel:%28%2B91%29%209500402305>. On Tue, Aug 11, 2015 at 6:56 PM, Stefan Manegold <Stefan.Manegold@cwi.nl <mailto:Stefan.Manegold@cwi.nl>> wrote: Hi, indeed; see also `man mserver5` and `man monetdb` or https://www.monetdb.org/Documentation/mserver5-man-page and https://www.monetdb.org/Documentation/monetdb-man-page Stefan ----- On Aug 11, 2015, at 3:11 PM, Brian Hood brianh6854@googlemail.com <mailto:brianh6854@googlemail.com> wrote: > Hi Vijay, > > Well if you take a look at your merovingian.log you can see the command that > starts the database. > > /usr/bin/mserver5 --dbpath=/data2/threatmonitor/threatmonitor --set > merovingian_uri=mapi:monetdb://orville.lan:50000/threatmonitor --set > mapi_open=false --set mapi_port=0 --set > mapi_usock=/data2/threatmonitor/threatmonitor/.mapi.sock --set > monet_vault_key=/data2/threatmonitor/threatmonitor/.vaultkey --set > gdk_nr_threads=2 --set max_clients=64 --set sql_optimizer=default_pipe --set > monet_daemon=yes > > > These 2 i think are the ones your looking for. > > --set gdk_nr_threads=2 --set max_clients=64 > > Regards, > > Brian Hood > > > On Tue, Aug 11, 2015 at 7:14 AM, Vijay Krishna < vijayakrishna55@gmail.com <mailto:vijayakrishna55@gmail.com> > > wrote: > > > > Oops.. My bad.. Thanks Brian.. > > Yes I was indeed looking for connection pools... Something like maximum thread > concurrency option in innodb... > > Is there any such available with MonetDB? > > I had gone through this option... mserver5 --set max_clients=<num> > Is this the only way to control parallel threads?? > > > > > > Thanks & Regards, > > Vijayakrishna.P. > Mobile : (+91) 9500402305 <tel:%28%2B91%29%209500402305> . > > On Mon, Aug 10, 2015 at 7:48 PM, Brian Hood < brianh6854@googlemail.com <mailto:brianh6854@googlemail.com> > wrote: > > > > Hi Vijay, > > I think the words your looking for a connection pooling. > > InnoDB Buffer pool in MySQL is the amount of data stored in memory which you set > to a certain size and has nothing to do with network traffic. > > Regards, > > Brian Hood > > On Mon, Aug 10, 2015 at 8:45 AM, Vijay Krishna < vijayakrishna55@gmail.com <mailto:vijayakrishna55@gmail.com> > > wrote: > > > > Hi, > > Does MonetDB have any built in buffer pools to handle parallel requests? > > I see the default number of parallel connections accepted and processed at a > time by MonetDB is 64.. > While I see that there are ways to increase it, is there anyway built in to pool > incoming requests until the live 64 requests are processed? > > For example, our application would receive 100 requests at a time, while we wish > to limit the number of queries handled at a time by MonetDB to 50. Is there any > way to automatically handle the remaining 50 (something like innodb buffer > pool) ?? > > Thanks & Regards, > > Vijayakrishna.P. > Mobile : (+91) 9500402305 <tel:%28%2B91%29%209500402305> . > > _______________________________________________ > users-list mailing list > users-list@monetdb.org <mailto:users-list@monetdb.org> > https://www.monetdb.org/mailman/listinfo/users-list > > > > _______________________________________________ > users-list mailing list > users-list@monetdb.org <mailto:users-list@monetdb.org> > https://www.monetdb.org/mailman/listinfo/users-list > > > > _______________________________________________ > users-list mailing list > users-list@monetdb.org <mailto:users-list@monetdb.org> > https://www.monetdb.org/mailman/listinfo/users-list > > > > _______________________________________________ > users-list mailing list > users-list@monetdb.org <mailto:users-list@monetdb.org> > https://www.monetdb.org/mailman/listinfo/users-list -- | Stefan.Manegold@CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold/ <http://www.CWI.nl/~manegold/> | Science Park 123 (L321) | | +31 (0)20 592-4212 <tel:%2B31%20%280%2920%20592-4212> | 1098 XG Amsterdam (NL) | _______________________________________________ users-list mailing list users-list@monetdb.org <mailto:users-list@monetdb.org> https://www.monetdb.org/mailman/listinfo/users-list _______________________________________________ users-list mailing list users-list@monetdb.org <mailto:users-list@monetdb.org> https://www.monetdb.org/mailman/listinfo/users-list _______________________________________________ users-list mailing list users-list@monetdb.org <mailto:users-list@monetdb.org> https://www.monetdb.org/mailman/listinfo/users-list
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
participants (4)
-
Brian Hood -
Martin Kersten -
Stefan Manegold -
Vijay Krishna