I am sorry didn¹t realize mailing list is not their in my reply.
On 7/20/15, 14:24, "Ying Zhang" <Y.Zhang(a)cwi.nl> wrote:
>Hello Anu,
>
>Would you please post this to the MonetDB users mailing list? Thanks!
>
>Regards,
>
>Jennie
>
>> On Jul 20, 2015, at 19:44, Anu Raj Srivastava <Anu.Srivastava(a)rms.com>
>>wrote:
>>
>> Hi Hannes,
>>
>> I am using apache dbcp2 connection Pool. I am running queries in a
>>constant loop with 8 concurrent invocations. And each 8 threads are
>>running 9 queries each again and again.
>>
>> Java code is quite straight forward:-
>>
>> private ConnectionPoolPG() {
>> try {
>> final Properties properties = new Properties();
>>
>>properties.load(ConnectionPoolPG.class.getClassLoader().getResourceAsStre
>>am("monetDB.properties"));
>> dataSource =
>>BasicDataSourceFactory.createDataSource(properties);
>> schemaName = properties.getProperty("schemaname");
>> } catch (Exception e) {
>> throw new ExposureStoreException("Unable to instantiate
>>connection pool: " + e.getMessage(),
>> EAPAException.ERR_STORE_CONNECTION_CONFIG_ERROR);
>> }
>> }
>>
>> public void AllQueryPerformance() throws Exception {
>> while (true) {
>> if (queries != null) {
>> for (String query : queries) {
>> try (Connection connection =
>>ConnectionPoolPG.getInstance().getConnection()) {
>> try (Statement statement =
>>connection.createStatement()) {
>> statement.execute(query);
>> }
>> }
>> }
>> }
>> }
>> }
>>
>> -----Original Message-----
>> From: Hannes Mühleisen [mailto:Hannes.Muehleisen@cwi.nl]
>> Sent: Sunday, July 19, 2015 4:47 AM
>> To: Communication channel for developers of the MonetDB suite.
>> Cc: Ying Zhang; Anu Raj Srivastava
>> Subject: Re: MonetDB causing disk space to fill up
>>
>>
>>> On 19 Jul 2015, at 13:23, Hannes Mühleisen <Hannes.Muehleisen(a)cwi.nl>
>>>wrote:
>>>
>>> Hi,
>>>
>>>> On 16 Jul 2015, at 23:11, Ying Zhang <Y.Zhang(a)cwi.nl> wrote:
>>>>
>>>> On Jul 13, 2015, at 22:32, Anu Raj Srivastava
>>>><Anu.Srivastava(a)rms.com> wrote:
>>>>>
>>>>> Hey Jennie,
>>>>>
>>>>> I did upgrade to latest version but I am facing the same issue.
>>>>>Further debugging into the issue it seems like it is ConnectionPool
>>>>>issue. When I use new connection for each query MonetDB seems to be
>>>>>able to clean up the resources
>>>>
>>>> Ok, MonetDB is doing what it¹s supposed to do.
>>>>
>>>>> but if I use ConnectionPool for some reason DiskSpace keep filling
>>>>>up.
>>>>
>>>> If a connection is kept open, I¹d expect MonetDB to be less eager to
>>>>clean up resources.
>>>> However, we generally recommend using a ConnectionPool, to avoid too
>>>>much overhead caused by making a new connection for every query. So
>>>>my suggestion for now would be to keep using the ConnectionPool, but
>>>>try to find the reason why your disk is filled up.
>>>>
>>>>> Do you have any idea why that could happen?
>>>>
>>>> Hmmm, no. We¹ve fixed quite some leaks in Jul2015...
>>>> Some more thinking:
>>>> - After how many iterations of repeatedly executing the queries is
>>>>your disk filled up?
>>>> - Can you please try to run your queries with mclient to observe the
>>>>behaviour of MonetDB (and disk usage)?
>>>>
>>>> @Hannes: sorry to disturb you during holiday. But if possible, would
>>>>you please have a look at this thread. Does anything ring a bell?
>>>>Thanks!
>>>
>>> I tested this with ac3p0 connection pool (limited to 4 concurrent
>>>connections) and repeating queries from 10 threads (SELECT
>>>MIN(something) FROM Š). I was not able to observe any (!) increase in
>>>storage footprint. So without a minimal reproducible example of Java
>>>code + DB schema + sample data, it is hard to tell what is going on. Is
>>>this running in auto commit? Using temporary tables?
>>
>> Small follow-up: When forcing the allocation of a large intermediate
>>(SELECT MIN(something+1) FROM Š), additional storage space is indeed
>>claimed. The attached plot shows dbfarm size over time while 10 threads
>>are running 1000 queries each using a 5-connection pool. You can see how
>>the amount of additional storage drops back to exactly where it was once
>>the pool shuts down at around 350s, and that there is no accumulation of
>>additional storage over the course of the pool being active.
>>
>> Hannes
>>
>
Hello Developers,
How can I insert record(s) between two rows of SQL table?
For example, I have table with three column and there are some records
in it:
| column1 | column2 | column3 |
-------------------------------
| 123 | asffasf | true |
-------------------------------
| 432 | lkjlkhk | false |
-------------------------------
| 555 | sadda | true |
-------------------------------
| ... | ... | ... |
-------------------------------
I need to add a new record (555, "asdas", false) between first and
second record of the existing table.
Result what I expect:
| column1 | column2 | column3 |
-------------------------------
| 123 | asffasf | true |
-------------------------------
| 555 | asdas | false |
-------------------------------
| 432 | lkjlkhk | false |
-------------------------------
| 555 | sadda | true |
-------------------------------
| ... | ... | ... |
-------------------------------
--
Best regards,
Sherzod mailto:shmutalov@gmail.com
Hello Developers,
I need some help to understand monetdb kernel/sql structure.
Please, provide example code/paper or link for follow situation:
I have a table (SQL), and I want to change structure of it
(add/drop/rename/order column). I know, I can do it with SQL,
but I need to view examples in MAL or C.
I want to write function like this:
int add_column(str schemaName, str tableName, str newColumnName)
{
// return 1 if column successfully created
}
Before posting this message I saw the source code of SQL kernel
and found some functions like sys_drop_column,
mvc_create_column
and others. But, I don't know how to execute these functions from
my own user defined C/MAL function.
--
Best regards,
Sherzod mailto:shmutalov@gmail.com
Hi,
I have a bunch of test queries which I am testing against MonetDB. Queries all by itself run fine but if you run the same set of queries in a loop disk runs out of space. I am not sure why would memory map files won't clean up automatically and why do we have to start the server again to clean up these files? Is there a workaround to this problem or is this a known issue in MonetDB?
Thanks,
Anu
To: sjoerd(a)monetdb.org
I build Jul2015 branch and tested with my test app and mserver5
errors disappeared :) Nice
But, still have problem with Mapi library, where first 1-4 connections
get lost, because WSAStartup error.
Then I added changes to my test program, and when program starts it
preconfigures Windows Sockets, then starts client thread. Now it works:
https://gist.github.com/shmutalov/3894fdd22ec62334fe17
Regards,
Sherzod Mutalov
Thank you for answer, <b>Sjoerd Mullender!
You said,
> </b>In all, I think the situation is better in Jul2015
Does these problems have been fixed in nightly build?
Regards,
Sherzod Mutalov
Good day all,
I am testing mserver5 with multithreaded client and have stability
problems.
I wrote a simple app in C#, which uses wrapper around Mapi (pure calls to
libmapi.dll functions). The app creates 1000 tasks (threads),
each thread opens connection to mserver5, then sends a simple query:
"SELECT {0}, NOW();", where "{0}" is thread number, and
fetchs the result.
Problems are:
1. When my application starts to execute threads, always 1st, 2nd or 3rd
connection will be lost, and I can't reconnect from that thread.
2. Sometimes there is error on mserver5, which causes connection lost on
the client:
"!SABAOTHgetMyStatus: MALException:sabaoth.getmystatus:could
not find my own database?!?"
mserver5 is running on the local machine, same as client.
I tryied to set a gdk_nr_threads option to 1, 8, 16, and get the same
problems.
I am running on Windows 7 x64, 16GB RAM, Intel i7-5500 CPU
Regards,
Sherzod Mutalov
Dear all,
MonetDB Solutions, the supporting company of MonetDB, is looking for core database developers. If you’re interested, please see the job description here: https://www.monetdbsolutions.com/company/careers
Regards,
Jennie