possible monetdb catalogue corruption and how to recover from it?
Hi List,
We are seeing a situation with monetdb v11.17.17 "Jan2014-SP2” release where at the time of dropping table , data base got crashed. After that we are not able to drop this table.
Any command to drop this table is returning with success but we can not create same table again as it complains about table exists/ in use.
But when we try to select * from this table or do a count on this it does complains saying No Such Table exist.
My hunch is somehow database catalogue got corrupted, what is recommended way to bring back database to good health after any such condition occurs?
We have not tried restarting monetdb instance yet.
Regards, Ashish Kumar Singh Mobile: +91-9972045095 Skype: toashishk@gmail.com
Hai Ashish,
Do you happen to have any more detailed info of the crash? Such as was the drop table a single statement transaction, or part of a multi statement transaction? Is the problem reproducible?
Can you please try to restart the server to see if it helps cleaning up?
Jan2014 is a pretty old version. I'd highly recommend you to upgrade. Jul2015 will be released later this month.
Regards,
Jennie
On 08 Jul 2015, at 12:37, Ashish Kumar Singh ashishkumar.singh@altair.com wrote:
Hi List,
We are seeing a situation with monetdb v11.17.17 "Jan2014-SP2” release where at the time of dropping table , data base got crashed. After that we are not able to drop this table.
Any command to drop this table is returning with success but we can not create same table again as it complains about table exists/ in use.
But when we try to select * from this table or do a count on this it does complains saying No Such Table exist.
My hunch is somehow database catalogue got corrupted, what is recommended way to bring back database to good health after any such condition occurs?
We have not tried restarting monetdb instance yet.
Regards, Ashish Kumar Singh Mobile: +91-9972045095 Skype: toashishk@gmail.com
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
Hi Jennie,
Thanks for reply!
Drop table is a single statement transaction. Problem is not reproducible though easily. We encountered it after 150 iterations of drop and create table each happening in 1 hour interval but our ETL tool. We did a restart and after this problem was resolved. I think database was able to do a cleanup.
What will be a recommended way of doing a health check on database catalogue when such problem happens as practically problem does happen right?
Thanks for the heads up on upcoming release we shall plan to upgrade to it.
Regards, Ashish
From: users-list on behalf of Ying Zhang Reply-To: "users-list@monetdb.orgmailto:users-list@monetdb.org" Date: Thursday, 9 July 2015 1:27 pm To: "users-list@monetdb.orgmailto:users-list@monetdb.org" Subject: Re: possible monetdb catalogue corruption and how to recover from it?
Hai Ashish,
Do you happen to have any more detailed info of the crash? Such as was the drop table a single statement transaction, or part of a multi statement transaction? Is the problem reproducible?
Can you please try to restart the server to see if it helps cleaning up?
Jan2014 is a pretty old version. I'd highly recommend you to upgrade. Jul2015 will be released later this month.
Regards,
Jennie
On 08 Jul 2015, at 12:37, Ashish Kumar Singh <ashishkumar.singh@altair.commailto:ashishkumar.singh@altair.com> wrote:
Hi List,
We are seeing a situation with monetdb v11.17.17 "Jan2014-SP2” release where at the time of dropping table , data base got crashed. After that we are not able to drop this table.
Any command to drop this table is returning with success but we can not create same table again as it complains about table exists/ in use.
But when we try to select * from this table or do a count on this it does complains saying No Such Table exist.
My hunch is somehow database catalogue got corrupted, what is recommended way to bring back database to good health after any such condition occurs?
We have not tried restarting monetdb instance yet.
Regards, Ashish Kumar Singh Mobile: +91-9972045095 Skype: toashishk@gmail.commailto:toashishk@gmail.com
_______________________________________________ users-list mailing list users-list@monetdb.orgmailto:users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
On 09/07/15 10:38, Ashish Kumar Singh wrote:
Hi Jennie,
Thanks for reply!
Drop table is a single statement transaction. Problem is not reproducible though easily. We encountered it after 150 iterations of drop and create table each happening in 1 hour interval but our ETL tool. We did a restart and after this problem was resolved. I think database was able to do a cleanup.
What will be a recommended way of doing a health check on database catalogue when such problem happens as practically problem does happen right?
Thanks for the heads up on upcoming release we shall plan to upgrade to it.
Regards, Ashish
From: users-list on behalf of Ying Zhang Reply-To: "users-list@monetdb.org mailto:users-list@monetdb.org" Date: Thursday, 9 July 2015 1:27 pm To: "users-list@monetdb.org mailto:users-list@monetdb.org" Subject: Re: possible monetdb catalogue corruption and how to recover from it?
Hai Ashish,
Do you happen to have any more detailed info of the crash? Such as was the drop table a single statement transaction, or part of a multi statement transaction? Is the problem reproducible?
Can you please try to restart the server to see if it helps cleaning up?
Jan2014 is a pretty old version. I'd highly recommend you to upgrade. Jul2015 will be released later this month.
Regards,
Jennie
On 08 Jul 2015, at 12:37, Ashish Kumar Singh <ashishkumar.singh@altair.com mailto:ashishkumar.singh@altair.com> wrote:
Hi List,
We are seeing a situation with monetdb v11.17.17 "Jan2014-SP2” release where at the time of dropping table , data base got crashed. After that we are not able to drop this table.
Could not be reproduced on Jul2015 using either 2000 separate sessions and a single session with 2000 create/drop pairs
Any command to drop this table is returning with success but we can not create same table again as it complains about table exists/ in use.
But when we try to select * from this table or do a count on this it does complains saying No Such Table exist.
My hunch is somehow database catalogue got corrupted, what is recommended way to bring back database to good health after any such condition occurs?
That's on our wishlist for future tools/extensions.
regards, Martin
We have not tried restarting monetdb instance yet.
Regards, Ashish Kumar Singh Mobile: +91-9972045095 Skype: toashishk@gmail.com mailto:toashishk@gmail.com
users-list mailing list users-list@monetdb.org mailto:users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
Hi Jennie,
Any idea when Jul2015 is going to be released?
Regards, Ashish
From: users-list on behalf of Ashish Singh Reply-To: "users-list@monetdb.orgmailto:users-list@monetdb.org" Date: Thursday, 9 July 2015 2:08 pm To: "users-list@monetdb.orgmailto:users-list@monetdb.org" Subject: Re: possible monetdb catalogue corruption and how to recover from it?
Hi Jennie,
Thanks for reply!
Drop table is a single statement transaction. Problem is not reproducible though easily. We encountered it after 150 iterations of drop and create table each happening in 1 hour interval but our ETL tool. We did a restart and after this problem was resolved. I think database was able to do a cleanup.
What will be a recommended way of doing a health check on database catalogue when such problem happens as practically problem does happen right?
Thanks for the heads up on upcoming release we shall plan to upgrade to it.
Regards, Ashish
From: users-list on behalf of Ying Zhang Reply-To: "users-list@monetdb.orgmailto:users-list@monetdb.org" Date: Thursday, 9 July 2015 1:27 pm To: "users-list@monetdb.orgmailto:users-list@monetdb.org" Subject: Re: possible monetdb catalogue corruption and how to recover from it?
Hai Ashish,
Do you happen to have any more detailed info of the crash? Such as was the drop table a single statement transaction, or part of a multi statement transaction? Is the problem reproducible?
Can you please try to restart the server to see if it helps cleaning up?
Jan2014 is a pretty old version. I'd highly recommend you to upgrade. Jul2015 will be released later this month.
Regards,
Jennie
On 08 Jul 2015, at 12:37, Ashish Kumar Singh <ashishkumar.singh@altair.commailto:ashishkumar.singh@altair.com> wrote:
Hi List,
We are seeing a situation with monetdb v11.17.17 "Jan2014-SP2” release where at the time of dropping table , data base got crashed. After that we are not able to drop this table.
Any command to drop this table is returning with success but we can not create same table again as it complains about table exists/ in use.
But when we try to select * from this table or do a count on this it does complains saying No Such Table exist.
My hunch is somehow database catalogue got corrupted, what is recommended way to bring back database to good health after any such condition occurs?
We have not tried restarting monetdb instance yet.
Regards, Ashish Kumar Singh Mobile: +91-9972045095 Skype: toashishk@gmail.commailto:toashishk@gmail.com
_______________________________________________ users-list mailing list users-list@monetdb.orgmailto:users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
On 06/08/15 16:35, Ashish Kumar Singh wrote:
Hi Jennie,
Any idea when Jul2015 is going to be released?
The intention is very soon. I intend to create a release candidate tomorrow.
Regards, Ashish
From: users-list on behalf of Ashish Singh Reply-To: "users-list@monetdb.org mailto:users-list@monetdb.org" Date: Thursday, 9 July 2015 2:08 pm To: "users-list@monetdb.org mailto:users-list@monetdb.org" Subject: Re: possible monetdb catalogue corruption and how to recover from it?
Hi Jennie,
Thanks for reply!
Drop table is a single statement transaction. Problem is not reproducible though easily. We encountered it after 150 iterations of drop and create table each happening in 1 hour interval but our ETL tool. We did a restart and after this problem was resolved. I think database was able to do a cleanup.
What will be a recommended way of doing a health check on database catalogue when such problem happens as practically problem does happen right?
Thanks for the heads up on upcoming release we shall plan to upgrade to it.
Regards, Ashish
From: users-list on behalf of Ying Zhang Reply-To: "users-list@monetdb.org mailto:users-list@monetdb.org" Date: Thursday, 9 July 2015 1:27 pm To: "users-list@monetdb.org mailto:users-list@monetdb.org" Subject: Re: possible monetdb catalogue corruption and how to recover from it?
Hai Ashish,
Do you happen to have any more detailed info of the crash? Such as was the drop table a single statement transaction, or part of a multi statement transaction? Is the problem reproducible?
Can you please try to restart the server to see if it helps cleaning up?
Jan2014 is a pretty old version. I'd highly recommend you to upgrade. Jul2015 will be released later this month.
Regards,
Jennie
On 08 Jul 2015, at 12:37, Ashish Kumar Singh <ashishkumar.singh@altair.com mailto:ashishkumar.singh@altair.com> wrote:
Hi List,
We are seeing a situation with monetdb v11.17.17 "Jan2014-SP2” release where at the time of dropping table , data base got crashed. After that we are not able to drop this table.
Any command to drop this table is returning with success but we can not create same table again as it complains about table exists/ in use.
But when we try to select * from this table or do a count on this it does complains saying No Such Table exist.
My hunch is somehow database catalogue got corrupted, what is recommended way to bring back database to good health after any such condition occurs?
We have not tried restarting monetdb instance yet.
Regards, Ashish Kumar Singh Mobile: +91-9972045095 Skype: toashishk@gmail.com mailto:toashishk@gmail.com
_______________________________________________ users-list mailing list users-list@monetdb.org mailto:users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
- -- Sjoerd Mullender
Hi,
We have a similar bug on the development branch of monetdb.
After creating and dropping lots of tables, our program finaly crash on CREATE TABLE which already exists (but not really) :
SELECT name FROM tables WHERE name = 'atable' ; => no results, so we want to create it. CREATE TABLE atable (a int) ; => table name already in use ! (our program crashes here)
Then , manually we can do :
DROP TABLE atable ; => OK CREATE TABLE atable (a int) ; => OK SELECT * FROM atable => OK SELECT name FROM tables WHERE name = 'atable' ; => no results SELECT name FROM tables WHERE name ilike '%atable%' ; => no results SELECT name FROM _tables WHERE name = 'atable' ; => no results
Maybe this but have the same origin.
We keep this broken database for testing if you have some questions about BATs info. Should i post a bug report for the devel branch ?
Thanks, Guillaume
Le 06/08/2015 16:35, Ashish Kumar Singh a écrit :
Hi Jennie,
Any idea when Jul2015 is going to be released?
Regards, Ashish
From: users-list on behalf of Ashish Singh Reply-To: "users-list@monetdb.org mailto:users-list@monetdb.org" Date: Thursday, 9 July 2015 2:08 pm To: "users-list@monetdb.org mailto:users-list@monetdb.org" Subject: Re: possible monetdb catalogue corruption and how to recover from it?
Hi Jennie,
Thanks for reply!
Drop table is a single statement transaction. Problem is not reproducible though easily. We encountered it after 150 iterations of drop and create table each happening in 1 hour interval but our ETL tool. We did a restart and after this problem was resolved. I think database was able to do a cleanup.
What will be a recommended way of doing a health check on database catalogue when such problem happens as practically problem does happen right?
Thanks for the heads up on upcoming release we shall plan to upgrade to it.
Regards, Ashish
From: users-list on behalf of Ying Zhang Reply-To: "users-list@monetdb.org mailto:users-list@monetdb.org" Date: Thursday, 9 July 2015 1:27 pm To: "users-list@monetdb.org mailto:users-list@monetdb.org" Subject: Re: possible monetdb catalogue corruption and how to recover from it?
Hai Ashish,
Do you happen to have any more detailed info of the crash? Such as was the drop table a single statement transaction, or part of a multi statement transaction? Is the problem reproducible?
Can you please try to restart the server to see if it helps cleaning up?
Jan2014 is a pretty old version. I'd highly recommend you to upgrade. Jul2015 will be released later this month.
Regards,
Jennie
On 08 Jul 2015, at 12:37, Ashish Kumar Singh <ashishkumar.singh@altair.com mailto:ashishkumar.singh@altair.com> wrote:
Hi List,
We are seeing a situation with monetdb v11.17.17 "Jan2014-SP2” release where at the time of dropping table , data base got crashed. After that we are not able to drop this table.
Any command to drop this table is returning with success but we can not create same table again as it complains about table exists/ in use.
But when we try to select * from this table or do a count on this it does complains saying No Such Table exist.
My hunch is somehow database catalogue got corrupted, what is recommended way to bring back database to good health after any such condition occurs?
We have not tried restarting monetdb instance yet.
Regards, Ashish Kumar Singh Mobile: +91-9972045095 Skype: toashishk@gmail.com mailto:toashishk@gmail.com
users-list mailing list users-list@monetdb.org mailto:users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
Hello Guillaume,
On Aug 06, 2015, at 17:04 , Guillaume Savary guillaume.savary@securactive.net wrote:
Hi,
We have a similar bug on the development branch of monetdb.
After creating and dropping lots of tables, our program finaly crash on CREATE TABLE which already exists (but not really) :
SELECT name FROM tables WHERE name = 'atable' ; => no results, so we want to create it. CREATE TABLE atable (a int) ; => table name already in use ! (our program crashes here)
Your program crashes? Not MonetDB server?
Then , manually we can do :
DROP TABLE atable ; => OK CREATE TABLE atable (a int) ; => OK SELECT * FROM atable => OK SELECT name FROM tables WHERE name = 'atable' ; => no results SELECT name FROM tables WHERE name ilike '%atable%' ; => no results SELECT name FROM _tables WHERE name = 'atable' ; => no results
I can reproduce this with a fresh database on the default branch.
Maybe this but have the same origin.
We keep this broken database for testing if you have some questions about BATs info. Should i post a bug report for the devel branch ?
If it’s reproducible, it’s surely a bug. But is it possible for you to provide a sequence of SQL queries, with which we can reproduce your problem?
Regards, Jennie
Thanks, Guillaume
Le 06/08/2015 16:35, Ashish Kumar Singh a écrit :
Hi Jennie,
Any idea when Jul2015 is going to be released?
Regards, Ashish
From: users-list on behalf of Ashish Singh Reply-To: "users-list@monetdb.org" Date: Thursday, 9 July 2015 2:08 pm To: "users-list@monetdb.org" Subject: Re: possible monetdb catalogue corruption and how to recover from it?
Hi Jennie,
Thanks for reply!
Drop table is a single statement transaction. Problem is not reproducible though easily. We encountered it after 150 iterations of drop and create table each happening in 1 hour interval but our ETL tool. We did a restart and after this problem was resolved. I think database was able to do a cleanup.
What will be a recommended way of doing a health check on database catalogue when such problem happens as practically problem does happen right?
Thanks for the heads up on upcoming release we shall plan to upgrade to it.
Regards, Ashish
From: users-list on behalf of Ying Zhang Reply-To: "users-list@monetdb.org" Date: Thursday, 9 July 2015 1:27 pm To: "users-list@monetdb.org" Subject: Re: possible monetdb catalogue corruption and how to recover from it?
Hai Ashish,
Do you happen to have any more detailed info of the crash? Such as was the drop table a single statement transaction, or part of a multi statement transaction? Is the problem reproducible?
Can you please try to restart the server to see if it helps cleaning up?
Jan2014 is a pretty old version. I'd highly recommend you to upgrade. Jul2015 will be released later this month.
Regards,
Jennie
On 08 Jul 2015, at 12:37, Ashish Kumar Singh ashishkumar.singh@altair.com wrote:
Hi List,
We are seeing a situation with monetdb v11.17.17 "Jan2014-SP2” release where at the time of dropping table , data base got crashed. After that we are not able to drop this table.
Any command to drop this table is returning with success but we can not create same table again as it complains about table exists/ in use.
But when we try to select * from this table or do a count on this it does complains saying No Such Table exist.
My hunch is somehow database catalogue got corrupted, what is recommended way to bring back database to good health after any such condition occurs?
We have not tried restarting monetdb instance yet.
Regards, Ashish Kumar Singh Mobile: +91-9972045095 Skype: toashishk@gmail.com
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
users-list mailing list
users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
-- Guillaume Savary Securactive R&D
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
Sorry to answer in the same thead but It seems to be catalog corruption bug too.
Le 09/08/2015 15:33, Ying Zhang a écrit :
SELECT name FROM tables WHERE name = 'atable' ; => no results, so we want to create it. CREATE TABLE atable (a int) ; => table name already in use ! (our program crashes here) Your program crashes? Not MonetDB server?
Our program crashes because it does the "SELECT * FROM tables" to list them and then tries to create a table (which is not listed in the results), but this table already exists. So it crashes. This program only do : SELECT FROM tables + CREATE/DROP TABLE + COPY FROM. Thats all. But it does it thousand times a day. We had this database state after running a few days without interruptions.
Then , manually we can do :
DROP TABLE atable ; => OK CREATE TABLE atable (a int) ; => OK SELECT * FROM atable => OK SELECT name FROM tables WHERE name = 'atable' ; => no results SELECT name FROM tables WHERE name ilike '%atable%' ; => no results SELECT name FROM _tables WHERE name = 'atable' ; => no results
I can reproduce this with a fresh database on the default branch.
You meen "I cannot" I guess. We cannot reproduce it on our side. That's why we keep this database alive.
If it’s reproducible, it’s surely a bug. But is it possible for you to provide a sequence of SQL queries, with which we can reproduce your problem?
As, i said above, we cannot reproduce it for now. That's why we didn't make any bug-report yet. But when I saw Ashish's message, I thought that maybe it's the same/a similar bug.
participants (5)
-
Ashish Kumar Singh -
Guillaume Savary -
Martin Kersten -
Sjoerd Mullender -
Ying Zhang