[MonetDB-users] DB Corruption
We were troubleshooting some odd results coming back from our database, and found that it's missing some entries from a column. This appears to be causing some sort of offset. Does anyone have any ideas on what can be done to resolve it or what may have caused it? Is there any information we could send that would be helpful in troubleshooting this or for a bug report?
If we do a count(*) and then a count('single_column') we are getting two different results (when they should be indentical)
Thanks,
Joseph Brower
On 2012-03-24 00:14, Joseph Brower wrote:
We were troubleshooting some odd results coming back from our database, and found that it's missing some entries from a column. This appears to be causing some sort of offset. Does anyone have any ideas on what can be done to resolve it or what may have caused it? Is there any information we could send that would be helpful in troubleshooting this or for a bug report?
If we do a count(*) and then a count('single_column') we are getting two different results (when they should be indentical)
count (*) counts all rows, whereas count(single_column) only count non-null values. Are there any nulls in the single_column?
There shouldn't be any nulls. The records were all added by a "Copy Into" command. The files that we imported had no nulls in it at all.
On 03/24/2012 01:57 AM, Sjoerd Mullender wrote:
On 2012-03-24 00:14, Joseph Brower wrote:
We were troubleshooting some odd results coming back from our database, and found that it's missing some entries from a column. This appears to be causing some sort of offset. Does anyone have any ideas on what can be done to resolve it or what may have caused it? Is there any information we could send that would be helpful in troubleshooting this or for a bug report?
If we do a count(*) and then a count('single_column') we are getting two different results (when they should be indentical)
count (*) counts all rows, whereas count(single_column) only count non-null values. Are there any nulls in the single_column?
This SF email is sponsosred by: Try Windows Azure free for 90 days Click Here http://p.sf.net/sfu/sfd2d-msazure
MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
Joseph,
since you did not specify your queries in more detail, I assume they are as simple as (1) select count(*) from <your_table>; and (2) select count(single_column) from <your_table>;
In that case, you could try the following sanity check:
(3) select count(*) from <your_table> where single_column is null;
and check whether that returns the difference between your counts.
If that indeed returns the difference between your global count(*) and count(single_column), this would confirm the existence of NULL values, and you'd need to invest which of the values in your CVS file have been interpreted as NULL and why.
In case the result of queries (2) and (3) do not add up to the number returned by query (1), you might want to check the output(s) of
TRACE select count(*) from <your_table>; TRACE select count(single_column) from <your_table>; TRACE select count(*) from <your_table> where single_column is null;
(or have us check them, provided/once we find the time).
Stefan
On Sat, Mar 24, 2012 at 09:37:38AM -0600, Joseph Brower wrote:
There shouldn't be any nulls. The records were all added by a "Copy Into" command. The files that we imported had no nulls in it at all.
On 03/24/2012 01:57 AM, Sjoerd Mullender wrote:
On 2012-03-24 00:14, Joseph Brower wrote:
We were troubleshooting some odd results coming back from our database, and found that it's missing some entries from a column. This appears to be causing some sort of offset. Does anyone have any ideas on what can be done to resolve it or what may have caused it? Is there any information we could send that would be helpful in troubleshooting this or for a bug report?
If we do a count(*) and then a count('single_column') we are getting two different results (when they should be indentical)
count (*) counts all rows, whereas count(single_column) only count non-null values. Are there any nulls in the single_column?
This SF email is sponsosred by: Try Windows Azure free for 90 days Click Here http://p.sf.net/sfu/sfd2d-msazure
MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
This SF email is sponsosred by: Try Windows Azure free for 90 days Click Here http://p.sf.net/sfu/sfd2d-msazure
MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
We just checked. There are no nulls in the database at all. Additionally, there is only one column that has a different count.
Thanks,
Joseph Brower
On 03/24/2012 09:58 AM, Stefan Manegold wrote:
Joseph,
since you did not specify your queries in more detail, I assume they are as simple as (1) select count(*) from<your_table>; and (2) select count(single_column) from<your_table>;
In that case, you could try the following sanity check:
(3) select count(*) from<your_table> where single_column is null;
and check whether that returns the difference between your counts.
If that indeed returns the difference between your global count(*) and count(single_column), this would confirm the existence of NULL values, and you'd need to invest which of the values in your CVS file have been interpreted as NULL and why.
In case the result of queries (2) and (3) do not add up to the number returned by query (1), you might want to check the output(s) of
TRACE select count(*) from<your_table>; TRACE select count(single_column) from<your_table>; TRACE select count(*) from<your_table> where single_column is null;
(or have us check them, provided/once we find the time).
Stefan
On Sat, Mar 24, 2012 at 09:37:38AM -0600, Joseph Brower wrote:
There shouldn't be any nulls. The records were all added by a "Copy Into" command. The files that we imported had no nulls in it at all.
On 03/24/2012 01:57 AM, Sjoerd Mullender wrote:
On 2012-03-24 00:14, Joseph Brower wrote:
We were troubleshooting some odd results coming back from our database, and found that it's missing some entries from a column. This appears to be causing some sort of offset. Does anyone have any ideas on what can be done to resolve it or what may have caused it? Is there any information we could send that would be helpful in troubleshooting this or for a bug report?
If we do a count(*) and then a count('single_column') we are getting two different results (when they should be indentical)
count (*) counts all rows, whereas count(single_column) only count non-null values. Are there any nulls in the single_column?
This SF email is sponsosred by: Try Windows Azure free for 90 days Click Here http://p.sf.net/sfu/sfd2d-msazure
MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
This SF email is sponsosred by: Try Windows Azure free for 90 days Click Here http://p.sf.net/sfu/sfd2d-msazure _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
What is the column type? Is the situation repeatable using a fresh database?
regards, Martin
On 3/25/12 6:34 AM, Joseph Brower wrote:
We just checked. There are no nulls in the database at all. Additionally, there is only one column that has a different count.
Thanks,
Joseph Brower
On 03/24/2012 09:58 AM, Stefan Manegold wrote:
Joseph,
since you did not specify your queries in more detail, I assume they are as simple as (1) select count(*) from<your_table>; and (2) select count(single_column) from<your_table>;
In that case, you could try the following sanity check:
(3) select count(*) from<your_table> where single_column is null;
and check whether that returns the difference between your counts.
If that indeed returns the difference between your global count(*) and count(single_column), this would confirm the existence of NULL values, and you'd need to invest which of the values in your CVS file have been interpreted as NULL and why.
In case the result of queries (2) and (3) do not add up to the number returned by query (1), you might want to check the output(s) of
TRACE select count(*) from<your_table>; TRACE select count(single_column) from<your_table>; TRACE select count(*) from<your_table> where single_column is null;
(or have us check them, provided/once we find the time).
Stefan
On Sat, Mar 24, 2012 at 09:37:38AM -0600, Joseph Brower wrote:
There shouldn't be any nulls. The records were all added by a "Copy Into" command. The files that we imported had no nulls in it at all.
On 03/24/2012 01:57 AM, Sjoerd Mullender wrote:
On 2012-03-24 00:14, Joseph Brower wrote:
We were troubleshooting some odd results coming back from our database, and found that it's missing some entries from a column. This appears to be causing some sort of offset. Does anyone have any ideas on what can be done to resolve it or what may have caused it? Is there any information we could send that would be helpful in troubleshooting this or for a bug report?
If we do a count(*) and then a count('single_column') we are getting two different results (when they should be indentical)
count (*) counts all rows, whereas count(single_column) only count non-null values. Are there any nulls in the single_column?
This SF email is sponsosred by: Try Windows Azure free for 90 days Click Here http://p.sf.net/sfu/sfd2d-msazure
MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
This SF email is sponsosred by: Try Windows Azure free for 90 days Click Here http://p.sf.net/sfu/sfd2d-msazure _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
This SF email is sponsosred by: Try Windows Azure free for 90 days Click Here http://p.sf.net/sfu/sfd2d-msazure _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
We ended up working around the problem for now, but we do need to find resolution to it. We think what may have happened is that we had a crash or a restart while a copy into operation was running.
Is it possible that could have caused it?
Thanks,
Joseph Brower
On 03/25/2012 01:15 AM, Martin Kersten wrote:
What is the column type? Is the situation repeatable using a fresh database?
regards, Martin
On 3/25/12 6:34 AM, Joseph Brower wrote:
We just checked. There are no nulls in the database at all. Additionally, there is only one column that has a different count.
Thanks,
Joseph Brower
On 03/24/2012 09:58 AM, Stefan Manegold wrote:
Joseph,
since you did not specify your queries in more detail, I assume they are as simple as (1) select count(*) from<your_table>; and (2) select count(single_column) from<your_table>;
In that case, you could try the following sanity check:
(3) select count(*) from<your_table> where single_column is null;and check whether that returns the difference between your counts.
If that indeed returns the difference between your global count(*) and count(single_column), this would confirm the existence of NULL values, and you'd need to invest which of the values in your CVS file have been interpreted as NULL and why.
In case the result of queries (2) and (3) do not add up to the number returned by query (1), you might want to check the output(s) of
TRACE select count(*) from<your_table>; TRACE select count(single_column) from<your_table>; TRACE select count(*) from<your_table> where single_column is null;
(or have us check them, provided/once we find the time).
Stefan
On Sat, Mar 24, 2012 at 09:37:38AM -0600, Joseph Brower wrote:
There shouldn't be any nulls. The records were all added by a "Copy Into" command. The files that we imported had no nulls in it at all.
On 03/24/2012 01:57 AM, Sjoerd Mullender wrote:
On 2012-03-24 00:14, Joseph Brower wrote:
We were troubleshooting some odd results coming back from our database, and found that it's missing some entries from a column. This appears to be causing some sort of offset. Does anyone have any ideas on what can be done to resolve it or what may have caused it? Is there any information we could send that would be helpful in troubleshooting this or for a bug report?
If we do a count(*) and then a count('single_column') we are getting two different results (when they should be indentical)
count (*) counts all rows, whereas count(single_column) only count non-null values. Are there any nulls in the single_column?
This SF email is sponsosred by: Try Windows Azure free for 90 days Click Here http://p.sf.net/sfu/sfd2d-msazure
MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
This SF email is sponsosred by: Try Windows Azure free for 90 days Click Here http://p.sf.net/sfu/sfd2d-msazure _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
This SF email is sponsosred by: Try Windows Azure free for 90 days Click Here http://p.sf.net/sfu/sfd2d-msazure _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
This SF email is sponsosred by: Try Windows Azure free for 90 days Click Here http://p.sf.net/sfu/sfd2d-msazure _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
participants (5)
-
Joseph Brower -
Joseph Brower -
Martin Kersten -
Sjoerd Mullender -
Stefan Manegold