A little more information:
When I tested with just 50 rows that had a tinyint column with positive values interspersed with –1, and only 1 place where there were more than 2 consecutive rows with “-1” values for this column, there was no problem. The –1’s were correctly stored and I could execute the query below ( note there was a typo in the query presented in the original email – no space between “minusone” and “test”):
select * from minusonetest where chr=2 and coelorachis > -1;
However, when I store a table containing 2 Billion rows where a particular “tinyint" column has a large string of –1 values (7507 consecutive rows with –1 for the column) then the query above causes the errors below. For some reason, accessing the data when the byte (tinyint) column contains many consecutive –1’s is a problem.
I have not tried this for “int” or “real”, so don’t know if it is just a “tinyint” problem.
Thanks for any insight you can provide - Lynn
From: users-list <users-list-bounces+lcj34=cornell.edu@monetdb.orgmailto:users-list-bounces+lcj34=cornell.edu@monetdb.org> on behalf of Lynn Carol Johnson <lcj34@cornell.edumailto:lcj34@cornell.edu> Reply-To: Communication channel for MonetDB users <users-list@monetdb.orgmailto:users-list@monetdb.org> Date: Wednesday, May 25, 2016 at 1:31 PM To: Communication channel for MonetDB users <users-list@monetdb.orgmailto:users-list@monetdb.org> Subject: storing -1 in a tinyint
Hi all -
I apologize for all the emails, but I am banging on monetdb a lot lately.
Today I’ve run into a problem storing a –1 in a “tinyint” column. My java code writes bytes to a binary file. This file will be loaded via binary bulk load into a “tinyint” column. The values in this column range from –1 to 3.
I am writing the negative value to the byte file as below:
byte missingData = (byte)-1; writer.writeByte(missingData);
The data loads correctly into monetdb, and I can print it:
sql>\d minusonetest
CREATE TABLE "sys"."minusonetest" (
"chr" INTEGER,
"pos" INTEGER,
"coelorachis" TINYINT
);
sql>
sql>select * from minusonetest where chr=2 limit 10;
+------+------+-------------+
| chr | pos | coelorachis |
+======+======+=============+
| 2 | 1 | -1 |
| 2 | 2 | -1 |
| 2 | 3 | -1 |
| 2 | 4 | -1 |
| 2 | 5 | -1 |
| 2 | 6 | -1 |
| 2 | 7 | -1 |
| 2 | 8 | -1 |
| 2 | 9 | -1 |
| 2 | 10 | -1 |
+------+------+-------------+
10 tuples (10.7s)
sql>
BUT, when I run the select statement below, it tosses me out of the DB and prints errors to the log file:
select * from minusone test where chr=2 and coelorachis > -1;
The select fails for any value in the “coelorachis > x” clause above (i.e., > -1, > 0, > 1, etc)
If I run the above select and have NOT written any negative numbers to the column, all is good. It only fails if there are negative numbers stored in the tinyint column.
The merovingian.log file has these messages:
2016-05-25 12:47:23 ERR testDB[21730]: *** Error in `/usr/bin/mserver5': free(): invalid next size (fast): 0x00007fb0ac003920 ***
2016-05-25 12:47:23 ERR testDB[21730]: ======= Backtrace: =========
2016-05-25 12:47:23 ERR testDB[21730]: /lib64/libc.so.6(+0x7d023)[0x7fb0df3bc023]
2016-05-25 12:47:23 ERR testDB[21730]: /lib64/libbat.so.12(GDKfree+0x13)[0x7fb0e1b21233]
2016-05-25 12:47:23 ERR testDB[21730]: /lib64/libbat.so.12(BATsubunique+0x18a9)[0x7fb0e1d23f69]
2016-05-25 12:47:23 ERR testDB[21730]: /lib64/libbat.so.12(BATsubunique+0xbe7)[0x7fb0e1d232a7]
2016-05-25 12:47:23 ERR testDB[21730]: /lib64/libbat.so.12(BATimprints+0x581)[0x7fb0e1cef791]
2016-05-25 12:47:23 ERR testDB[21730]: /lib64/libbat.so.12(BATsubselect+0x4ecf)[0x7fb0e1ac532f]
2016-05-25 12:47:23 ERR testDB[21730]: /lib64/libbat.so.12(BATthetasubselect+0x14b)[0x7fb0e1ac78eb]
2016-05-25 12:47:23 ERR testDB[21730]: /lib64/libmonetdb5.so.19(ALGthetasubselect2+0x1c4)[0x7fb0e210db64]
2016-05-25 12:47:23 ERR testDB[21730]: /lib64/libmonetdb5.so.19(runMALsequence+0xdeb)[0x7fb0e20ac55b]
2016-05-25 12:47:23 ERR testDB[21730]: /lib64/libmonetdb5.so.19(+0x45e5b)[0x7fb0e20ade5b]
2016-05-25 12:47:23 ERR testDB[21730]: /lib64/libpthread.so.0(+0x7dc5)[0x7fb0df707dc5]
2016-05-25 12:47:23 ERR testDB[21730]: /lib64/libc.so.6(clone+0x6d)[0x7fb0df43528d]
2016-05-25 12:47:23 ERR testDB[21730]: ======= Memory map: ========
2016-05-25 12:47:23 ERR testDB[21730]: 00400000-00405000 r-xp 00000000 08:03 22036485 /usr/bin/mserver5
2016-05-25 12:47:23 ERR testDB[21730]: 00604000-00605000 r--p 00004000 08:03 22036485 /usr/bin/mserver5
2016-05-25 12:47:23 ERR testDB[21730]: 00605000-00606000 rw-p 00005000 08:03 22036485 /usr/bin/mserver5
2016-05-25 12:47:23 ERR testDB[21730]: 00606000-00608000 rw-p 00000000 00:00 0
2016-05-25 12:47:23 ERR testDB[21730]: 00892000-01e3e000 rw-p 00000000 00:00 0 [heap]
2016-05-25 12:47:23 ERR testDB[21730]: 7fabab28a000-7fabab29f000 r-xp 00000000 08:03 22020111 /usr/lib64/libgcc_s-4.8.3-20140911.so.1
2016-05-25 12:47:23 ERR testDB[21730]: 7fabab29f000-7fabab49e000 ---p 00015000 08:03 22020111 /usr/lib64/libgcc_s-4.8.3-20140911.so.1
2016-05-25 12:47:23 ERR testDB[21730]: 7fabab49e000-7fabab49f000 r--p 00014000 08:03 22020111 /usr/lib64/libgcc_s-4.8.3-20140911.so.1
2016-05-25 12:47:23 ERR testDB[21730]: 7fabab49f000-7fabab4a0000 rw-p 00015000 08:03 22020111 /usr/lib64/libgcc_s-4.8.3-20140911.so.1
Any ideas? There is no problem if I use Byte.MIN_VALUE instead of -1. This translates to NULL and is selectable.
Lynn