Question on MonetDB Source Code Compilation
All,
Here is a quick question. I have compiled "MonetDB v11.27.9 (Jul2017-SP2)" version of source code and while running below sample SQL am finding an issue. Could anyone help me to trace issue? Is this due to the compiler version?
Issue Details: While using SUM() or any function without an alias the header datatype returned is showing as HUGEINT for base INTEGER column, where as it should have been BIGINT. This is causing a failure in my BI application as it's having issues in identifying HUGEINT.
Below are the details.
Source Code Version : MonetDB v11.27.9 (Jul2017-SP2) Linux OS Version : CentOS release 6.3 (Final) C Compile Version : gcc version 4.4.7 20120313 (Red Hat 4.4.7-16) (GCC)
Also, I see below note in MonetDB documentation. Which version of GCC should be appropriate for compilation with __128 bit integer.
Note: HUGEINT is only available since the Jul2015 release and on platforms with a C-compiler that supports the __int128 or __int128_t data type (e.g., recent gcc, clang, & icc on Linux or MacOS X)
Sample Table DDL : CREATE TABLE INT_VALID (COL_INT INTEGER, COL_BIGINT BIGINT, COL_HUGEINT HUGEINT, COL_DOUBLE DOUBLE, COL_NUMERIC NUMERIC(10));
Compiled Version sql>SELECT SUM(col_int) FROM INT_VALID; mapi_query_part:36:SELECT SUM(col_int) FROM INT_VALID;
fetch next block: start at:10093 got next block: length:105 text:&1 4 1 1 1 2452 % DM_POS_TSV_MRT_P6A.L4 # table_name % L3 # name % hugeint # type % 5 # length [ 11110 ]
Binary Version sql>SELECT SUM(col_int) FROM INT_VALID; mapi_query_part:36:SELECT SUM(col_int) FROM INT_VALID;
fetch next block: start at:1168 got next block: length:105 text:&1 4 1 1 1 13998 % DM_POS_TSV_MRT_P6A.L4 # table_name % L3 # name % bigint # type % 5 # length [ 11110 ]
+---------+------------+--------------------------+-------------+ | col_int | col_bigint | col_double | col_numeric | +=========+============+==========================+=============+ | 1111 | 1234567 | 1234.44 | 1111 | | 2222 | 2345678 | 34355.66 | 2222 | | 3333 | 3456789 | 66423.66 | 3333 | | 4444 | 45678912 | 8765.66 | 4444 | +---------+------------+--------------------------+-------------+
Regards, Sreejith
Hi,
please find here my earlier explanation why MonetDB chooses the largest available data type for aggregations: https://www.monetdb.org/pipermail/users-list/2018-January/010102.html
The largest integer type available in MonetDB depends the what the compiler supports and what is chosen at compile time (see configure options). Recent versions of GNU gcc and Intel icc support (up to) 128-bit on most "Unix-like" OSs, while (AFAIK) Microsoft Visual Studio does not support 128-bit integers on Windows. For an overview of SQL types supported by MonetDB see also: https://www.monetdb.org/Documentation/Manuals/SQLreference/BuiltinTypes
From your example, I conclude that you compiled your MonetDB with 128-bit
support (the default, if the compiler supports it), while for your binary version you did not install the 128-bit integer (aka. "hugeint") RPMs (MonetDB5-server-hugeint and MonetDB-SQL-server5-hugeint).
To achieve the same behavior with both your binary-installed and your self-compiled version of MonetDB, you can either install the above mentioned hugeint RPMs to enable 128-bit integers ("hugeint") or configure the sources with --disable-int128 and re-compile from scratch to disable 128-bit integers ("hugeint").
In either case, you'd need to re-create your databases.
Best, Stefan
----- On Jan 17, 2018, at 5:44 PM, Sreejith Sharma Sreejith.Sharma@harman.com wrote:
All,
Here is a quick question. I have compiled “MonetDB v11.27.9 (Jul2017-SP2)” version of source code and while running below sample SQL am finding an issue. Could anyone help me to trace issue? Is this due to the compiler version?
Issue Details:
While using SUM() or any function without an alias the header datatype returned is showing as HUGEINT for base INTEGER column, where as it should have been BIGINT. This is causing a failure in my BI application as it’s having issues in identifying HUGEINT.
Below are the details.
Source Code Version : MonetDB v11.27.9 (Jul2017-SP2)
Linux OS Version : CentOS release 6.3 (Final)
C Compile Version : gcc version 4.4.7 20120313 (Red Hat 4.4.7-16) (GCC)
Also, I see below note in MonetDB documentation. Which version of GCC should be appropriate for compilation with __128 bit integer.
Note : HUGEINT is only available since the Jul2015 release and on platforms with a C-compiler that supports the __int128 or __int128_t data type (e.g., recent gcc, clang, & icc on Linux or MacOS X)
Sample Table DDL :
CREATE TABLE INT_VALID (COL_INT INTEGER, COL_BIGINT BIGINT, COL_HUGEINT HUGEINT, COL_DOUBLE DOUBLE, COL_NUMERIC NUMERIC(10));
Compiled Version
sql>SELECT SUM(col_int) FROM INT_VALID;
mapi_query_part:36:SELECT SUM(col_int) FROM INT_VALID;
fetch next block: start at:10093
got next block: length:105
text:&1 4 1 1 1 2452
% DM_POS_TSV_MRT_P6A.L4 # table_name
% L3 # name
% hugeint # type
% 5 # length
[ 11110 ]
Binary Version
sql>SELECT SUM(col_int) FROM INT_VALID;
mapi_query_part:36:SELECT SUM(col_int) FROM INT_VALID;
fetch next block: start at:1168
got next block: length:105
text:&1 4 1 1 1 13998
% DM_POS_TSV_MRT_P6A.L4 # table_name
% L3 # name
% bigint # type
% 5 # length
[ 11110 ]
+---------+------------+--------------------------+-------------+
| col_int | col_bigint | col_double | col_numeric |
+=========+============+==========================+=============+
| 1111 | 1234567 | 1234.44 | 1111 |
| 2222 | 2345678 | 34355.66 | 2222 |
| 3333 | 3456789 | 66423.66 | 3333 |
| 4444 | 45678912 | 8765.66 | 4444 |
+---------+------------+--------------------------+-------------+
Regards,
Sreejith
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
Thanks Stefan for your suggestion. I will try and let you know.
-----Original Message----- From: users-list [mailto:users-list-bounces+sreejith.sharma=harman.com@monetdb.org] On Behalf Of Stefan Manegold Sent: Thursday, January 18, 2018 12:03 AM To: Communication channel for MonetDB users users-list@monetdb.org Subject: [EXTERNAL] Re: Question on MonetDB Source Code Compilation
Hi,
please find here my earlier explanation why MonetDB chooses the largest available data type for aggregations: https://clicktime.symantec.com/a/1/_XmmlDJf2ywVr4Yh_BEELCWUegINiFOZgqLVhPmYA...
The largest integer type available in MonetDB depends the what the compiler supports and what is chosen at compile time (see configure options). Recent versions of GNU gcc and Intel icc support (up to) 128-bit on most "Unix-like" OSs, while (AFAIK) Microsoft Visual Studio does not support 128-bit integers on Windows. For an overview of SQL types supported by MonetDB see also: https://clicktime.symantec.com/a/1/baayVyq7Scx_TCybbsSRlp8BKtxcbujGCV3lxWtIa...
From your example, I conclude that you compiled your MonetDB with 128-bit support (the default, if the compiler supports it), while for your binary version you did not install the 128-bit integer (aka. "hugeint") RPMs (MonetDB5-server-hugeint and MonetDB-SQL-server5-hugeint).
To achieve the same behavior with both your binary-installed and your self-compiled version of MonetDB, you can either install the above mentioned hugeint RPMs to enable 128-bit integers ("hugeint") or configure the sources with --disable-int128 and re-compile from scratch to disable 128-bit integers ("hugeint").
In either case, you'd need to re-create your databases.
Best, Stefan
----- On Jan 17, 2018, at 5:44 PM, Sreejith Sharma Sreejith.Sharma@harman.com wrote:
All,
Here is a quick question. I have compiled “MonetDB v11.27.9 (Jul2017-SP2)” version of source code and while running below sample SQL am finding an issue. Could anyone help me to trace issue? Is this due to the compiler version?
Issue Details:
While using SUM() or any function without an alias the header datatype returned is showing as HUGEINT for base INTEGER column, where as it should have been BIGINT. This is causing a failure in my BI application as it’s having issues in identifying HUGEINT.
Below are the details.
Source Code Version : MonetDB v11.27.9 (Jul2017-SP2)
Linux OS Version : CentOS release 6.3 (Final)
C Compile Version : gcc version 4.4.7 20120313 (Red Hat 4.4.7-16) (GCC)
Also, I see below note in MonetDB documentation. Which version of GCC should be appropriate for compilation with __128 bit integer.
Note : HUGEINT is only available since the Jul2015 release and on platforms with a C-compiler that supports the __int128 or __int128_t data type (e.g., recent gcc, clang, & icc on Linux or MacOS X)
Sample Table DDL :
CREATE TABLE INT_VALID (COL_INT INTEGER, COL_BIGINT BIGINT, COL_HUGEINT HUGEINT, COL_DOUBLE DOUBLE, COL_NUMERIC NUMERIC(10));
Compiled Version
sql>SELECT SUM(col_int) FROM INT_VALID;
mapi_query_part:36:SELECT SUM(col_int) FROM INT_VALID;
fetch next block: start at:10093
got next block: length:105
text:&1 4 1 1 1 2452
% DM_POS_TSV_MRT_P6A.L4 # table_name
% L3 # name
% hugeint # type
% 5 # length
[ 11110 ]
Binary Version
sql>SELECT SUM(col_int) FROM INT_VALID;
mapi_query_part:36:SELECT SUM(col_int) FROM INT_VALID;
fetch next block: start at:1168
got next block: length:105
text:&1 4 1 1 1 13998
% DM_POS_TSV_MRT_P6A.L4 # table_name
% L3 # name
% bigint # type
% 5 # length
[ 11110 ]
+---------+------------+--------------------------+-------------+
| col_int | col_bigint | col_double | col_numeric |
+=========+============+==========================+=============+
| 1111 | 1234567 | 1234.44 | 1111 |
| 2222 | 2345678 | 34355.66 | 2222 |
| 3333 | 3456789 | 66423.66 | 3333 |
| 4444 | 45678912 | 8765.66 | 4444 |
+---------+------------+--------------------------+-------------+
Regards,
Sreejith
users-list mailing list users-list@monetdb.org https://clicktime.symantec.com/a/1/PBHZ6d6Ct9H92RlEAQHNLsc5R3qz7xYvM4M hjAOjmNU=?d=TSqK0IGPM-XP_noQ5QNg6_s30FM0SVKfJzVmDVHvN0jNBkIWhw6izV9JP4 seY3jUR0HwOAQ43yyYalrGpjCYca-4zSv_hbnN7jPwC2g2cuhfe7yV0AvWrcS1CtMwfEoQ O9MXkqOJ9kcbW9CLYaCPcLyS3bTJwqwoMZ9ah-d40RmP6UNh2zkDGlb9hVUr21JLbJHWY- SQKZmKL-_h-3Lcw58_VpPwv9sJkczoL1s7REKQv7PaL0y-wo6orFfwcgWKhUeyIsU1JAjk m5bELKrIv7wSH5z6yyiFVyqo9_cYfwq_9OnQGVEizQFsHKxUnL72UTFTBL9gxLILbtvWdx UcZA2MU3Cf6XDrp3zVhkaJjiO_lMKdUQ9ljz_H2SrWLubdM3wU1flvingEtxNCovk2hwDn eZF3KF7hWQVLXQVB8y3baUbwvwFSv7TP9_gD22vRknU-kGVs&u=https%3A%2F%2Fwww.m onetdb.org%2Fmailman%2Flistinfo%2Fusers-list
Hi Stefan,
As suggested, I have tried compiling disabling 128 bit integer. However, when I try to create a DB and bring it up, it crashes with below mentioned error. Is there anything else am missing?
Here is the steps I followed -
./configure --prefix=/data/MDB_COMPILE/MonetDB-11.27.11_W128 --disable-int128 Make Make clean Make install
* Important options: OID size: 64 bits largest integer type: 64 bits largest decimal type: 18 digits
Error ----------- 2018-01-22 02:26:44 MSG DB1[46369]: # MonetDB 5 server v11.27.11 "Jul2017-SP3" 2018-01-22 02:26:44 MSG DB1[46369]: # Serving database 'DB1', using 128 threads 2018-01-22 02:26:44 MSG DB1[46369]: # Compiled for x86_64-pc-linux-gnu/64bit 2018-01-22 02:26:44 MSG DB1[46369]: # Found 1.968 TiB available main-memory. 2018-01-22 02:26:44 MSG DB1[46369]: # Copyright (c) 1993-July 2008 CWI. 2018-01-22 02:26:44 MSG DB1[46369]: # Copyright (c) August 2008-2017 MonetDB B.V., all rights reserved 2018-01-22 02:26:44 MSG DB1[46369]: # Visit https://www.monetdb.org/ for further information 2018-01-22 02:26:44 MSG DB1[46369]: # Listening for UNIX domain connection requests on mapi:monetdb:///data/DBFARM1/DB1/.mapi.sock 2018-01-22 02:26:44 MSG DB1[46369]: # MonetDB/SQL module loaded 2018-01-22 02:26:44 MSG merovingian[40860]: target connection is on local UNIX domain socket, passing on filedescriptor instead of proxying 2018-01-22 02:26:48 MSG merovingian[40860]: database 'DB1' (46369) has crashed (dumped core)
Regards, Sreejith
-----Original Message----- From: users-list [mailto:users-list-bounces+sreejith.sharma=harman.com@monetdb.org] On Behalf Of Stefan Manegold Sent: Thursday, January 18, 2018 12:03 AM To: Communication channel for MonetDB users users-list@monetdb.org Subject: [EXTERNAL] Re: Question on MonetDB Source Code Compilation
Hi,
please find here my earlier explanation why MonetDB chooses the largest available data type for aggregations: https://clicktime.symantec.com/a/1/_XmmlDJf2ywVr4Yh_BEELCWUegINiFOZgqLVhPmYA...
The largest integer type available in MonetDB depends the what the compiler supports and what is chosen at compile time (see configure options). Recent versions of GNU gcc and Intel icc support (up to) 128-bit on most "Unix-like" OSs, while (AFAIK) Microsoft Visual Studio does not support 128-bit integers on Windows. For an overview of SQL types supported by MonetDB see also: https://clicktime.symantec.com/a/1/baayVyq7Scx_TCybbsSRlp8BKtxcbujGCV3lxWtIa...
From your example, I conclude that you compiled your MonetDB with 128-bit support (the default, if the compiler supports it), while for your binary version you did not install the 128-bit integer (aka. "hugeint") RPMs (MonetDB5-server-hugeint and MonetDB-SQL-server5-hugeint).
To achieve the same behavior with both your binary-installed and your self-compiled version of MonetDB, you can either install the above mentioned hugeint RPMs to enable 128-bit integers ("hugeint") or configure the sources with --disable-int128 and re-compile from scratch to disable 128-bit integers ("hugeint").
In either case, you'd need to re-create your databases.
Best, Stefan
----- On Jan 17, 2018, at 5:44 PM, Sreejith Sharma Sreejith.Sharma@harman.com wrote:
All,
Here is a quick question. I have compiled “MonetDB v11.27.9 (Jul2017-SP2)” version of source code and while running below sample SQL am finding an issue. Could anyone help me to trace issue? Is this due to the compiler version?
Issue Details:
While using SUM() or any function without an alias the header datatype returned is showing as HUGEINT for base INTEGER column, where as it should have been BIGINT. This is causing a failure in my BI application as it’s having issues in identifying HUGEINT.
Below are the details.
Source Code Version : MonetDB v11.27.9 (Jul2017-SP2)
Linux OS Version : CentOS release 6.3 (Final)
C Compile Version : gcc version 4.4.7 20120313 (Red Hat 4.4.7-16) (GCC)
Also, I see below note in MonetDB documentation. Which version of GCC should be appropriate for compilation with __128 bit integer.
Note : HUGEINT is only available since the Jul2015 release and on platforms with a C-compiler that supports the __int128 or __int128_t data type (e.g., recent gcc, clang, & icc on Linux or MacOS X)
Sample Table DDL :
CREATE TABLE INT_VALID (COL_INT INTEGER, COL_BIGINT BIGINT, COL_HUGEINT HUGEINT, COL_DOUBLE DOUBLE, COL_NUMERIC NUMERIC(10));
Compiled Version
sql>SELECT SUM(col_int) FROM INT_VALID;
mapi_query_part:36:SELECT SUM(col_int) FROM INT_VALID;
fetch next block: start at:10093
got next block: length:105
text:&1 4 1 1 1 2452
% DM_POS_TSV_MRT_P6A.L4 # table_name
% L3 # name
% hugeint # type
% 5 # length
[ 11110 ]
Binary Version
sql>SELECT SUM(col_int) FROM INT_VALID;
mapi_query_part:36:SELECT SUM(col_int) FROM INT_VALID;
fetch next block: start at:1168
got next block: length:105
text:&1 4 1 1 1 13998
% DM_POS_TSV_MRT_P6A.L4 # table_name
% L3 # name
% bigint # type
% 5 # length
[ 11110 ]
+---------+------------+--------------------------+-------------+
| col_int | col_bigint | col_double | col_numeric |
+=========+============+==========================+=============+
| 1111 | 1234567 | 1234.44 | 1111 |
| 2222 | 2345678 | 34355.66 | 2222 |
| 3333 | 3456789 | 66423.66 | 3333 |
| 4444 | 45678912 | 8765.66 | 4444 |
+---------+------------+--------------------------+-------------+
Regards,
Sreejith
users-list mailing list users-list@monetdb.org https://clicktime.symantec.com/a/1/PBHZ6d6Ct9H92RlEAQHNLsc5R3qz7xYvM4M hjAOjmNU=?d=TSqK0IGPM-XP_noQ5QNg6_s30FM0SVKfJzVmDVHvN0jNBkIWhw6izV9JP4 seY3jUR0HwOAQ43yyYalrGpjCYca-4zSv_hbnN7jPwC2g2cuhfe7yV0AvWrcS1CtMwfEoQ O9MXkqOJ9kcbW9CLYaCPcLyS3bTJwqwoMZ9ah-d40RmP6UNh2zkDGlb9hVUr21JLbJHWY- SQKZmKL-_h-3Lcw58_VpPwv9sJkczoL1s7REKQv7PaL0y-wo6orFfwcgWKhUeyIsU1JAjk m5bELKrIv7wSH5z6yyiFVyqo9_cYfwq_9OnQGVEizQFsHKxUnL72UTFTBL9gxLILbtvWdx UcZA2MU3Cf6XDrp3zVhkaJjiO_lMKdUQ9ljz_H2SrWLubdM3wU1flvingEtxNCovk2hwDn eZF3KF7hWQVLXQVB8y3baUbwvwFSv7TP9_gD22vRknU-kGVs&u=https%3A%2F%2Fwww.m onetdb.org%2Fmailman%2Flistinfo%2Fusers-list
participants (2)
-
Sharma, Sreejith -
Stefan Manegold