December 2011

December 2011 sjoerd Fri, 04/27/2012 - 14:22

MonetDB Dec2011 Release



Dec 2011-SP2 bugfix release

Java Module
  • Implemented missing Number types support in PreparedStatement.setObject()
Bug Fixes
  • 2915: Query (mclient or jdbc) with 2 outer joins appears to crash connectionor server
  • 2947: Queries where a "is null" clause is combined with two equals clauses fail to produce the expected result.
  • 2968: Memory Leaks
  • 2974: thetajoin produces invalid/inconsistent bat
  • 2988: Subquery in select part goes wrong
  • 2998: Equivalent multi-way join queries give different results
  • 3002: String plus interval result in program errors upon comparison
  • 3011: count(*) in subquery results in 1 for query without results
  • 3012: bitwise comparison fails after a cast
  • 3029: test sql/test/Dependencies/Tests/Dependencies.sql fails after recent changes
  • 3031: Virtually all SQL tests fail on Windows
  • 3036: optimizers are not getting set correctly through set optimizer
  • 3038: ~190 SQL tests fail on Darwin, FreeBSD & Solaris (only) since introduction of "(extended) relational algebra"
  • 3043: WITH clause is not allowed in derived table
  • 3045: algebra.project(0@0,bat) fails at propcheck
  • 3052: Wrong Error Code on Rollback Exception


Dec 2011-SP1 bugfix release

Build Environment
  • Fixed some of the package names for Debian/Ubuntu. Packages for libraries should contain the major number of the library version number. This was not always the case.
  • On Debian and Ubuntu, install Python modules in dist-packages instead of site-packages. This fixed bug 2997.
  • Fixed a crash that happened at the end of a database upgrade to the Dec2011 database scheme. The crash happened during cleanup after the database was upgraded, so it was merely inconvenient.
  • Stripped off implementation-specific parts from error messages before they get presented to the user.
Java Module
  • Fixed bug in DatabaseMetaData.getSchemas() method that caused an SQL error when called with catalog and schema argument.
  • Resolved a bug where JDBC and Control connections could terminate abruptly with 'Connection closed' messages
MonetDB5 Server
  • Fixed potential crash by dealing better with non-standard types.
  • Fixed a bug in the multiplex-funnel where certain clients would abort on responses for update queries.
  • Fixed problem where version and mserver properties for monetdbd were not always successfully retrieved. Bug #2982.
  • Fixed problem where shutdown of monetdbd would lead to shutting down database 'control' which does not exist. Bug #2983.
  • Fixed issue causing (harmless) 'error reading from control channel' messages. Bug #2984.
  • Resolved problem where remote start/stop/etc. commands with monetdb would report error 'OK'. Bug #2984.
Bug Fixes
  • 2862: foreign key self references cause violation upon alter table add constraint...
  • 2933: "TypeException: algebra.join undefined" when evaluating EXISTS (SELECT ...) predicate
  • 2946: Invalid BAT if left join condition has only columns from right table
  • 2964: LEFT OUTER JOIN with constants returns errors
  • 2969: Result precision issue with Decimal data type
  • 2973: Date coercion functionality tests
  • 2979: misc merovingian control problems
  • 2982: monetdbd status and mserver properties have random values
  • 2983: monetdbd attempts to stop non-existing database 'control'
  • 2984: monetdb create/release/start/stop returns error "OK" for remote connections
  • 2995: Test geom/sql/Tests/example.sql crashes since changeset 6399077ff8a4
  • 2997: monetdb-python packages installed in wrong dir on Ubuntu 10.04 LTS (lucid)
  • 3004: Incorrect evaluation of conditions in WHERE part of SQL statement
  • 3009: Segmentation fault on simple query
  • 3013: calc.and is not yet defined for dbl
  • 3014: dayofweek comment is wrong in
  • 3017: rel_bin.c:611: exp_bin: Assertion `0' failed. with between on multicolumns
  • 3020: Assertion error in debugging mode
  • 3034: INNER JOIN with Subquery produces mal errors
  • 3041: WHERE condition ignored after 2 LEFT JOINs with subqueries


Dec 2011 feature release

  • mclient: The csv output format can now also be of the form csv=c and csv+c where c is the column separator. The form with csv+ produces a single header line with column names.
  • Changed a bug where the sign() function returned the same type as its argument instead of always an INTEGER.
  • Added a fix for bug #2834, which caused weird (failing) behaviour with PreparedStatements.
  • Fixed a bug when using default values for interval columns. Bug 2877.
  • A bug was fixed where deleted rows weren't properly accounted for in all operations. This was bug 2882.
  • A bug was fixed which caused an update to an internal table to happen too soon. The bug could be observed on a multicore system with a query INSERT INTO t (SELECT * FROM t) when the table t is "large enough". This was bug 2883.
Java Module
  • Implemented getMetaData() method of PreparedStatement.
  • Fixed an AssertionError for special results from e.g. EXPLAIN queries.
  • Fixed overflow error when batching large statements, bug #2952
  • Resolved a concurrency problem where ResultSet's date-related getters could cause odd stack traces when used by multiple threads at the same time, bug #2950.
  • JDBC now implements JDBCv4.1 which makes it possible to be built with Java 7 and up. JDBCv4.1 is a maintenance release of JDBCv4, and hence can be compiled with Java 6, although the added methods obviously are not part of the java.sql interfaces.
  • SQLExceptions thrown now carry a SQLSTATE. Until the server starts sending correct SQLSTATEs for all errors, server originated errors without SQLSTATE are considered generic data exceptions (22000).
  • Fixed a bug where closing ResultSets and PreparedStatements could lead to errors on concurrent running queries using the same Connection due to a race condition.
  • Changed version scheme of JDBC driver and MCL jar to be more standard, from monetdb-X.Y-<thing>.jar to monetdb-<thing>-X.Y.jar, bug #2943
  • The embedded monet instance capability of MonetConnection was removed.
  • Bump JDBC version to 2.0 (Liberica). JDBC now implements JDBCv4 which makes it possible to be built with Java 6 and up. Java 5 and before are no longer supported, and can use the 1.x releases of the driver.
  • INTERVAL columns are now treated as decimals, since they can have sub-second precision.
Client Package
  • Fix making connections with stethoscope to hosts without monetdbd. Bug #2944.
  • Fixed some bugs in the PHP interface affecting the mapi_ping(), monetdb_insert_id() and auto_commit() functions. Bugs #2936, #2937, #2938.
  • ODBC: Fixed SQLNumResultCols and SQLDescribeCol to return useful information after a call to SQLPrepare and before any SQLExecute.
  • mclient: The exact interpretation of the -i (--interactive) option and the - filename argument have changed. The - filename argument means read from standard input and no longer implies that no \ interpretation is done on the input. Instead, \ interpretation is done if either standard input is a terminal, or if the -i option is given. The -i option no longer causes mclient to read from standard input. It only means to do \ interpretation when reading from standard input. Use the - filename argument to read from standard input. Note that if no -s option is specified and no filename arguments are present, mclient still reads from standard input.
  • ODBC: Improved internal query for SQLSpecialColumns. Before, the query returned all columns taking part in a PRIMARY KEY *and* all columns taking part in a UNIQUE constraint. Now it returns only one or the other set.
  • ODBC: The database name is now used as the catalog name throughout. Functions that return a catalog name return the database name, and functions that match on catalog name match it with the database name.
  • ODBC: Implemented an easier way to create a log file of interactions with the ODBC driver. You can now add a connection attribute "LOGFILE=filename" to the connection string parameter of SQLBrowseConnect and SQLDriverConnect, and to the relevant part of the Windows registry or odbc.ini file. This value is only used if there is no environment variable ODBCDEBUG.
  • Quoting of object names for mclient's \d command is now more flexible and consistent with standard SQL quoting rules, bug #2846.
  • ODBC: Implemented SQLColumnPrivileges function.
  • Install new program, mnc, which provides netcat functionality, but based on MonetDB's communication libraries
  • Report full URI to database instead of just database when available in mclient.
  • The time format of the timer output can now be controlled with an optional argument to the -i option. ms, s and m force the time to be formatted as milliseconds, seconds or minutes + seconds respectively.
  • ODBC: Implemented a workaround in SQLTables for bug 2908.
  • Small improvement to mclient's table rendering for tables without any rows. Previously, the column names in the header could be squeezed to very small widths, degrading readability.
  • Python DB API connect() function now supports PEP 249-style arguments user and host, bug #2901
  • mclient now checks the result of encoding conversions using the iconv library.
  • Fixed a source of crashes in mclient when a query on the command line using the -s option is combined with input on standard input (e.g. in the construct mclient -s 'COPY INTO t FROM STDIN ...' < file.csv).
  • Fixed bug 2897 where slow (network) reads could cause blocks to not be fully read in one go, causing errors in the subsequent use of those blocks. With thanks to Rémy Chibois.
  • Fixed a bug in mclient where processing queries from files could result in ghost empty results to be reported in the output
  • Fixed Perl DBD rowcount for larger results, bug #2889
  • Perl: We now distinguish properly between TABLE and GLOBAL TEMPORARY (the latter are recognized by being in the "tmp" schema).
  • Perl: fixed a bunch of syntax errors. This fixes bug 2884. With thanks to Rémy Chibois.
  • Perl: Fixed DBD::monetdb table_info and tabletype_info. This fixes bug 2885. With thanks to Rémy Chibois.
  • mclient: fix display of varchar columns with only NULL values.
  • Fixed a bug in mclient/msqldump where an internal error occurred during dump when there are GLOBAL TEMPORARY tables.
MonetDB5 Server
  • A couple of memory leaks have been plugged.
  • Introduced new variable max_clients that allows to define how many concurrent connections are allowed to be made against the database.
  • Protect against concurrent exceptions If multiple parallel blocks access the exception variables and perform GDKfree on old messages, then we may enter a case that one thread attempts a second free. Simple lock-based protection is the first line of defense.
  • bpm and partitions have been moved to the attic. It is replaced by the partition optimizer, still under construction.
  • The MAL debugger next/step operations semantics have been fixed. The profiler also now clearly shows entrance/exit of a MAL function.
  • The dataflow optimizer uses a less strict side-effect test for BAT new operations and better recognizes trivial plans.
  • Fixed a problem where MAL variables weren't properly cleared before reuse of the data strucutre. This problem could cause the data flow scheduler to generate dependencies between instructions that didn't actually exist, which in turn could cause circular dependencies among instructions with deadlock as a result. Bugs 2865 and 2888.
  • Fixed crash in monetdb when an invalid property was retrieved using the get command, bug #2953.
  • monetdb get output is now grouped by database instead of by property
  • Unlike in previous releases, 'monetdbd get mserver' now returns the path to the mserver5 binary only for dbfarms that have a running monetdbd, instead of only for those that are not served by a monetdbd.
  • Add nclients property that controls the maximum number of concurrent clients allowed to access the database
  • Starting monetdbd without any arguments or without dbfarm is no longer supported. A dbfarm now must be provided for each command.
  • The control passphrase has been turned into a hash of the password, for a more stronger authorisation model as used by mclients. On upgrade, any existing passphrase is converted to the hashed version of the passphrase.
  • The monetdbd controlport option has been removed in favour of a boolean control option. On upgrade, when controlport was set to a non-zero value and a passphrase was set, control is set to true, or false otherwise.
  • The monetdbd discoveryport option has been removed in favour of a boolean discovery option. On upgrade, when discoveryport was set to a non-zero value, discovery is set to true, or false otherwise.
  • Fixed a bug where monetdbd's socket files from /tmp were removed when a second monetdbd was attempted to be started using the same port.
  • Resolved problem where monetdbd would terminate abnormally when databases named 'control', 'discovery' or 'merovingian' were stopped.
  • monetdbd get status now also reports the version of the running monetdbd
  • Improved response time of 'monetdb start' when the database fails to start.
  • Fixed a bug in monetdbd where starting a failing database could incorrectly be reported as a 'running but dead' database.
  • To avoid confusion, all occurrences of merovingian were changed into monetdbd for error messages sent to a client.
MonetDB Common
  • Fixed memory detection on 32-bits Solaris systems with more memory than can be addressed in 32-bits mode, bug #2935
Bug Fixes
  • 2740: Segmentation fault using a column inside of a functioncall
  • 2802: update JDBC driver to JDBC4
  • 2846: \d command fails if the schema is provided explicitly
  • 2873: SQL: time with precision
  • 2891: Make fetch results progressively
  • 2910: SQLSetStmtOption(..., SQL_CONCURRENCY, SQL_CONCUR_READ_ONLY) should be allowed
  • 2914: mserver5 crash for huge_nr_of_columns.Bug-2609 on ppc-macos
  • 2934: Database upgrade with SQL catalog upgrade messes up privileges
  • 2935: mserver5 doesn't detect memory on Solaris/Sparc
  • 2936: bug into mapi_ping()
  • 2937: minor bug into monetdb_insert_id() (php_monetdb.php)
  • 2938: minor bug into auto_commit() (php_monetdb.php)
  • 2939: "PHP Notice" in mapi_execute()
  • 2940: PHP split() function deprecated
  • 2944: stethoscope on can't connect to database without monetdbd (discovery enabled)
  • 2950: MonetResultSet use of SimpleDateFormat not threadsafe
  • 2952: utf-8 characters in long string
  • 2953: segfault for monetdb get with invalid property
  • 2959: Nested SELECT with EXISTS causes the server to crash
  • 2963: invalid tuple received from server, got 17 columns, expected 24, ignoring
  • 2965: joinPath runs indeterminately
  • 2975: CSV file is kept write-locked after mclient failed to COPY
  • 2981: Different answer from querys in the ODBC compared to mcliente or other DBMSs