Release history

Release history mk Sun, 05/15/2011 - 11:55

The trail of release notes as of the first time we made the code base available on SourceForge.

Our current release notes can be found in here.

December 2016

December 2016 sjoerd Mon, 07/17/2017 - 15:49

Dec 2016-SP5 bugfix release

Bug Fixes
  • 6290: Crash (and assertion failure) with a correlated subquery with NOT IN in the select-list
  • 6291: crash if executes a function with sample operator
  • 6294: Sqlitelogictest crash
  • 6296: Another sqllitelogictest crash :(
  • 6297: 7th sqllitelogictest crash :(
  • 6300: Protect against missing BATs (sqlsmith)
  • 6314: Lateral crash report (sqlsmith)
  • 6315: Exist operator on type bigint missing (sqlsmith)
  • 6316: Coalesc and limit error (sqlsmith)
  • 6317: Two-column integer aggregation extremely slow
  • 6318: Daemon crashes if .merovignian_properties for a database contains more than 42 entries
  • 6320: The daemon ignores all but the last entry in the .merovignian_properties file when spawning mserver5

Dec 2016-SP4 bugfix release

MonetDB5 Server
  • Fixed a bug causing a crash during cleanup when mserver5 is stopped with monetdb stop database.
  • Some memory leaks were plugged.
MonetDB Common
  • A potential deadlock was fixed in order index creation.
  • A bug that could happen during recovery of the write-ahead log (WAL) was fixed. See changeset 98ad79c555cc for details.
SQL
  • Some memory leaks were plugged.
  • Upgrade code was added for an old change in the sys.settimeout function.
  • A bug was fixed with the automatic "vacuum" operation on system tables.
Bug Fixes
  • 6110: cast of a SQL boolean value to a string or clob or (var)char is wrong
  • 6254: Crash (and assertion failure) after querying a view which uses a correlated subquery in the select-list
  • 6256: Assertion Trigger on FULL OUTER JOIN with more than two BETWEEN clauses
  • 6257: wrong count values (1 instead of 0) for correlated aggregation queries
  • 6258: Vulnerability in FITS and NETCDF data vaults
  • 6259: crash on select query from sqlitelogictests
  • 6260: Sqlitelogictest crash
  • 6288: Function cannot find column in merge table
  • 6295: msqldump writes unescaped timestamp values when using inserts

Dec 2016-SP3 bugfix release

MonetDB Common
  • Fixed a bug when appending string bats that are fully duplicate eliminated. It could happend that the to-be-appended bat had an empty string at an offset and at that same offset in the to-be-appended-to bat there happened to be a (sequence of) NULL(s). Then this offset would be used, even though it might nog be the right offset for the empty string in the to-be-appended-to bat. This would result in multiple offsets for the empty string, breaking the promise of being duplicate eliminated.
Bug Fixes
  • 6138: Weak duplicate elimination in string heaps > 64KB
  • 6183: ResultSet returns double quoted column name if name contains space characters
  • 6216: Assertion raised (sqlsmith)
  • 6219: Crash in rel_optimizer (sqlsmith)
  • 6227: Monetdb fails on remote tables
  • 6228: mclient crashes if real column is multiplied by it itself
  • 6229: ANALYZE, unexpected end of input
  • 6230: ANALYZE, syntax error
  • 6237: semijoin with empty right bat does not return immediately
  • 6242: Crash on rel_reduce_groupby_exps (sqlsmith)
  • 6243: Static optimization gives wrong result (1 + NULL = -127)
  • 6245: Nested query crashes all versions of MonetDB or gives wrong result starting from Dec2016-SP2
  • 6246: update statements: references to a table do not bind to its alias
  • 6247: Type analysis issue (sqlsmith)
  • 6248: update statements: the semantic stage does not resolve the relation in the from clause
  • 6250: Assertion failure when querying a Blob column with order by DESC
  • 6251: Crash after adding an ordered index on sys.statistics column and querying sys.statistics
  • 6253: FITS Data Vaults does not work when using user/pw and other than sys schema name

Dec 2016-SP2 bugfix release

Bug Fixes
  • 4034: argnames array in rapi.c has fixed length (that was too short)
  • 6080: mserver5: rel_bin.c:2391: rel2bin_project: Assertion `0' failed.
  • 6081: Segmentation fault (core dumped)
  • 6082: group.subgroup is undefined if group by is used on an expression involving only constants
  • 6111: Maximum number of digits for hge decimal is listed as 39 in sys.types. Should be 38 as DECIMAL(39) is not supported.
  • 6112: Crash upgrading Jul2015->Jun2016
  • 6130: Query rewriter crashes on a NULL pointer when having a correlated subquery
  • 6133: A crash occurs when preparing an INSERT on joined tables during the query semantic phase
  • 6141: Getting an error message regarding a non-GROUP-BY column rather than an unknown identifier
  • 6177: Server crashes
  • 6186: Null casting causes no results (silent server crash?)
  • 6189: Removing a NOT NULL constraint from a PKey column should NOT be allowed
  • 6190: CASE query crashes database
  • 6191: MT_msync failed with "Cannot allocate memory"
  • 6192: Numeric column stores wrong values after adding large numbers
  • 6193: converting to a smaller precision (fewer or no decimals after decimal point) should round/truncate consistently
  • 6194: splitpart returns truncated last part if it contains non ascii caracters
  • 6195: Cast from huge decimal type to smaller returns wrong results
  • 6196: Database crashes after generate_series query
  • 6198: COALESCE could be more optimized
  • 6201: MonetDB completely giving up on certain queries - no error and no result
  • 6202: querying a table with an ordered index on string/varchar column crashes server and makes server unrestartable!
  • 6203: copy into: Failed to import table Leftover data 'False'
  • 6205: Integer addition overflow
  • 6206: casting strings with more than one trailing zero ('0') to decimal goes wrong
  • 6209: Aggregation over complex OR expressions produce wrong results
  • 6210: Upgrading a database from Jun2015 or older crashes the server
  • 6213: SQLsmith causes server to crash
  • 6217: Segfault in rel_optimizer (sqlsmith)
  • 6218: grouped quantiles with all null group causes following groups to return null
  • 6224: mal_parser: cannot refer to types containing an underscore

Dec 2016-SP1 bugfix release

Bug Fixes
  • 4039: Slow mserver5 start after drop of tables (> 1 hour)
  • 4048: Segfault on vacuum with parallel updates
  • 6079: pushselect optimizer bug on MAL snippet
  • 6140: INNER JOIN gives the results of a CROSS JOIN
  • 6150: Query giving wrong results, extra records are appearing
  • 6175: The program can't start because python27.dll is missing from your computer.
  • 6178: AVG + GROUP BY returns NULL for some records that should have results
  • 6179: mergetable optimizer messes up sample
  • 6182: sys.shutdown triggers assertion in clients.c
  • 6184: Incorrect result set - Extra records in result set

Dec 2016 feature release

There are lots of changes in this (Dec2016) release and the previous (Jun2016).  Here is an outline.

  • The client interfaces that are not written in C, that is to say, the Java (JDBC driver), Perl, PHP, Python, and Ruby interfaces all now have their own repository and are no longer part of the main MonetDB repository.  (Actually, the Ruby interface was already separate in the Jun2016 release.)  These client interfaces are still supported but now have an independent release schedule.  Their first releases are synchronized with this release. Debian and Ubuntu users, please note that the package names for the client interfaces have changed.  If you have the old versions installed, they will not get updated automatically.  Use: sudo apt-get install libdbd-monetdb-perl php-monetdb python-pymonetdb python3-pymonetdb removing the ones from the list above you don't want.
  • For years we have been working towards a version that we've internally called "headless".  This is now complete.  This has had consequences for quite a few internal interfaces, also ones that could potentially be used by extensions written by third parties. The trickiest one is probable BATkey() which has the same calling interface but does something different.  The compiler should complain about other functions since their arguments changed or the functions were completely removed.
  • In the Jun2016 release it was possible to have user-defined functions (UDF) callable from SQL that were written in the R language.  In this release, the same is now possible for UDFs written in Python.  Also see this blog.
  • The possibility of having 32 bit OID values in a system that was compiled for a 64 bit architecture has been removed.  The size of the OID type is now always the same as the "natural" size of the architecture.  This was a compile-time feature and we haven't made any binary releases in a long time in which this feature was used. The automatic conversion from a database in which the size of the OID type was 32 to one in which the size is 64 is still in place. This will be removed in a future version.
  • The internal type "wrd" has been removed.  The "wrd" type was an architecture-dependent integer type, 32 bits wide on a 32-bit system and 64 bits wide on a 64-bit system.  Databases where this type is used will automatically have the type converted to either "int" or "lng", the internal types for 32-bit and 64-bit integers.
  • Some old and unused code and interfaces have been removed.
  • By making some adjustments to internal parameters, we have been able to get a significant improvements on query speed for many types of queries (in informal tests we have seen a 20% improvement compared to the previous release when running the queries from the TPC-H benchmark).
  • A new type of experimental column index has been introduced: the ordered index.  Since creating this index is expensive, it is only used when explicitly created.  To create an index in SQL use CALL createorderindex(schema,table,column) where the procedure arguments are all strings.
  • We have fixed quite a few memory leaks and other bugs.

 

MonetDB5 Server
  • Removed the zorder module with functions zorder.encode, zorder.decode_x and zorder.decode_y.

  • Removed command bbp.getHeat().

  • Removed bat.setColumn with two arguments and bat.setRole. Use bat.setColumn with one argument instead.

  • Removed function BKCappend_reverse_val_wrap: it was unused.

  • The "wrd" type has been removed from GDK and MAL. The type was defined to be a 32 bit integer on 32 bit architectures and a 64 bit integer on 64 bit architectures. We now generally use "lng" (always 64 bits) where "wrd" was used.

  • Keep a collection of full traces. Each time the SQL user applies the TRACE option, the full json trace is retained within the <dbpath>/<dbname>/sql_traces

Build Environment
  • New packages MonetDB-python2 (Fedora) and monetdb-python2 (Debian/Ubuntu) have been created for Python 2 integration into MonetDB.

  • With OID size equal to ABI/word size, mserver5 does not need to print the OID size, anymore.

  • Removed obsolete code associated with long gone static linking option.

  • Removed configure option --enable-oid32 to compile with 32 bit OIDs on a 64 bit architecture.

.
MonetDB Common
  • The tnokey values must now be 0 if it is not known whether all values in a column are distinct.

  • The 2-bit tkey field in the bat descriptor has been split into two single bit fields: tkey and tunique. The old tkey&BOUND2BTRUE value is now stored in tunique.

  • Implemented conversion to str from any type (not just the internal types).

  • VALcopy and VALinit both return their first argument on success or (and that's new) NULL on (allocation) failure.

  • BATattach now can also create a str BAT from a file consisting of null-terminated strings. The input file must be encoded using UTF-8.

  • BATattach now copies the input file instead of "stealing" it.

  • Removed the lastused "timestamp" from the BBP.

  • Removed batStamp field from BAT descriptor, and removed the BBPcurstamp function.

  • Removed unused functions BBPhot and BBPcold.

  • Removed BATderiveTailProps and BATderiveProps. Just set the properties you know about, or use BATsettrivprop.

  • Removed the macro BUNfirst. It can be replaced by 0.

  • Changed BATroles by removing the argument to set the name of the head column.

  • The head column is now completely gone. MonetDB is completely "headless".

  • The format of the BBP.dir file was simplified. Since the head column is VOID, the only value that needs to be stored is the head seqbase.

  • BATs now only have a single (logical) name.

  • The function BATmirror is gone. The HEAD column is always VOID (with a non-nil seqbase) and the TAIL column carries the data. All functions that deal with data work on the TAIL column.

  • BATkey now works on the TAIL column instead of the HEAD column.

  • Replaced BATseqbase with BAThseqbase and BATtseqbase, the former for setting the seqbase on the HEAD, the latter for setting the seqbase on the TAIL.

  • Replaced function BATnew with COLnew with slightly different arguments: the first argument of COLnew is the SEQBASE of the head column (which is always VOID).

  • The "wrd" type has been removed from GDK and MAL. The type was defined to be a 32 bit integer on 32 bit architectures and a 64 bit integer on 64 bit architectures. We now generally use "lng" (always 64 bits) where "wrd" was used.

SQL
  • Removed functions sys.zorder_encode, sys.zorder_decode_x, and sys.zorder_decode_y.

  • The experimental recycler code is moved to the attic.

  • The syntax of bat.new(:oid,:any) has been changed by dropping the superflous :oid. All BATs are now binary associations between a void column and a materialized value column. (except for the internal :bat[:void,:void] representation of simple oid ranged tails.)

Bug Fixes
  • 3357: Implement setQueryTimeout()
  • 3445: Add support for database name to dotmonetdb file
  • 3597: SQL to MAL listing looses types
  • 3973: JDBC hangs
  • 3976: Performance enhancement to LIKE without wildcards
  • 4005: Correlated update causes incorrect null constraint violation
  • 4016: merge table only optimises for point query
  • 4040: sys.storage call can take a long time
  • 4047: Segfault when updating a dropped table
  • 4050: Database corruption when running low on inode
  • 4057: missing bulk operations between constant and bat
  • 4061: SIGSEGV in candscan_lng
  • 4066: Deadlocked monetdbd
  • 6068: Error message about incompatible BBP version should be clearer
  • 6069: query with union all silently crashes
  • 6070: setting negative session query timeout should not be possible/allowed
  • 6071: where clause with cast and floor fails to sigsegv
  • 6072: Bind to UPD delta column does not get/show type information in EXPLAIN
  • 6073: Missing type information for constants in MAL explain
  • 6074: SET ROLE command does not work
  • 6075: gdk_calc.c:13113: BATcalcifthenelse_intern: Assertion `col2 != NULL' failed.
  • 6076: rel_optimizer.c:5426: rel_push_project_up: Assertion `e' failed.
  • 6077: mserver5: rel_optimizer.c:5444: rel_push_project_up: Assertion `e' failed.
  • 6078: rel_bin.c:2402: rel2bin_project: Assertion `0' failed.
  • 6084: Merge table point to wrong columns if columns in partition tables are deleted
  • 6108: monetdb5-sql sysv init script does not wait for shutdown
  • 6114: segfault raised in the query rewriter due to a null pointer
  • 6115: Assertion hit in the codegen
  • 6116: Codegen does not support certain kind of selects on scalar subqueries
  • 6117: Assertion hit in the query rewriting stage during the push down phase
  • 6118: SIGSEGV in strPut due to shared heap
  • 6119: Assertion hit in the MAL optimiser on a complex query
  • 6120: QUANTILE() treats NULL as if it is zero
  • 6121: SELECT a.col IN ( b.col FROM b ) FROM a statements with no error but no result
  • 6123: Assertion hit in the codegen #2
  • 6124: CASE <column> WHEN NULL THEN 0 ELSE 1 END returns wrong result
  • 6125: Stack overflow in the query rewriter with a query having an OR condition and a nested SELECT subexpression
  • 6126: batcalc.== can't handle void BATs
  • 6139: Debian libmonetdb13 conflicts with libmonetdb5-server-geom

June 2016

June 2016 panos Wed, 12/21/2016 - 12:37

Jun 2016-SP2 bugfix release

Build Environment
  • We now use the CommonCrypto library instead of the OpenSSL library on Darwin.

Bug Fixes
  • 3939: Assert failure on concurrent queries when querying sys.queue
  • 4019: Casting a timestamp from a string results in NULL
  • 4025: expressions in the WHERE clause that evaluates incorrectly
  • 4038: After upgrade from 11.21.19, jdbc couldn't list tables for non sys users
  • 4044: Server crash when trying to delete a table has been added to a merge table with "cascade" at the end
  • 4049: Wrong results for queries with "OR" and "LEFT JOIN"
  • 4052: Infinite loop in rel_select
  • 4054: copy into file wrongly exports functions
  • 4058: Server crashes with a particular conditional query
  • 4059: Geom functions only visible by user monetdb
  • 4060: BAT leak in some aggregate queries
  • 4062: Error: SELECT: no such binary operator 1800 790 9186'like(varchar,varchar)' when used in query running in other schema than sys
  • 4064: Assertion: column not found
  • 4067: Relevant column name not printed when a CSV parsing error occurs
  • 4070: Extra condition in join predicate of explicit join produces wrong MAL code
  • 4074: Cannot use prepared statements when caching disabled
  • 6065: CTE with row number and union fails within MAL

Jun 2016-SP1 bugfix release

.
  • Lots of memory leaks have been plugged across the whole system.

Java Module
  • Corrected PROCEDURE_TYPE output value of method DatabaseMetaData.getProcedures(). It used to return procedureReturnsResult. Now it returns procedureNoResult. Corrected ORDINAL_POSITION output value of method DatabaseMetaData.getProcedureColumns(). It used to start with 0, but as procedures do not return a result value it now starts with 1 for all the procedure arguments, as defined by the JDBC API.

  • Improved output of method DatabaseMetaData.getProcedures(). The REMARKS column now contains the procedure definition as stored in sys.functions.func. The SPECIFIC_NAME column now contains the procedure unique identifier as stored in sys.functions.id. This allows the caller to retrieve the specific overloaded procedure which has the same name, but different arguments. Also improved output of method DatabaseMetaData.getProcedureColumns(). The SPECIFIC_NAME column now contains the procedure unique identifier as stored in sys.functions.id. This allows the caller to retrieve the proper arguments of the specific overloaded procedure by matching the SPECIFIC_NAME value.

  • Improved output of method DatabaseMetaData.getFunctions(). The REMARKS column now contains the function definition as stored in sys.functions.func. The SPECIFIC_NAME column now contains the function unique identifier as stored in sys.functions.id. This allows the caller to retrieve the specific overloaded function which has the same name, but different arguments. Also improved output of method DatabaseMetaData.getFunctionColumns(). The SPECIFIC_NAME column now contains the function unique identifier as stored in sys.functions.id. This allows the caller to retrieve the proper arguments of the specific overloaded function by matching the SPECIFIC_NAME value.

Bug Fixes
  • 4014: KILL signal
  • 4021: Analyze query does not escape input [security]
  • 4026: JDBC driver incorrectly converts TINYINT fields to String instead of an integer type.
  • 4028: inputs not the same size
  • 4031: mclient doesn't accept - argument to refer to stdin
  • 4032: no decimal places after update. ODBC driver
  • 4035: SQL Function call bug
  • 4036: Possible sql_catalog corruption due to unclean backuped tail

Jun 2016 feature release

MonetDB5 Server
  • Extended the storage() table producing function to also accept storage([schemaname [, tablename [, columnname]]])

  • The :bat[:oid,:any] type descriptor has been turned into its columnar version :bat[:any]

  • Converted the *.mal scripts into *.malC versions to prepare for removal of the mserver console.

  • Removed functions mat.hasMoreElements, mat.info, mat.mergepack, mat. newIterator, mat.project, mat.pack2, mat.sortReverse, mat.sort, and mat.slice.

  • Removed grouped aggregate functions from the aggr module in which the groups were indicated by the head column of the bat to be aggregated. Use the interface with a separate group bat instead.

  • The server now stops executing a query when the client disconnects.

  • Removed algebra.join. Use algebra.subjoin instead.

  • Removed algebra.antijoin. Use algebra.subantijoin or algebra.subthetajoin instead.

  • The MAL function 'leftfetchjoin' is renamed to its relational algebra version 'projection'

  • The generic property handling scheme has been removed. It was used in just a few places, for widely different purposes and contained cruft.

  • Removed functions str.iconv and str.codeset. Internally, strings are always in UTF-8 encoding, so we can't allow code set conversions.

  • Removed algebra.like with a BAT argument. Use algebra.likesubselect instead.

  • Removed algebra.leftjoin. Use algebra.subleftjoin or algebra.leftfetchjoin instead.

  • Removed algebra.tdiff and algebra.tinter.

  • Removed algebra.sample. Use sample.subuniform instead.

  • Removed algebra.select. Use algebra.subselect instead.

  • Removed algebra.revert.

  • Removed bat.order and bat.orderReverse functions.

  • Changed client.getUsers to return two bats, one with the user id and one with the user name.

  • Implemented algebra.subdiff and algebra.subinter.

  • Removed algebra.tdifference and algebra.tintersect.

  • Removed algebra.tunion.

Build Environment
  • A new package MonetDB-lidar (Fedora) or libmonetdb5-server-lidar (Debian/Ubuntu) has been created to work with LiDAR data.

  • Implemented a systemd configuration file for a monetdbd.service on systems that support it (Fedora, newer Ubuntu).

Client Package
Geom Module
  • The geom module has been completely overhauled. Types are now specified as GEOMETRY(POINT) instead of POINT, old functions have been removed, new ones introduced. However, even with all the changes, a database upgrade should still be possible (as always, make a backup first).

MonetDB Common
  • Removed BATconst. Use BATconstant instead.

  • Changed BATconstant. It now has a new first argument with the seqbase for the head column.

  • Removed BATmmap. It was no longer used.

  • BUNdelete and BATdel don't accept a foce argument and only allow deleting values that have not yet been committed. BUNdelete exchanges the deleted value with the last value (if the deleted value isn't the last value). BATdel compacts the BAT by shifting values after the deleted values up. The list of to-be-deleted values in BATdel must be sorted and unique.

  • Removed BUNreplace from list of exported functions. It wasn't used, and there is still BUNinplace and void_inplace that do more-or-less the same thing.

  • Changed BATderiveHeadProps to BATderiveTailProps (and it now works on the tail column).

  • Removed unused functions BATalpha, BATdelta, and BATprev.

  • Removed function VIEWcombine. Use BATdense instead.

  • Removed "left" parameter from BUNinplace. It wasn't used since the BAT it works on is dense headed.

  • Created function BATdense to easily create a [void,void] BAT with specified seqbases for head and tail and a count.

  • Removed function BATmark. When all head columns are dense, BATmark basically only created a new [void,void] BAT.

  • Renamed BATsubsort to BATsort.

  • Removed "sub" from the names of the function BATsubselect, BATthetasubselect, BATsubcross, BATsubleftjoin, BATsubouterjoin, BATsubthetajoin, BATsubsemijoin, BATsubdiff, BATsubjoin, BATsubbandjoin, BATsubrangejoin, and BATsubunique.

  • Removed BATsubleftfetchjoin: it was not used.

  • Removed BATcross1. Use BATsubcross instead.

  • Removed all versions of the SORTloop macro.

  • Removed Batkdiff. Use BATsubdiff instead.

  • Removed BATselect. Use BATsubselect instead.

  • Removed BATsemijoin. Use BATsubsemijoin instead.

  • Removed BATjoin. Use BATsubjoin instead.

  • Removed BATleftjoin. Use BATsubleftjoin or BATproject instead.

  • Removed BATleftfetchjoin. Use BATproject instead.

  • Removed BUNins from the list of exported functions.

  • Removed legacy functions BATuselect and BATuselect_.

  • Removed legacy functions BATantijoin, BATbandjoin, BATouterjoin, BATrangejoin, and BATthetajoin.

  • Removed function BATrevert.

  • BATordered now works on the TAIL column.

  • Removed obsolete functions BATorder() and BATorder_rev().

  • Implemented BATsubdiff which returns a list of OIDs (sorted, i.e. usable as candidate list) of tuples in the left input whose value does not occur in the right input.

  • Removed function BATkintersect. It wasn't used anymore. It's functionality can be obtained by using BATsubsemijoin.

  • Removed the function BATkunion.

Java Module
  • Fixed problem in DatabaseMetaData.getUDTs() when it was called with types parameter filled. It used to throw SQException with message: SELECT: identifier 'DATA_TYPE' unknown. Now it returns the UDTs which match the provided array of data types.

  • Implemented MonetDatabaseMetaData.supportsConvert() and MonetDatabaseMetaData.supportsConvert(int fromType, int toType) methods. It used to always return false. Now it returns true for the supported conversions. This fixes Bug 3460.

  • ResultSet.setFetchSize(): added a dummy implementation to get rid of the SQLFeatureNotSupportedException. In MonetDB, it does not make sense to set the fetch size of a result set. If one really wants to set the fetch size, one should use Statement.setFetchSize() instead.

  • Fixed resource leak in ResultSetMetaData. It created and cached a ResultSet object for each column but never closed the ResultSet objects.

  • Corrected DatabaseMetaData methods which accept a catalog filter argument. Those methods will now filter the results on the specified catalog name, whereas previously the catalog filter argument was ignored.

  • Corrected output of column KEY_SEQ of DatabaseMetaData methods: getPrimaryKeys(), getImportedKeys(), getExportedKeys() and getCrossReference(). It now starts at 1 instead of 0 previously.

  • Corrected DatabaseMetaData.getSchemas() by returning 2 instead of 3 columns.

  • Improved DatabaseMetaData.getColumns() by returning two additional columns: IS_AUTOINCREMENT and IS_GENERATEDCOLUMN.

  • Improved DatabaseMetaData.getTypeInfo(). It now returns better information on LITERAL_PREFIX, LITERAL_SUFFIX, CREATE_PARAMS, CASE_SENSITIVE, FIXED_PREC_SCALE and MAXIMUM_SCALE for some data types. Also the returned rows are now ordered by DATA_TYPE, TYPE_NAME, PRECISION as required by the specs. Also corrected output column names "searchable" into "SEARCHABLE" and "MAXIMUM SCALE" into "MAXIMUM_SCALE" to match the JDBC specification.

  • Corrected DatabaseMetaData.getPseudoColumns(). It used to return 12 empty rows. Now it returns no rows as MonetDB does not have pseudo columns.

  • Implemented method DatabaseMetaData.getClientProperties(). It used to always return a resultset with 4 completely empty rows. It now returns a resultset with the possible connection properties.

  • Implemented method DatabaseMetaData.getUDTs(). It used to return an empty resultset. Now it returns the User Defined Types such as inet, json, url and uuid.

  • Corrected the returned table types in DatabaseMetaData.getTableTypes(). It now returns all 10 table types (as stored in sys.table_types) instead of the previously 8 hardcoded table types. For old MonetDB servers which do not have the sys.table_types table, the old behavior is retained.

  • Implemented methods DatabaseMetadata.getProcedures() and DatabaseMetadata.getProcedureColumns(). They used to return an empty resultset. Now they return the expected Procedures and ProcedureColumns. Also getProcedureColumns() now returns a resultset with all 20 columns instead of 13 columns previously.

  • Method getFunctionColumns() in DatabaseMetadata used to throw an SQLException: getFunctionColumns(String, String, String, String) is not implemented. This method is now implemented and returns a resultset.

  • Method getFunctions() in DatabaseMetadata used to throw an SQLException: SELECT: no such column 'functions.sql' This has been corrected. It now returns a resultset as requested.

  • The resultsets of DatabaseMetadata methods now no longer return a value for the *_CAT columns as MonetDB does not support Catalogs.

  • Fixed a memory leak in MonetDatabaseMetaData.java for a static cache which kept references to closed Connection objects.

  • Improved JdbcClient program when presenting query data to console. It used to send an SQL catalog query for each query result column which slowed down the interactive response considerably. These additional SQL catalog queries have been eliminated.

  • Corrected MonetResultSet.getObject(String columnName). It no longer throws a NullPointerException in cases where internally a MonetVirtualResultSet is used.

  • Fixed Connection.isValid(): this method should never attempt to close the connection, even if an error has occurred.

SQL
  • Disallow GRANT <some-user> TO <role-or-use>. Only explicitly created roles can be granted.

  • Extended grantable schema privileges: when a user is granted the "sysadmin" role, the user now hcan not only create schemas, but also drop schemas.

  • Introduced COPY INTO/ COPY FROM global privileges. These privileges allows other users than 'monetdb' to be granted the privileges to use the MonetDB bulk data loading/exporting features, i.e., COPY FROM <file> and COPY INTO <file>.

Bug Fixes
  • 2407: Merovingian: allow binds to given ip/interface
  • 2815: No SRID support
  • 3361: constants as MAL function parameters prevent intermediate reuse
  • 3374: UNIQUE constraint does not set tkey property on the corresponding BAT
  • 3460: incomplete implementation of JDBC driver supportsConvert(), supportsConvert(int fromType, int toType) methods in MonetDatabaseMetaData.java
  • 3711: JDBC connection using jdbcclient.jar is very slow
  • 3877: MonetDBLite should allow close then re-open databases? without restarting R
  • 3911: Invalid connect() call in 'redirect' mode
  • 3920: query with DISTINCT and correlated scalar subquery in SELECT list causes Assertion failure and crash of mserver5
  • 3927: COUNT( distinct my_clob ) , QUANTILE( my_double , number ) fails in dev build
  • 3956: MonetDBLite unable to execute LIMIT 1 statement
  • 3972: Drastic Memory leak of 600GBs while generating plan for Query with 25 joins
  • 3974: Prepared statement rel_bin.c:2378: rel2bin_project: Assertion `0' failed.
  • 3975: Suspicious code in store_manager() on exit path
  • 3978: SQL returns TypeException 'aggr.subcorr' undefined for sys.corr function
  • 3980: JOIN with references on both sides crashes mserver
  • 3981: Incorrect LEFT JOIN when FROM clause contains nested subqueries
  • 3983: Creation of a Foreign Key which partially maps to a primary key is accepted without a warning
  • 3984: Multiple paths in the .profile
  • 3985: ruby-monetdb-sql gem fails for negative timezone offset (USA, etc.)
  • 3987: Segfault on malformed csv import
  • 3991: MonetDBLite feature request: default monetdb.sequential to FALSE on windows
  • 3994: MonetDBLite dbDisconnect with shutdown=TRUE freezes the console on windows
  • 3995: NullPointerException when calling getObject()
  • 3997: calling scalar functions sys.isaUUID(str) or sys.isaUUID(uuid) fail
  • 3999: length() returns wrong length for strings which have spaces at the end of the string
  • 4010: RELEASE SAVEPOINT after ALTER TABLE crashes mserver5
  • 4011: sys.sessions.user column always shows 'monetdb'
  • 4013: GDKextendf does not free up memory when it fails due to insufficient resources
  • 4015: Daemon crashes on database release command

July 2015

July 2015 sjoerd Thu, 06/23/2016 - 09:49

Jul 2015-SP4 bugfix release

MonetDB Common
  • Fixed a bug that caused various instances where old data returned or where crashes occurred. The problem was that internally data wasn't always marked dirty when it was being changed, causing later processing to not deal with the changed data correctly.

Bug Fixes
  • 3905: MonetDB doesn't handle ANY/SOME/ALL operator correctly
  • 3929: R aggregate not recognized when using 3 or more parameters
  • 3965: Not possible to quote/escape single quote character in the name of the file to load.
  • 3968: Missing double use of column names

Jul 2015-SP3 bugfix release

MonetDB5 Server
  • Fixed potential crash in MAL debugger when accessing BATs by index. Functionality dropped as it is also a security leak.

Bug Fixes
  • 2972: SQL URL functionality contains errors
  • 3549: bulk string operations very slow
  • 3881: Server crashes on bulk load
  • 3890: Window function + group by in subselect, rel2bin_project: Assertion `0' failed
  • 3891: MonetDB crashes when executing SQL with window function
  • 3900: null handling in some sql statements is incorrect
  • 3906: Multi-column 1-N table-function with mitosis produces different column counts
  • 3908: LEFT JOIN with OR conditions triggers assertion
  • 3909: Incorrect column name in OR condition of LEFT JOIN crashes mserver
  • 3910: COPY INTO table (column1, column2) got wrong result
  • 3912: When table/column names conflicts, data ends in multiple tables!
  • 3917: Date difference returns month_interval instead of day_interval
  • 3918: MonetDB.R version 1.0.1 incorrectly constructs the batfile script
  • 3919: Table conflict when the table name and fields are identical
  • 3921: Creating a table from a complex query crashes mserver or triggers assertion
  • 3922: AVG( column ) returns NaN rather than Inf when column contains Inf
  • 3928: When killing a virtual machine, sql_logs/sql/log is empty
  • 3930: Wrong typecast on character columns in prepared statements when using Umlaute
  • 3932: CASE expressions with constants are not evaluated correctly
  • 3933: replace "exit" by "throw new Exception"
  • 3937: bad BAT properties with binary copy into and NULL values
  • 3938: Wrong error message on violating foreign key constraint
  • 3940: Date calculation and comparison produce wrong result
  • 3941: Wrong coercion priority
  • 3948: SQL: select * from sys.sys.table_name; is accepted but should return an error
  • 3951: extern table_funcs not visible from Windows DLL for extensions like vaults (crashes)
  • 3952: Stream table gives segfault
  • 3953: MIN/MAX of a UUID column produces wrong results
  • 3954: Consolidate table assertion error
  • 3955: (incorrect) MAL loop instead of manifold triggered by simple change in target list

Jul 2015-SP2 bugfix release

Bug Fixes
  • 2014: 'null' from copy into gets wrong
  • 3817: opt_pushselect stuck with multi-table UDF
  • 3835: windows does not release ram after operations
  • 3836: rand() only gets evaluated once when used as an expression
  • 3838: Update column with or without parenthesis produce different results
  • 3840: savepoints may crash the database
  • 3841: mclient fails with response "Challenge string is not valid"
  • 3842: SQL execution fails to finish and reports bogus error messages
  • 3845: Too many VALUES in INSERT freeze mserver5
  • 3847: Wrong SQL results for a certain combination of GROUP BY / ORDER BY / LIMIT
  • 3848: mserver segfault during bulk loading/updating
  • 3849: HUGEINT incorrect value
  • 3850: DEL character not escaped
  • 3851: expression that should evaluate to FALSE evaluates to TRUE in SELECT query
  • 3852: CASE statement produces GDK error on multithreaded database: BATproject does not match always
  • 3854: Complex expression with comparison evaluates incorrectly in WHERE clause
  • 3855: Foreign key referencing table in a different schema - not allowed.
  • 3857: Large LIMIT in SELECT may abort the query
  • 3861: Using window functions cause a crash
  • 3864: Error in bulk import for chinese character
  • 3871: NOT x LIKE triggers "too many nested operators"
  • 3872: mserver crashes under specific combination of JOIN and WHERE conditions
  • 3873: mserver5: gdk_bat.c:1015: setcolprops: Assertion `x != ((void *)0) || col->type == 0' failed.
  • 3879: Database crashes when querying with several UNION ALLs.
  • 3887: Querying "sys"."tracelog" causes assertion violation and crash of mserver5 process
  • 3889: read only does not protect empty tables
  • 3895: read only does not protect this table

Jul 2015-SP1 bugfix release

Client Package
  • In the SQL formatter of mclient (the default) we now properly align East Asian wide characters.

Bug Fixes
  • 3789: Query on large string table fails on HEAPextend
  • 3794: table function sys.rejects() and view sys.rejects() are listed are metadata objects but give an (incorrect) error when they are queried
  • 3797: COPY INTO with incorrect number columns
  • 3798: SELECT query with INTERSECT causes assertion failure
  • 3800: LIKE is broken for many patterns
  • 3802: Disk space never freed: a logical ref is keeped on a deleted BATs
  • 3803: SQL query parser fails to parse valid SELECT query with a CASE .. END in it. It fails with parser error: identifier 'x' ambiguous
  • 3804: `monetdb status` command crashes under certain conditions
  • 3809: Inefficient plan is generated for queries with many (>= 24) joined tables which take a long time or an HEAPalloc error. I get Error: GDK reported error. HEAPalloc: Insufficie
  • 3810: Fix statistics gathering
  • 3811: NOT LIKE not working if the operand doesn't contains wildcards.
  • 3813: COPY INTO fails on perfectly clean CSV file
  • 3814: Server crash when using bitwise NOT operation in SQL query
  • 3816: Server crashes when trying to convert timestamp to str with incorrect format
  • 3818: Crash when performing UNION/GROUP BY over tables with different columns
  • 3819: order of tables in FROM-clause has negative impact on generated plan (using crossproducts instead of joins)
  • 3820: mclient accepts table with repeated constraint which causes crash on insert
  • 3821: Unexpected error when using a number instead of a boolean
  • 3822: Yet another LIKE operator issue
  • 3823: JDBC Connection to a schema - setSchema() error
  • 3825: MonetDB not cleaning intermediate results which leads to filling up disk space and ultimately server crash
  • 3827: Certains comparisons between UUID produce a MAL error
  • 3828: Schema corruption after several ALTER TABLE statements and server restart
  • 3829: Certains simple WHERE clause cause MonetDB to segfault without explanation
  • 3830: Coalesce typing inconsistencies
  • 3833: NULL literals refused at many places
  • 3834: Date comparison returns incorrect results
  • 3839: msqldump generates incorrect syntax ON UPDATE (null)

Jul 2015 feature release

Do not skip upgrades! If you are running a release older than the Oct2014 series (11.19.X), do not skip upgrading to the Oct2014 release before upgrading to the Jul2015 release.  In fact, you should first upgrade to the Oct2014-SP2 (11.19.9).  Don't forget to start the server and to connect a client to the server for each of your databases.

One of the new features in this release is the possibility to use "best effort" when importing CSV files into the server.  However, we have found a bug that causes this feature to misbehave when the CSV file cannot be read correctly (exactly when one might want to use best effort).  Hence we recommend to not use the feature.  See bug 3797.

MonetDB5 Server
  • Implemented batcalc.min and batcalc.max. Made calc.min and calc.max generic so that no other implementations are needed.

  • Removed function batcalc.ifthen.

Build Environment
  • Upgraded the license to the Mozilla Public License Version 2.0.

Client Package
  • Added a new output format to mclient: --format=expanded (or -fx). In this format, column values are shown in full and below each other.

MonetDB Common
  • BUNtvar and BUNhvar macros no longer work for TYPE_void columns.

  • Changed interfaces of a lot of GDK-level functions. When they modify a BAT, don't return the same BAT or NULL, but instead return GDK_SUCCEED or GDK_FAIL.

  • Changed a bunch of hash-related functions to work on the tail column. The functions that have been changed to work on the tail column are: BAThash, BATprepareHash, HASHgonebad, HASHins, and HASHremove.

Java Module
  • Improved JDBC driver to not throw NullPointerException anymore when calling isNullable() or getPrecision() or getScale() or getColumnDisplaySize() or getSchemaName() or getTableName() or getColumnClassName() on a ResultSetMetaData object.

  • We now compile the Java classes using the latest Java 1.8 version, and we tell it to compile for Java 1.7.

SQL
  • Differences between time, timestamp, and date values now return properly typed interval types (second or month intervals) instead of integers.

  • Added support for 128-bit integers (called HUGEINT) on platforms that support this.

  • Removed support for the mseed library.

  • Removed support for RDF.

  • Removed DataCell. It was experimental code that was never enabled.

Bug Fixes
  • 2618: Implement master slave scheme
  • 3361: constants as MAL function parameters prevent intermediate reuse
  • 3364: Cannot set role back to a user's default role
  • 3365: Unable to grant object privileges while having a non-default current_role
  • 3440: Sequence type errors
  • 3449: mserver crash on start - Freebsd 10 amd64
  • 3476: Cannot revoke object access
  • 3496: autocompletion table names does not work correctly
  • 3556: when 2 multiplexed functions in MAL plan, only one is mapped correctly to bat<mod>.function primitive
  • 3564: Request: add support for postgresql specific scalar function: split_part(string text, delimiter text, field int)
  • 3625: SIGSEGV because mat array can overrun in opt_mergetable.c
  • 3627: SQRT in CASE does not work as of Oct2014
  • 3654: configure --enable-fits requires extra commands after creating a database instance
  • 3673: mclient 'expanded' row formatter
  • 3674: Obfuscate event tracing
  • 3679: No error is given when incorrect timezone value is specified for a timetz column
  • 3686: Wrong associativity of multiply/divide
  • 3702: Filter function not found if created in a user schema
  • 3708: wrong scoping for cross-schema view references
  • 3716: alter table my_merge_table drop table t1; crashes mserver5 with Segmentation fault
  • 3724: Wrong size calculation in BATsubjoin
  • 3730: SAMPLE function not sampling randomly
  • 3732: memory leak (of InstrRecord) in opt_mergetable
  • 3733: "(TRUE OR <Exp>) AND <Exp>" is evaluated incorrectly
  • 3735: python connection with unix_socket
  • 3736: crash if mclient disconnects abruptly during a query
  • 3738: Database inconsistency when using savepoint
  • 3739: CASE statements do not handle NULLs in the IN () operator properly
  • 3740: select epoch(now()); types timestamptz(7,0) and bigint(64,0) are not equal
  • 3742: Division By Zero
  • 3744: cast to int gives different results for decimal than double
  • 3747: joins fail in the presence of nulls
  • 3748: Missing META-INF/services/java.sql.Driver in JDBC package
  • 3753: Hang on json field parsing
  • 3754: select from a REMOTE TABLE referring local table crashes mserver5
  • 3756: column type conversion sticks to subsequent queries
  • 3758: "COPY INTO ..." doesn't work, if executing from 2 processes concurrently.
  • 3759: select data from "sys"."rejects" returns unexpected error and when next select data from "sys"."sessions" causes an assertion failure in mal_interpreter.c:646.
  • 3760: SQL parser has problem with (position of) a scalar subquery in a SELECT-list
  • 3761: SQL executor has problem with (position of) a subquery in a SELECT-list. Inconsistent behavior.
  • 3763: JDBC PreparedStatement for a table with 14 Foreign Keys crashing the Database
  • 3764: DROPping multiple users causes a crash
  • 3765: Re-granting a revoked privilege does not work
  • 3766: VIEW not visible if created under a different schema
  • 3767: CREATE TEMP TABLE using "LIKE" incorrectly handled
  • 3769: SIGSEGV when combining a cast/column alias with a UNION ALL view
  • 3770: combined conditions on declared table in User Defined Function definition crashes monetdb
  • 3771: Owner of the schema loses rights if assumes the monetdb role.
  • 3772: Any user can grant a role.
  • 3773: quantile(col, 0) and quantile(col, 1) fail
  • 3774: mclient is unaware of merge tables and remote tables
  • 3775: COPY INTO: Backslash preceding field separator kills import
  • 3778: Crash on remote table schema mismatch
  • 3779: server crashes on MAX() on SELECT DISTINCT something combo
  • 3783: Behavioural change in Jul2015 for 'timestamp minus timestamp'
  • 3784: Assertion failed: (bn->batCapacity >= cnt), function BAT_scanselect, file gdk_select.c, line 1008.
  • 3785: sum(interval) causes overflow in conversion to bte
  • 3786: ResultSet.close() never sends Xclose to free resources
  • 3787: "b and g must be aligned" from complex group/union query
  • 3791: HEAPextend: failed to extend to 2420077101056

October 2014

October 2014 sjoerd Fri, 08/28/2015 - 13:11

Important consideration when upgrading to this release. In this release we have fixed bug 3474. This bug made it possible that there were duplicates despite there being a unique or primary key constraint present. This could only happen for multi-column constraints. MonetDB maintains an internal column containing a hash for multi-column constraints, and there was a bug in the maintenance code of this column. During an upgrade, this column needs to be recreated. This is done by dropping and adding back the constraints. If there are no duplicates, adding back the constrains should work without problem, but if there are duplicates, adding the constraints will fail. There is unfortunately no easy way to tell the user, so after having made the first connection to the server (when the upgrade happens), check the merovingian log (if you use merovingian -- also known as monetdbd) or the server output window (if you don't use merovingian).

Jaql support is removed. The experimental code for Jaql support has been removed. If you have RPM or DEB packages for Jaql installed from an earlier version (MonetDB-jaql and monetdb5_jaql respectively), you will need to uninstall them before the upgrade.

Upgrade code fixed. The code to upgrade from previous releases still had some bugs. These have now been fixed. Upgrades directly from the Feb2013 release should work again. Also, upgrading to this version should fix some problems that were caused by the upgrade to the Oct2014 release.

Oct 2014-SP4 bugfix release

Due to a code conflict it is not possible to upgrade to this bug fix release from a release older than the Oct2014 release.  If you are running an older release, first upgrade to the Oct2014 release, and then to Oct2014-SP4.

Bug Fixes
  • 3705: Assertion failure in rel_bin.c:2274: rel2bin_project: Assertion `0' failed.
  • 3707: var() possibly not working in debug builds
  • 3712: Concurrency issue on querying the SQL catalog
  • 3713: Long startup cost for simple session
  • 3715: Crash with two ALTER TABLE statements in a transaction
  • 3718: Adding and dropping a non existing tablename to/from a merge table is accepted without an error
  • 3719: Assertion failure in /MonetDB-11.19.11/gdk/gdk_bat.c:2841: BATassertHeadProps: Assertion `!b->H->key || cmp != 0' failed.
  • 3720: Incorrect results on joining with same table
  • 3723: Assertion failure in rel_bin.c:2548: rel2bin_groupby: Assertion `0' failed.
  • 3725: LEFT JOIN bug with CONST value
  • 3731: left shift for IP addresses not available to non-system users

Oct 2014-SP3 bugfix release

Build Environment
  • We now also create debug packages for Debian and Ubuntu.
MonetDB Common
  • Replaced the rangejoin implementation with one that uses imprints if it can.
Bug Fixes
  • 3466: UPDATE statements fails with "GDKerror: MT_mremap() failed"
  • 3602: Surprising overload resolution of generate_series
  • 3613: SQL data dictionary contains columns names which are also special keywords. This causes unexpected/unneeded SQL query errors
  • 3645: Network address operators such as << and <<= do not work
  • 3647: missing BAT for a column leads to crash in gtr_update_delta
  • 3648: memory corruption on unclean connection shutdown with local temporary tables
  • 3650: Naming of persistent BATs is fragile
  • 3653: PREPARE crashes mserver if unbound variable is function parameter
  • 3655: SQL WHERE -1 in (-1) issue?
  • 3656: error message after calling fitsload()
  • 3660: Incorrect Results for Comparison Operators on inet Datatype
  • 3661: Ship debug symbols for pre-built binaries
  • 3662: UPDATE row with row value constructor crashes monetdb server
  • 3663: Incorrect result ROW_NUMBER in subquery
  • 3664: SQLstatementIntern missing parameter when using jsonstore
  • 3665: inter-session starvation issue, particularly affects sys.queue
  • 3666: casting text column to inet truncating text column and resulting inet for first occurrence only
  • 3667: insert of negative value for oid column aborts mserver5 process with assertion failure
  • 3669: ALTER TABLE <tbl_nm> ADD CONSTRAINT <tbl_uc1> UNIQUE (col1, col2, col3) causes Assertion failure and abort
  • 3671: ODBC-Access on Windows 2012 does not work - E_FAIL
  • 3672: libbat_la-gdk_utils.o: relocation R_X86_64_PC32 against `MT_global_exit' can not be used when making a shared object
  • 3676: merovingian hangs trying to exit
  • 3677: Crash in BATgroup_internal (caused by 87379087770d?)
  • 3678: Ruby driver installation ignores prefix
  • 3680: Prepared statements fail on execution with message 'Symbol type not found'
  • 3684: Wrong query result set WHERE "IS NULL" or "NOT IN" clauses uses in combination with ORDER, LIMIT and OFFSET
  • 3687: 'bat.insert' undefined
  • 3688: Crash at exit (overrun THRerrorcount?)
  • 3689: No more connections accepted if a single client misbehaves
  • 3690: find_fk: Assertion `t && i' failed.
  • 3691: conversion of whitespaces string to double or float is accepted without an error during insert
  • 3693: algebra.join undefined (caused by non-existing variables in the plan)
  • 3696: Inconsistent behavior between dbl (SQL double) and flt (SQL real) data types and across platforms
  • 3697: mserver5[26946]: segfault at 0 ip 00007f3d0e1ab808 sp 00007f3cefbfcad0 error 4 in lib_sql.so[7f3d0e180000+16c000]
  • 3699: segfault again! (during last week I found 3 segfault bugs already)
  • 3703: INSERT INTO a MERGE TABLE crashes mserver5
  • 3704: Unknown identifier from subquery
  • 3705: Assertion failure in rel_bin.c:2274: rel2bin_project: Assertion `0' failed.
  • 3706: Assertion failure in gdk_bat.c: BATassertHeadProps: Assertion `!b->H->sorted || cmp <= 0' failed.
  • 3709: "BATproject: does not match always" on abusive use of ALTER TABLE SET READ ONLY

Oct 2014-SP2 bugfix release

Client Package
  • Changes to the Perl interface, thanks to Stefan O'Rear:
    • removes "use sigtrap", because this has global effects and should not be used by modules, only by the application.
    • allows Perl 5.8.1+ Unicode strings to be passed to quote() and included in statements (UTF-8 encoded, as expected by Monet's str module)
    • quote and unquote now use the same quoting rules as the MonetDB server, allowing for all characters except NUL to be round-tripped
    • several character loops have been reimplemented in regex for much greater performance
    • micro-optimizations to the result fetch loop
    • block boundaries are preserved in piggyback data so that Xclose is not appended or prepended to a SQL command
    • diagnostic messages #foo before a result header are ignored, this is necessary to use recycler_pipe
    • fail quickly and loudly if we receive a continuation prompt (or any other response that starts with a non-ASCII character)
    • header lines must start with %, not merely contain %, fixing a bug when querying a table where string values contain %
    • after closing a large resultset, account for the fact that a reply will come and do not lose sync
    • allow a MAPI_TRACE environment variable to dump wire protocol frames to standard output
    • fixes maximum MAPI block size to match the server limit of 16k. previously would crash on blocks larger than 16k
SQL
  • Fixed a typo in a column name of the sys.tablestoragemodel view (auxillary changed to auxiliary).
Bug Fixes
  • 3467: Field aliases with '#' character excise field names in result set.
  • 3605: relational query without result
  • 3619: Missing dll on MonetDB Start
  • 3622: Type resolution error
  • 3624: insert of incomplete or invalid ip address values in inet column is silently accepted but the values are not stored (they become/show nil)
  • 3626: casting a type without alias results in program contains errors
  • 3628: mclient and ODBC driver report 'type mismatch' when stddev_pop used in a select which returns 0 rows
  • 3629: IF THEN ELSEIF always evaluates the first test as true
  • 3630: segv on rel_order_by_column_exp
  • 3632: running make clean twice gives an error in clients/ruby/adapter
  • 3633: Wrong result for HAVING with floating-point constant
  • 3640: Missing implementation of scalar function: sql_sub(<date>, <month interval>)
  • 3641: SQL lexer fails to detect string end if it the last character is U+FEFF ZERO WIDTH NO-BREAK SPACE
  • 3642: Combined WHERE conditions less-than plus equals-to produce incorrect results
  • 3643: Missing implementations of scalar function: sql_sub(<timetz>, arg2)
  • 3644: COPY INTO fails to import "inet" data type when value has prefix length in CIDR notation
  • 3646: ORDER BY clause does not produce proper results on 'inet' datatype
  • 3649: recycler crashes with concurrent transactions

Oct 2014 SP1 bugfix release

MonetDB5 Server
  • Fixed adding of 0 intervals to dates.
Build Environment
  • The libraries included in the Windows installers have been upgraded. We now use libxml2-2.9.2, openssl-1.0.1j, pcre-8.36, and zlib-1.2.8.
MonetDB Common
  • Implemented a change to the way in which string bats are appended. We now try harder to limit the growth of the string heap.
  • Fixed some problems with BATsample. It was possible for BATsample to return a value that was just beyond the end of the sampled BAT. Also, on some systems the range of the rand() function is rather limited (0..32767) and trying to get a sample larger than this range would result in an infinite loop.
SQL
  • Fixed sys.queue() implementation to report on other queries being executed.
  • A number of bugs were fixed in the code to upgrade a database from previous releases. This version should fix the upgrade of a database that had been upgraded to the Oct2014 release, but also properly upgrade directly from Jan2014 and Feb2013 releases.
Bug Fixes
  • 3580: cosmetic change (append newline)
  • 3609: Incorrect use of generate_series
  • 3611: quantile() and median() commands crash when used 2x on the same variable on a null table
  • 3612: assertion failure when deleting rows from table to which a FK constraint is defined
  • 3620: ORDER BY broken when using UNION ALL
  • 3621: Hexadecimal literal vs decimal literal

Known issue when upgrading old versions. We have noticed that upgrading directly from a version older than the Jan2014 release gave an error. Upgrading to Jan2014 first and then on to Oct2014 should work.

Backup. As always, make a backup of your databases before upgrading.

Oct 2014 feature release

MonetDB5 Server
  • Removed algebra.materialize.
  • Removed algebra.kunique and algebra.tunique. They were subsumed by algebra.subunique.
  • Remove algebra.antiuselect and algebra.thetauselect. They were subsumed by algebra.subselect.
  • Removed algebra.topN and its imlementation BATtopN. The function was not used.
  • Removed aggr.histogram and its implementation BAThistogram. The function was not used, and did not produce output in the "headless" (i.e. dense-headed) format. Histograms can be created as a by-product of group.subgroup.
Build Environment
  • Jacqueline, the MonetDB/JAQL frontend, has been removed. The frontend never grew beyond being experimental, and there is no interest anymore to maintain the code.
Merovingian
  • monetdb create: add -p flag to set monetdb user password on creation, and therefore allow creating the database in unlocked state
mapilib
  • Changed mapi_timeout argument from seconds to milliseconds.
stream
  • Changed mnstr_settimeout function so that the specified timeout is now in milliseconds (used to be seconds), and that it also needs an extra argument specifying a callback function (no arguments, int result) that should return TRUE if the timeout should cause the function to abort or continue what it was doing.
MonetDB Common
  • Added "multifarm" capability. It is now possible to separate persistent and transient BATs into different directories (presumably on different disks). This can be done by using the --dbextra option of mserver5 (see the man page).
SQL
  • Added PostgreSQL compatible string TRIM, LTRIM, RTRIM, LPAD and RPAD functions
  • Stop support for upgrading directly from a database created with a server from the Oct2012 release or older. You can upgrade via the Feb2013 or Jan2014 release.
Bug Fixes
  • 2945: evaluation of SQL "between SYMMETRIC" requires MAL iterator because there is no (bulk) MIN/MAX(a,b)
  • 3204: monetdb create: allow setting admin password during creation
  • 3390: Missing definition for pushSht in monetdb5/mal/mal_builder.h
  • 3402: We should have a C implementation of mal.multiplex.
  • 3422: Segmentation fault after large table insert
  • 3459: incomplete implementation of JDBC driver getNumericFunctions(), getStringFunctions(), getSystemFunctions(), getTimeDateFunctions(), getSQLKeywords() methods in MonetDatabaseM
  • 3471: JDBC driver: incorrect output result of SQL query: SELECT 1 ;
  • 3474: bulk and scalar versions of mkey.rotate_xor_hash differ
  • 3484: COPY INTO on a file works fine on Linux/OSX, but not on Windows
  • 3488: Slow SQL execution for correlated scalar subquery
  • 3489: SQL query with ORDER BY does not order its result as requested
  • 3490: SQL query kills the mserver5 (Segmentation fault)
  • 3491: SQL query kills the mserver5 (Segmentation fault)
  • 3493: Test monetdb5/modules/mal/Tests/pqueue.mal fails since recent checkins
  • 3494: Tests monetdb5/modules/mal/Tests/pqueue[23].mal lack correct/expected/intended output
  • 3495: Test sql/test/centipede/Tests/olap.sql lacks correct/expected/intended output
  • 3497: monetdb start reports crash in merovingian.log
  • 3498: SQL throws TypeException if aggregations and limit statements are both present
  • 3502: Database was killed by signal SIGBUS
  • 3504: COPY INTO does not allow OFFSET without specifying amount of records
  • 3505: expression with <boolean> = NOT <boolean> returns a syntax error but NOT <boolean> = <boolean> not
  • 3506: conversion to varchar terminates mserver
  • 3508: conversion of string '0 ' to type smallint or integer fails
  • 3510: timestamp + month interval generates bogus MAL?
  • 3511: When having multiple selections combined with aliases not all of them seem to be evalauted.
  • 3512: auto-conversion of string to `sht` type no longer works
  • 3513: COPY BINARY INTO fails on 6gb file; works fine on 3gb
  • 3516: inserting '0' into a column of datatype numeric fails
  • 3518: UNION with subqueries
  • 3521: large results of function exp() are not automatically returned as double
  • 3522: SQL catalog table sys.columns lists columns for table ids which do not exist in sys.tables
  • 3523: Window function over union gives no result
  • 3524: wrong error on missing aggregation column
  • 3527: select distinct - order by - limit 2 results in one single result
  • 3528: segfault at mal_session.c:521
  • 3532: several geom tests crash after manifold changes
  • 3534: missing table name with invalid column in join using (and problems after resolving it)
  • 3536: program contains error with join using integer and smallint
  • 3542: gdk/gdk_bat.c:2904: BATassertHeadProps: Assertion `!b->H->revsorted || cmp >= 0' failed.
  • 3543: invalid behavior and incorrect data results for SQL data type: numeric(4,4)
  • 3544: sys.reuse() corrupts data
  • 3546: Division by zero in CASE statement that should avoid it
  • 3547: Empty query when selecting a field from a view made of UNION ALL
  • 3551: Wrong ticks in TRACE
  • 3552: incorrect data results for "WHERE int_col <> 0"
  • 3554: Issue with subselect and ORDER BY
  • 3555: Order of evaluation inside CASE WHEN
  • 3558: numeric values (as strings) are incorrectly parsed/converted and invalid strings are accepted without error
  • 3560: Error "BATproject: does not match always" with subselect/groupby/having
  • 3562: mserver5: gdk_bat.c:2855: BATassertHeadProps: Assertion `!b->H->revsorted || cmp >= 0' failed.
  • 3563: incorrect results for scalar function locate(in_str, search_str, occurrence)
  • 3565: Wrong/confusing error message when trying to add a FK to a TEMP TABLE
  • 3572: Table names with escaped double quotes are rejected
  • 3573: alter table alter_not_null_test alter test set NOT NULL; is accepted when test contains null. This used to be restricted but isn't anymore
  • 3575: segmentation fault in mserver5 process
  • 3576: Dropping default value definitions from a table does not work as expected
  • 3577: SIGSEGV in BATins_kdiff
  • 3579: segmentation fault in mserver5 process
  • 3581: mserver5: rel_bin.c:2504: rel2bin_groupby: Assertion `0' failed.
  • 3582: mserver5: sql_mem.c:48: sql_ref_dec: Assertion `r->refcnt > 0' failed.
  • 3583: Possible buffer overflow in max(varchar)
  • 3585: Incorrect query terminates connection
  • 3586: mserver5: sql/storage/store.c:3610: sys_drop_func: Assertion `rid_func != oid_nil' failed.
  • 3592: SIGSEGV in MANIFOLDjob
  • 3593: delta_append_val: Assertion `!c || ((c)->S->count) == bat->ibase' failed.
  • 3594: gdk/gdk_bat.c:2855: BATassertHeadProps: Assertion `!b->H->revsorted || cmp >= 0' failed.
  • 3595: Race/heap corruption on thread exit
  • 3596: gdk_bat.c:2861: BATassertHeadProps: Assertion `!b->H->nonil || cmp != 0' failed.
  • 3597: SQL to MAL listing looses types
  • 3598: SQL bulk load should ignore leading/trailing spaces also with type decimal (as with integers & real/double)
  • 3599: Double-free of imprints
  • 3601: Trivial typo in debian/monetdb5-sql.init.d
  • 3603: "monetdb create -p" hangs monetdbd
  • 3604: Sys.queue ignored upon errors

January 2014

January 2014 sjoerd Fri, 10/31/2014 - 13:58

Jan 2014-SP3 bugfix release

Build Environment
  • Fix configure to continue without Python if the python binary is too old. This instead of always aborting configure if python happens to be too old.
Bug Fixes
  • 3487: dead link to "Professional services"
  • 3500: MonetDB driver wants an empty string for SQLTables and SQLColumns API calls, where other drivers expect NULL
  • 3514: mserver5 crash due (assertion failure in gdk_select.c)
  • 3515: mserver5 crash due (assertion failure in gdk_bat.c)
  • 3519: Uppercase TRUE/FALSE strings cannot be converted to boolean values

Jan 2014-SP2 bugfix release

Build Environment
  • Lots of minor fixes were made for potential defects found by Coverity Scan.
  • On Windows we now build the geom module against version 3.4.2 of the geos library.
Client Package
  • ODBC: Implemented {call procedure-name(...)} escape. The version {?=call ...} is not implemented.
Bug Fixes
  • 3424: numeric values at the front of strings determines whether CAST works successfully
  • 3439: Python driver drops milliseconds from timestamps
  • 3446: SET READ ONLY forgets previous changes
  • 3455: String columns unusable from 64-bit .NET via ODBC
  • 3456: Insert fails
  • 3457: When kernel of remote client crashes, the connection remains established on server side
  • 3458: mserver5 crash on SQL: SELECT COUNT(*) FROM SYS.TABLES HAVING COUNT(*) > 0
  • 3461: mserver5 crash on SQL: SELECT * FROM SYS.ARGS WHERE FUNC_ID NOT IN (SELECT ID FROM SYS.FUNCTIONS) OR FUNC_ID NOT IN (SELECT * FROM SYS.FUNCTIONS)
  • 3462: Invalid SQL (IN with subquery which returns multiple columns) is accepted
  • 3463: Crash on SELECT with SERIAL aggregation and GROUP BY column alias's
  • 3468: Local temporary table persists across sessions
  • 3469: Absolute network paths considered invalid for COPY INTO ... FROM statement.
  • 3473: Various memory leaks in SQL compilation
  • 3477: ODBC driver raises "unexpected end of input" for prepared string parameter from .NET application
  • 3481: Cannot run multiple COPY INTO statements in one 's'-command
  • 3482: Crossproduct error

Jan 2014-SP1 bugfix release

Build Environment
  • Configure now enables the SQL front end by default.
Merovingian
  • monetdb destroy -f now also works on running databases
Bug Fixes
  • 3442: COPY INTO ... LOCKED reports incorrect count
  • 3443: DROP INDEX crashes server with BATsubselect: invalid argument: b must have a dense head
  • 3444: AND after ON () of LEFT OUTER JOIN with certain expressions will cause crash
  • 3452: ODBC driver build fails on Mac OS X due to a conflicting types for the SQLColAttribute with the unixODBC library

Jan 2014 feature release

Build Environment
  • Created development packages for RPM based systems and Debian/Ubuntu containing the files needed to create extensions to the SQL front end.
  • Removed Mx, the literate programming tool. All code for the server is now pure C.
  • Created packages for RPM based systems and Debian/Ubunty containing the MonetDB interface to the GNU Scientific Library (gsl).
  • We no longer install the .la files in our Fedora/Debian/Ubuntu packages.
Client Package
  • ODBC: Implemented {fn scalar()} and {interval ...} escapes.
python2
  • Changed defaults for connecting (defaults to unix socket now).
  • Unix sockets partially working for control protocol.
  • Add support for unix socket.
python3
  • Changed defaults for connecting (defaults to unix socket now).
  • Unix sockets partially working for control protocol.
  • Add support for unix socket.
R
  • The R connector is now distributed in the source code packages.
MonetDB Common
  • The join code has been completely rewritten. It now optionally uses candidate lists, like the select code that was introduced in the previous release.
  • A new indexing structure for range selections on unsorted data has been added.
  • The vmtrim thread is no longer started by default on 64 bit architectures. The vmtrim thread monitors memory usage and drops BATs from memory when memory gets tight. However, in the age of large address spaces and virtual memory, the kernel does a good enough job. And in addition to dropping BATs, the thread also destroyed indexing structures which would have to be recreated the next time they were needed.
  • Cleaned up some of the parameters dealing with memory usage.
  • If available on the system, we now use atomic instructions for latching.
  • Removed some unused fields in the atomDesc structure. This change requires a complete recompilation of the whole suite.
  • Replaced the mutex implementation for both GNU C and Visual Studio with a home-grown implementation that uses atomic instructions (__sync_*() in gcc, _Interlocked*() in VS).
SQL
  • Added support for quantiles (generalization of median). Usage: SELECT quantile(column_name,0.25) FROM table_name; The value should be in the range 0..1.
Bug Fixes
  • 3040: Wrong NULL behavior in EXCEPT and INTERSECT
  • 3092: ODBC client doesn't support scalar function escape
  • 3198: SIGSEGV insert_string_bat (b=0x7fffe419d0a0, n=0x7fffc4006010, append=0) at gdk_batop.c:196
  • 3210: Unexpected concurrency conflict when inserting to 2 tables simultaneously and querying one of them
  • 3273: Add support to Python DBAPI package for timetz, inet and url types
  • 3285: no such table 'queryHistory'
  • 3298: GDKmmap messages and monetdb start db takes very long
  • 3354: Introduce query time-out
  • 3371: (i)like generates batloop instead of algebra.likesubselect
  • 3372: Large group by queries never complete - server at 100% cpu(all cores) until MonetDB stopped
  • 3383: Bad performance with DISTINCT GROUP BY
  • 3391: Bad performance with GROUP BY and FK with out aggregate function
  • 3393: "COPY .. INTO ..." - escape of string quotes
  • 3399: server crashed on simple (malformed) query
  • 3401: inconsistent/strange handling of invalid dates (e.g. 2013-02-29) in where clause
  • 3403: NOT NULL constraint can't be applied after deleting rows with null values
  • 3404: Assertion `h->storage == STORE_MMAP' failed.
  • 3408: nested concat query crashed server
  • 3411: (disguised) BETWEEN clause not recognised. Hence no rangejoin.
  • 3412: Boolean expressions in WHERE clause, result in incorrect resulsts
  • 3417: Nested Common Table Expressions Crash
  • 3418: Segmentation fault on a query from table expression
  • 3419: Database does not start after upgrade
  • 3420: Database does not start after upgrade
  • 3423: Group by alias with distinct count doesn't work
  • 3425: Temporal extraction glitches
  • 3427: Consistent use of current_timestamp and now()
  • 3428: Aggregation over two columns is broken
  • 3429: SAMPLE on JOIN result crashes server
  • 3430: Wrong temporary handling
  • 3431: SQLGetInfo returns incorrect value for SQL_FN_NUM_TRUNCATE
  • 3432: MonetDB SQL syntax incompatible with SQL-92 <delimited identifier> syntax
  • 3435: INDEX prevents JOIN from discovering matches
  • 3436: COPY INTO from file containing leading Byte Order Mark (BOM) causes corruption

Februari 2013

Februari 2013 sjoerd Thu, 02/20/2014 - 14:40

Feb 2013 feature release

Testing Environment
  • enabled "top-level" Mtest.py So far, while Mtest.py could be called in any subdirectory of the MonetDB source tree (and could then run all tests in the entire sub-tree), it was not possible to call Mtest.py in the top-level MonetDB source directory to run all tests. Instead, to run all tests, Mtest.py had to be called at least 4 times, once in each of these directories: "clients", "monetdb5", "sql", "geom". Now, it is possible to call Mtest.py once in the top-level MonetDB source directory to run all tests in one go. The behaviour of calling Mtest.py in any subdirectory, including the four mentioned above, did not changed, other than that now obsolete command line options "-p / --package <package>" and "-5 / --monetdb5" have been removed.
Java Module
  • merocontrol was changed to return server URIs, and lastStop time. Connections and dbpath were removed.
  • Mapi protocol v8 support was removed from MapiSocket. Protocol v8 has not been used by the servers any more since Apr2012 release
Client Package
  • Mapi protocol v8 support was removed from all client drivers. Protocol v8 has not been used by the servers any more since Apr2012 release
  • The tool mnc was removed from installations
  • msqldump: Implmented an option (--table/-t) to dump a single table.
  • Changed msqdump's trace option to be in line with mclient. In both cases, the long option is --Xdebug and the short option is -X.
MonetDB5 Server
  • mserver5: The --dbname and --dbfarm options have been replaced by the single --dbpath option.
  • The scheduler of mserver5 was changed to use a fixed set of workers to perform the work for all connected clients. Previously, each client connection had its own set of workers, easily causing resource problems upon multiple connections to the server.
Merovingian
  • Upgrade support for dbfarms from Mar2011 and Aug2011 was dropped
  • monetdb status now uses a more condensed output, to cater for the uris being shown, and prints how long a database is stopped, or how long ago it crashed
  • monetdb status now prints the connection uri for each database, when available. The connections and database path properties have been dropped.
  • monetdb status now prints last crash date only if the database has not been started since.
Bug Fixes
  • 2291: small doubles end up as NULL after arithmetic
  • 3215: Calculation Date function using interval and year
  • 3033: stethoscope needs better documentation
  • 3084: Timestamp arithmetic very slow (especially on Windows)
  • 3125: Python tests fail after recent Python API changes
  • 3172: assertion fails on table function with subselects as parameters
  • 3178: one scan is enough to implement ALGstdev_@1 in monetdb5/modules/kernel/algebra.mx
  • 3179: LIKE: batstr.like+algebra.uselect called instead of pcre.like_filter
  • 3193: Expressions not supported in the GROUP BY or ORDER BY clause.
  • 3216: "unknown property" error setting format and width in .monetdb file
  • 3217: gdk_posix fails to compile under NetBSD
  • 3221: can no execute large statements
  • 3222:  Monet DB giving wrong value from a table when a value is queried after insertion/ Data Loss
  • 3225: mserver segfault on select
  • 3227: MT_set_lock() call on an non-initialized lock

 

Feb 2013-SP1 bugfix release

Bug Fixes
  • 3228: Python client - mapi.Connection.__getbyes should throw exception when 0 bytes received from socket
  • 3229: GRANT does not work for UPDATE
  • 3230: GRANT SELECT priveleges not set in SQL FUNCTIONs
  • 3233: getting a crash of server with segmentation fault on SLES 11 64 SP2
  • 3235: MapiPP.pm unquote '0' as 'undef'
  • 3236: mclient \d produces no output
  • 3237: Assertion on BATgroup_internal
  • 3241: SEGFAULT on string processing when using optimizer.reduce()

In addition to these changes, we also fixed the database upgrade.  Due to internal changes in the Feb2013 release, some changes had to be made to the SQL catalog.  These changes were done incompletely in the Feb2013 release.  This has now been fixed in the Feb2013-SP1 release.  Both a direct upgrade from Oct2012 and from Feb2013 should work.

Feb 2013-SP2 bugfix release

MonetDB5 Server
  • Fixed argument parsing of mapi.reconnect() with 5 arguments.
MonetDB Common
  • Fixed a bug in case the candidate list is dense and completely outside the range of the bat being worked upon.
Java Module
  • The pre-compiled .jar files are now created using Java 7 instead of Java 6.
SQL
  • Added TEMPORARY to the non-reserved keywords, ie no need for double quotes when used as identifier.
Bug Fixes
  • 3245: SELECT from a view of a view
  • 3250: SELECT fails with subquery result missing when adding condition
  • 3251: Name of coercions optimizer wrong in opt_wrapper.c
  • 3257: stddev_pop and stddev_samp break during a GROUP BY
  • 3259: BETWEEN clause not producing rangejoin
  • 3261: Incorrect empty string values in query results
  • 3264: select + group by expression alias from a view kills database by signal SIGSEGV
  • 3276: Problem with starting monetdb on debian sqeeze

Feb 2013-SP3 bugfix release

MonetDB Common
  • System calls to flush files to disks were added. This may cause some slowdown, but it should provide better durability, especially in the face of power failures.
Java Module
  • Further improved setBigDecimal() method, based on patch by Ben Reilly in bug #3290
  • Fixed bug where PreparedStatement.setBigDecimal() wouldn't format its input well enough for the server causing odd errors.
  • Allow PreparedStatement.setXXX() methods to be called with null arguments, bug #3288
Bug Fixes
  • 3265: import decimal with copy into fails
  • 3269: copy into results into gdk_utils.c:728: GDKfree_: Assertion `(size & 2) == 0' failed.
  • 3277: count( case when COL1 in ( NUM1 , NUM2 ) then COL2 end ) cause problems in functions
  • 3278: SQL order by <column number> only sorts on first call
  • 3279: column alias combined with ORDER BY returns wrong data
  • 3280: median function breaks when the column contains NULL values
  • 3281: pqueue.enqueue_max type mismatch error
  • 3282: bulk loaded data corrupted when restart monetdb
  • 3286: TypeException when creating table with data by selecting from sys.storagemodel()
  • 3287: lower(name) clause in query causing "BATsubselect: invalid argument.." error
  • 3288: PreparedStatements may throw NullPointerException when setting parameter
  • 3289: SELECT: no such binary operator 'sql_add(boolean,boolean)'
  • 3290: Faulty type handling/casting in JDBC driver
  • 3295: mserver5 fails to start properly: "address JSONrenderobject not found"
  • 3296: Type checking in table functions
  • 3297: Data type changes when arithmetic in prepared statements.
  • 3300: explain select from storagemodel causes mserver5 to segfault
  • 3302: test exists-select.Bug-2933.sql fails after recent changes
  • 3303: Assertion on concurrent write

Feb 2013-SP4 bugfix release

Merovingian
  • monetdb now no longer compresses output when not connected to a terminal, bug #3292
Bug Fixes
  • 2781: SQL localtime() returns GMT time
  • 2962: client timezone information needs to be handled also for modifications to timestamp values
  • 3232: The function weekofyear returns results not according to the iso 8601 standard
  • 3255: M5 tests inlineFunction & ifthencst fail since changeset 207ed7d7e03a
  • 3292: monetdb status should print full line
  • 3307: distinct multicolumn performance and implementation suggestion for count distinct
  • 3310: Cast value to integer gives wrong result
  • 3311: strLength assertion on join and concat
  • 3313: Assertion in rel2bin_project
  • 3315: Incorrect behavior with subqueries in DELETE
  • 3317: correlated subselect does not work
  • 3318: UPDATE does not work with DECLEAREd TABLE in FUNCTION
  • 3319: UNIQUE column in DECLAREd TABLE in FUNCTION results in segfault
  • 3329: Function AVG fails during runtime in a procedure
  • 3330: Corrupt mitosis plan with group-by query
  • 3332: MEDIAN: no such operator 'median(decimal)'
  • 3336: DB files not removed if all rows are deleted, even after restart
  • 3337: Monet ODBC gives wrong schema information
  • 3339: Segvault in complex function call
  • 3340: crash on seeming incorrect SQL query
  • 3341: segfault caused by select rank() on a newly-loaded table
  • 3342: incorrect automatic cast from decimal to tinyint
  • 3345: copy binary into fails with the wrong error message
  • 3346: Deadlock in mal_dataflow's q_dequeue() (SciQL-2 branch)
  • 3348: SQL: LIKE incorrect with _
  • 3349: When selecting everything from a joined result, column contents are incorrect in one of my use cases.
  • 3351: gzip'ed files cannot be bulkloaded into DB
  • 3352: Median function with decimal type doesn't work
  • 3353: Readonly tables call for UNSET
  • 3358: Remote client connection stays in CLOSE_WAIT state
  • 3360: uninitialized memory causing bad output in query plans
  • 3362: ALTER TABLE operations silently remove previously set READ ONLY property
  • 3363: a join causes SIGSEGV after creation of Unique Key
  • 3367: Fully qualified order by column gives "relational query without result"
  • 3369: \0 Byte appears in MAPI

Feb 2013-SP5 bugfix release

Client Package
  • ODBC: Implemented retrieving variable-length data in parts with SQLGetData. See bug 3381.
MonetDB Common
  • Removed the heap cache. Since the fix for bug 3323 which made that the cache was actually getting used, bug 3376 made clear that it didn't work very well. In addition, on Linux at least, the heap cache slows things down.
Java Module
  • Rearranged order of returned columns of certain metadata functions to comply with the JDBC documentation. See bug 3356.
Bug Fixes
  • 3323: Heapcache bugs/performance issues
  • 3331: SAMPLE will return same result every time.
  • 3356: DatabaseMetaData.getColumns() doesn't work correctly when using index-based getters
  • 3367: Fully qualified order by column gives "relational query without result"
  • 3368: BAT sortedness info ignored on ORDER BY and TOPN
  • 3370: SQL environment settings are updated even when the set statement fails
  • 3373: Setting table to read-only (Bug 3362) fails on big-endian
  • 3375: LIKE join: BATfetchjoin does not hit always
  • 3376: COPY INTO fails with HEAPextend: failed to extend: MT_mremap() failed
  • 3377: Query interfering with next query in same transaction, after SP4
  • 3380: Python DBAPI driver throws exception when fetching EXPLAIN results
  • 3381: Windows ODBC driver hangs or crashes on simple queries

Feb 2013-SP6 bugfix release

MonetDB5 Server
  • Fixed a possible buffer overflow in the COPY INTO code.
  • Fixed a problem that when the server is using all available threads for a query, it was not possible to attach another client and have it execute even the smallest query. This is fixed by creating extra threads for each client entering the fray at the cost of having more threads that execute queries. But at least there is guaranteed progress for all clients (modulo the operating system scheduler). See bug #3258.
Merovingian
  • monetdbd(1) now refuses to startup if it cannot read the properties from the dbfarm, bug #3398
Client Package
  • mclient: Fixed a bug where the -H option only worked if the readline history file already existed. Now we properly create and use the history file.
  • ODBC: Fixed interpretation SQL_C_SLONG/SQL_C_ULONG/SQL_C_LONG to refer to a 32 bit integer always (i.e. "int" on 64 bit architectures despite the name and the Microsoft documentation). This seems to be the consensus.
  • ODBC: Fixed transaction level: MonetDB only supports the highest level (SQL_TXN_SERIALIZABLE), so setting the transaction level can be accepted and ignored.
MonetDB Common
  • Stopped using the deprecated sbrk() system call.
  • Fixed a problem when reverse sorting a sorted column.
  • Fixed bugs that deal with problems that could possibly occur when transactions are aborted and the server is restarted. See bug #3243.
  • A bug was fixed in the handling of grouped aggregates when all values in a group are null. See bug #3388.
SQL
  • Fixed a bug where the server at some point stopped saving compiled queries in the SQL query cache.
Bug Fixes
  • 3243: Segmentation fault (possible data corruption) after clean shutdown
  • 3258: Scheduling issues
  • 3368: BAT sortedness info ignored on ORDER BY and TOPN
  • 3374: UNIQUE constraint does not set tkey property on the corresponding BAT
  • 3382: Response to PREPARE emtpy if query contains a LIMIT
  • 3385: Simple query fails with 'identifier not found'
  • 3387: mclient does not properly double quote schema names when using autofill tab
  • 3388: case statement in "order by" clause doesn't work when used together with "group by"
  • 3389: median function with "group by" - SIGSEGV
  • 3392: ODBC datatype conversion for INTEGER not working properly
  • 3394: "Cannot find column type" error in temporary tables in functions
  • 3395: error occurred during a query: "'CASE WHEN" sentence
  • 3396: Improper UDF expansion
  • 3397: Error in ODBC-Driver when using Prepared Statements
  • 3398: Cannot stop monetdbd after erroneously starting an uninitialized dbfarm

 

October 2012

October 2012 sjoerd Tue, 02/19/2013 - 13:12

 

Oct 2012 feature release

Build Environment
  • Removed --enable-noexpand configure option.
Java Module
  • Fixed problem with PreparedStatements and setXXX() methods using column numbers instead of names, bug #3158
Client Package
  • mclient now accepts URIs as database to connect to.
  • all strings returned by python2 are unicode, removed use_unicode option
  • python2 and 3 type convertion speed improvements
  • python2 uses new style objects now (bug #3104)
  • split python2 and python3
MonetDB5 Server
  • Removed module attach since it wasn't used or even tested.
  • The MAL-to-C Compiler (mcc) was removed. The code wasn't tested and most likely non-functional.
Merovingian
  • Fixed problem where monetdbd would refuse to startup when discovery was set to false, bug #3155
MonetDB Common
  • Removed the gdk_embedded (and embedded) option. The code wasn't tested and most likely non-functional.
  • BAT-of-BATs is no longer allowed. It was already not allowed to make these types of BATs persistent, but now they can't be created at all anymore.
Bug Fixes
  • 3084: Timestamp arithmetic very slow (especially on Windows)
  • 3125: Python tests fail after recent Python API changes
  • 3155: Provide an option for setting discovery to false

 

No Debian and Ubuntu packages were built for this release (i.e. Oct2012) due to a packaging failure.  We expect to have resolved that problem by Oct2012-SP1.  We apologise for the inconvenience.

 

Oct 2012-SP1 bugfix release

Bug Fixes
  • 3155: Provide an option for setting discovery to false
  • 3161: invalid tuple received from server, got 138 columns, expected 476, ignoring
  • 3182: When using aggregate SQL functions, precision & decimal metadata for numeric/decimal type returned is 0,0

For this release (i.e. Oct2012-SP1) there are once again Debian and Ubuntu packages.  From this release onward, Ubuntu Lucid Lynx (10.04) is no longer supported.

 

Oct 2012-SP2 bugfix release

SQL
  • Fixed crash when performing an INSERT on a table with string-like column defaulting to NULL and omitting that column from VALUES, bug #3168
Java Module
  • Implemented type map support of Connection to allow custom mapping of UDTs to Java classes. By default the INET and URL UDTs are now mapped to nl.cwi.monetdb.jdbc.types.{INET,URL}. Most notably, ResultSet.getObject() and PreparedStatement.setObject() deal with the type map.
  • Fixed a problem in PreparedStatement where the prepared statement's ResultSetMetaData (on its columns to be produced) incorrectly threw exceptions about non existing columns. Bug #3192
Bug Fixes
  • 2579: SQL: incorrect cast from double to int
  • 2659: tuples INTERSECT vs count (... INTERSECT ...) differs
  • 2805: row_number doesn't work in complex query
  • 2886: warning: Catalog parameter '' ignored
  • 2977: timestamp minus date results in program error
  • 2978: now() + 1 results in illegal argument
  • 2999: Up-cast from SQL drops order information?
  • 3048: algebra.join undefined when using multiple arithmetic operations in SQL where-clause
  • 3124: insert into table with two foreign keys crashes mserver5
  • 3136: Memleak (in querycache?) while querying
  • 3143: Type resolution error in SQL procedural code
  • 3163: mserver5 crashes with large number of concurrent clients
  • 3168: crash when insert varchar/char/clob with default NULL
  • 3169: php driver uses call-time pass-by-reference
  • 3173: SQL queries with limit and offset clauses are unable to use bind parameters for either
  • 3181: Join query returns zero results when there is an index on the tables
  • 3185: inconsistency between documentation and implementation of command line options
  • 3186: case evaluted to early
  • 3188: Storage of client history
  • 3189: Overflow in calculation
  • 3190: When subtracting two columns of type "timestamp", no results are returned
  • 3191: rel_optimizer.c:5596: _rel_optimizer: Assertion `0' failed.
  • 3192: "No such column with index" error if CASE in SELECT clause
  • 3199: Inconspicuous sequence of prepare, execute, commit and close produces error
  • 3205: Inet datatype handling inconsistent between int, cast( as inet) and string

 

Oct 2012-SP3 bugfix release

Bug Fixes
  • 2579: SQL: incorrect cast from double to int
  • 2659: tuples INTERSECT vs count (... INTERSECT ...) differs
  • 2805: row_number doesn't work in complex query
  • 2886: warning: Catalog parameter '' ignored
  • 2977: timestamp minus date results in program error
  • 2978: now() + 1 results in illegal argument
  • 2999: Up-cast from SQL drops order information?
  • 3048: algebra.join undefined when using multiple arithmetic operations in SQL where-clause
  • 3124: insert into table with two foreign keys crashes mserver5
  • 3136: Memleak (in querycache?) while querying
  • 3143: Type resolution error in SQL procedural code
  • 3163: mserver5 crashes with large number of concurrent clients
  • 3168: crash when insert varchar/char/clob with default NULL
  • 3169: php driver uses call-time pass-by-reference
  • 3173: SQL queries with limit and offset clauses are unable to use bind parameters for either
  • 3176: Same query and different results
  • 3181: Join query returns zero results when there is an index on the tables
  • 3185: inconsistency between documentation and implementation of command line options
  • 3186: case evaluted to early
  • 3188: Storage of client history
  • 3189: Overflow in calculation
  • 3190: When subtracting two columns of type "timestamp", no results are returned
  • 3191: rel_optimizer.c:5596: _rel_optimizer: Assertion `0' failed.
  • 3192: "No such column with index" error if CASE in SELECT clause
  • 3199: Inconspicuous sequence of prepare, execute, commit and close produces error
  • 3205: Inet datatype handling inconsistent between int, cast( as inet) and string
  • 3208: Unable to execute prepared statements with a large number of arguments
  • 3209: exp_bin: Assertion `0' failed
  • 3212: Query cache does not work properly

 

July 2012

July 2012 fabian Tue, 10/23/2012 - 11:34

Jul 2012 feature release

Build Environment
  • Created seperate RPM and DEB packages for MonetDB/JAQL.
  • Removed --enable-bits option from configure.
  • Split the MonetDB-client-ruby RPM package into two and named them in accordance with the Fedora packaging guidelines as rubygem-<gem-name>.
SQL
  • COPY INTO now accepts optional parenthesis for file argument. Binary COPY INTO now requires 'COPY BINARY INTO'.
Client Package
  • ODBC: Fixed a bug where SQLNativeSql expected a statment handle instead of a connection handle.
MonetDB5 Server
  • Crackers code has been removed. Development continues in the holindex branch.
Merovingian
  • Fixed misc memory leaks, which caused monetdbd to grow in memory size over time.
  • Removed erroneously (re-)added master and slave properties, this functionality is currently not working.
MonetDB Common
  • Fixed intermittent problem that joins and selects return incorrect results and possibly inconsistent databases. The problems only occurred after a series of queries and updates, therefore it was hard to reproduce.
  • The sorted property, which was used to maintain whether a column in a BAT was sorted or reverse sorted, has been replaced by a pair of properties, sorted and revsorted. These new properties can be set independently (unlike the old sorted property), and so if both are set, the column must be constant. In addition, internal property checking has been overhauled. Now, when a property is set incorrectly, and when assertions are enabled, an assertion will go off. There is also a function which can derive properties.
  • Implemented proper overflow checking on all arithmetic operations.
Bug Fixes
  • 3016: Overflowing double multiplication leads to incorrect result
  • 3091: Since Apr2012-SP1 !TypeException:user.s11_1[59]:'bat.reverse' undefined in: _89:any := bat.reverse(_86:wrd)
  • 3099: ambiguous attributes in query with WITH clause and EXISTS clause
  • 3100: charset problem with the odbc driver for windows
  • 3103: display_size and internal_size of Cursor.description or incorrect
  • 3105: Leaking using Python
  • 3109: Wrong order in autoload mal script causes warning on startup
  • 3111: memory leak, monetdb set/get command
  • 3112: Median over temporal domains
  • 3113: Cannot set value of indexed column when doing UPDATE
  • 3114: Foreign Key prblems
  • 3116: Segfault in SQL

 

Jul 2012-SP1 bugfix release

Java Module
  • Fixed adaptive cache size used when retrieving results, not to cause divide by zero errors when memory gets short, bug #3119.
Client Package
  • mclient no longer prints the SQLSTATE at the start of each error returned by the SQL-server.
MonetDB5 Server
  • The server now distinguishes between starting and started states, such that monetdbd can wait for it to finish starting.
Merovingian
  • Starting a server now waits for as long as the server needs to possibly recover, bug #3134. In case of a long wait, the monetdbd logfile gives extra information on what the server is doing to recover.
  • Fixed a crash of monetdbd when local databases were unshared, bug #3135
  • Resolved a problem where automatic starting of a database initiated by multiple clients at the same time could cause failed starts. Bug #3107
Bug Fixes
  • 3075: inconsistent declaration of algebra.markH with 3 input arguments
  • 3090: crashed if using single identifier for where condition
  • 3093: sql fail if use scalar subquery with alias
  • 3107: concurrent connections to the same stopped database yield in multiple mserver5 starts by monetdbd
  • 3119: MonetConnection$ResultSetResponse throws java.lang.ArithmeticException: divide by zero
  • 3132: test/BugTracker-2011/func_iter_vs_bulk.Bug-2826 fails
  • 3134: Database gets killed by timeout during startup
  • 3135: monetdbd crash while creating & loading database

 

 

Jul 2012-SP2 bugfix release

Java Module
  • Fixed a bug where DatabaseMetaData.getURL() did return null:0 for hostname:port.
MonetDB5 Server
  • Changed the way nclients maximum was calculated to avoid 'out of client slots' errors way before the maximum was reached.
Merovingian
  • Resolved a problem where monetdb could fail to start a database with the message "database 'X' started up, but failed to open up a communication channel". Bug #3134, comment #7.
MonetDB Common
  • Fixed a bug in BATantijoin when either side is a singleton BAT. This fixes bug 3139.
Bug Fixes
  • 3117: test sql/test/BugTracker-2011/Tests/history.Bug-2909.sql produces extra profiling output since changeset 817328b74e2f
  • 3134: Database gets killed by timeout during startup
  • 3139: Not equal evaluation does not work
  • 3144: Memleak when connecting to a database (mclient and mapi)
  • 3145: mclient error when enabling history
  • 3150: Multiple string equality tests fail to match (Test_JdbcClient)

April 2012

April 2012 mk Thu, 04/28/2011 - 17:47

Apr 2012-SP2 bugfix release

Client Package
  • Resolved a cosmetical error where tab-characters would cause misalignments in tabular result views. For the time being, tabs are now represented as a single space in tabular view.
MonetDB Common
  • Limit number of detected CPU cores to 16 on 32-bits systems to avoid running quickly out of addressable resources followed by a kill from the OS.
Bug Fixes
  • 2594: NULL cannot automatically be casted to every other type
  • 3085: assertion fails upon execution of a valid SQL query
  • 3086: Modulo function does not work in select from table
  • 3087: query causes "Not yet implemented" message
  • 3089: mserver5 crashed if join condition has two column and incorrect order
  • 3090: crashed if using single identifier for where condition
  • 3096: Type of median error

 

 

Apr 2012-SP1 bugfix release

Build Environment
  • Windows: we now use OpenSSL 1.0.1b.
SQL
  • Databases that were upgraded from the Aug2011 release have an error in the catalog for SQL procedures. This is now fixed.
Merovingian
  • Fixed a bug where connecting to a stopped multiplex-funnel would result in a 'there are no available connections' error.
MonetDB Common
  • Implemented MT_getrss for Mac OS X systems, this allows the server to know about how much memory is currently in use.
Bug Fixes
  • 2454: SQL: add various statistical functions
  • 2916: non admin user cannot create temporary tables
  • 2949: SQL exception during execution of a correct user defined function that just returns a relation table constructed by a simple SQL select query
  • 2958: Grant does't work with schema syntax
  • 2960: mclient terminates connection during insertion of a large number of records from a file
  • 2972: SQL URL functionality contains errors
  • 2987: Broken type inference when using sql round
  • 2992: SQL module truncates DECIMALS when passing integers to MAL module
  • 3000: Inconsistency between read-only and read-write modes after dropping a schema
  • 3028: Segmentation fault in seqscan_eq_bte_bte_tloc_void_void
  • 3046: mserver5 terminates with segmentation fault
  • 3063: datacell functions no longer work after restart of database
  • 3072: Database upgrade from Dec2011 to Apr2012 broken for storage()
  • 3073: VERY slow foreign key check (due to missing batcalc.isnil() ?)
  • 3074: Name resolution in procedures
  • 3077: No errors on UDF returning table with too many columns
  • 3079: Converting an Oracle schema DDL for MonetDB fails to create a column named "no".

 

 

Apr 2012 feature release

Build Environment
  • Fixed some of the package names for Debian/Ubuntu. Packages for libraries should contain the major number of the library version number. This was not always the case.

  • On Debian and Ubuntu, install Python modules in dist-packages instead of site-packages. This fixed bug 2997.

SQL
  • Fixed a crash that happened at the end of a database upgrade to the Dec2011 database scheme. The crash happened during cleanup after the database was upgraded, so it was merely inconvenient.

  • Stripped off implementation-specific parts from error messages before they get presented to the user.

Java Module
  • Password reading by JdbcClient no longer results in strange artifacts

  • JdbcClient now returns exit code 1 in case of failures

  • Implemented missing Number types support in PreparedStatement.setObject()

  • Fixed bug in DatabaseMetaData.getSchemas() method that caused an SQL error when called with catalog and schema argument.

  • Resolved a bug where JDBC and Control connections could terminate abruptly with 'Connection closed' messages

Client Package
  • ODBC: Implemented the SQL_ATTR_CONNECTION_TIMEOUT attribute.

  • mclient now has a -a (--autocommit) option to turn off autocommit mode.

MonetDB5 Server
  • The type "chr" has been removed. chr has long been superseded by bte for 1 byte arithmetic plus it is pretty useless to hold single characters since we use Unicode and thus only a tiny subset of the supported character set would fit.

  • Fixed potential crash by dealing better with non-standard types.

  • A couple of memory leaks have been plugged.

Merovingian
  • The logfile and pidfile monetdbd properties are now displayed with dbfarm path when relative

  • Fixed a bug in the multiplex-funnel where certain clients would abort on responses for update queries.

  • Fixed problem where version and mserver properties for monetdbd were not always successfully retrieved. Bug #2982.

  • Fixed problem where shutdown of monetdbd would lead to shutting down database 'control' which does not exist. Bug #2983.

  • Fixed issue causing (harmless) 'error reading from control channel' messages. Bug #2984.

  • Resolved problem where remote start/stop/etc. commands with monetdb would report error 'OK'. Bug #2984.

MonetDB Common
  • The type "chr" has been removed. chr has long been superseded by bte for 1 byte arithmetic plus it is pretty useless to hold single characters since we use Unicode and thus only a tiny subset of the supported character set would fit.

Bug Fixes
  • 2994: test monetdb5/modules/kernel/Tests/vacuum.mal fails since changeset ada288cef37c
  • 3023: SQL tests fail as error messages now go to stdout iso. stderr
  • 3030: optimizer tests fail with new optimizer framework
  • 3032: numerous M5/MAL tests fail on SunOS/Solaris as TypeExecptions do no longer show arguments in command signatures

 

December 2011

December 2011 sjoerd Fri, 04/27/2012 - 14:22

MonetDB Dec2011 Release

 

 

Dec 2011-SP2 bugfix release

Java Module
  • Implemented missing Number types support in PreparedStatement.setObject()
Bug Fixes
  • 2915: Query (mclient or jdbc) with 2 outer joins appears to crash connectionor server
  • 2947: Queries where a "is null" clause is combined with two equals clauses fail to produce the expected result.
  • 2968: Memory Leaks
  • 2974: thetajoin produces invalid/inconsistent bat
  • 2988: Subquery in select part goes wrong
  • 2998: Equivalent multi-way join queries give different results
  • 3002: String plus interval result in program errors upon comparison
  • 3011: count(*) in subquery results in 1 for query without results
  • 3012: bitwise comparison fails after a cast
  • 3029: test sql/test/Dependencies/Tests/Dependencies.sql fails after recent changes
  • 3031: Virtually all SQL tests fail on Windows
  • 3036: optimizers are not getting set correctly through set optimizer
  • 3038: ~190 SQL tests fail on Darwin, FreeBSD & Solaris (only) since introduction of "(extended) relational algebra"
  • 3043: WITH clause is not allowed in derived table
  • 3045: algebra.project(0@0,bat) fails at propcheck
  • 3052: Wrong Error Code on Rollback Exception

 

Dec 2011-SP1 bugfix release

Build Environment
  • Fixed some of the package names for Debian/Ubuntu. Packages for libraries should contain the major number of the library version number. This was not always the case.
  • On Debian and Ubuntu, install Python modules in dist-packages instead of site-packages. This fixed bug 2997.
SQL
  • Fixed a crash that happened at the end of a database upgrade to the Dec2011 database scheme. The crash happened during cleanup after the database was upgraded, so it was merely inconvenient.
  • Stripped off implementation-specific parts from error messages before they get presented to the user.
Java Module
  • Fixed bug in DatabaseMetaData.getSchemas() method that caused an SQL error when called with catalog and schema argument.
  • Resolved a bug where JDBC and Control connections could terminate abruptly with 'Connection closed' messages
MonetDB5 Server
  • Fixed potential crash by dealing better with non-standard types.
Merovingian
  • Fixed a bug in the multiplex-funnel where certain clients would abort on responses for update queries.
  • Fixed problem where version and mserver properties for monetdbd were not always successfully retrieved. Bug #2982.
  • Fixed problem where shutdown of monetdbd would lead to shutting down database 'control' which does not exist. Bug #2983.
  • Fixed issue causing (harmless) 'error reading from control channel' messages. Bug #2984.
  • Resolved problem where remote start/stop/etc. commands with monetdb would report error 'OK'. Bug #2984.
Bug Fixes
  • 2862: foreign key self references cause violation upon alter table add constraint...
  • 2933: "TypeException: algebra.join undefined" when evaluating EXISTS (SELECT ...) predicate
  • 2946: Invalid BAT if left join condition has only columns from right table
  • 2964: LEFT OUTER JOIN with constants returns errors
  • 2969: Result precision issue with Decimal data type
  • 2973: Date coercion functionality tests
  • 2979: misc merovingian control problems
  • 2982: monetdbd status and mserver properties have random values
  • 2983: monetdbd attempts to stop non-existing database 'control'
  • 2984: monetdb create/release/start/stop returns error "OK" for remote connections
  • 2995: Test geom/sql/Tests/example.sql crashes since changeset 6399077ff8a4
  • 2997: monetdb-python packages installed in wrong dir on Ubuntu 10.04 LTS (lucid)
  • 3004: Incorrect evaluation of conditions in WHERE part of SQL statement
  • 3009: Segmentation fault on simple query
  • 3013: calc.and is not yet defined for dbl
  • 3014: dayofweek comment is wrong in mtime.mx:700
  • 3017: rel_bin.c:611: exp_bin: Assertion `0' failed. with between on multicolumns
  • 3020: Assertion error in debugging mode
  • 3034: INNER JOIN with Subquery produces mal errors
  • 3041: WHERE condition ignored after 2 LEFT JOINs with subqueries

 

Dec 2011 feature release

SQL
  • mclient: The csv output format can now also be of the form csv=c and csv+c where c is the column separator. The form with csv+ produces a single header line with column names.
  • Changed a bug where the sign() function returned the same type as its argument instead of always an INTEGER.
  • Added a fix for bug #2834, which caused weird (failing) behaviour with PreparedStatements.
  • Fixed a bug when using default values for interval columns. Bug 2877.
  • A bug was fixed where deleted rows weren't properly accounted for in all operations. This was bug 2882.
  • A bug was fixed which caused an update to an internal table to happen too soon. The bug could be observed on a multicore system with a query INSERT INTO t (SELECT * FROM t) when the table t is "large enough". This was bug 2883.
Java Module
  • Implemented getMetaData() method of PreparedStatement.
  • Fixed an AssertionError for special results from e.g. EXPLAIN queries.
  • Fixed overflow error when batching large statements, bug #2952
  • Resolved a concurrency problem where ResultSet's date-related getters could cause odd stack traces when used by multiple threads at the same time, bug #2950.
  • JDBC now implements JDBCv4.1 which makes it possible to be built with Java 7 and up. JDBCv4.1 is a maintenance release of JDBCv4, and hence can be compiled with Java 6, although the added methods obviously are not part of the java.sql interfaces.
  • SQLExceptions thrown now carry a SQLSTATE. Until the server starts sending correct SQLSTATEs for all errors, server originated errors without SQLSTATE are considered generic data exceptions (22000).
  • Fixed a bug where closing ResultSets and PreparedStatements could lead to errors on concurrent running queries using the same Connection due to a race condition.
  • Changed version scheme of JDBC driver and MCL jar to be more standard, from monetdb-X.Y-<thing>.jar to monetdb-<thing>-X.Y.jar, bug #2943
  • The embedded monet instance capability of MonetConnection was removed.
  • Bump JDBC version to 2.0 (Liberica). JDBC now implements JDBCv4 which makes it possible to be built with Java 6 and up. Java 5 and before are no longer supported, and can use the 1.x releases of the driver.
  • INTERVAL columns are now treated as decimals, since they can have sub-second precision.
Client Package
  • Fix making connections with stethoscope to hosts without monetdbd. Bug #2944.
  • Fixed some bugs in the PHP interface affecting the mapi_ping(), monetdb_insert_id() and auto_commit() functions. Bugs #2936, #2937, #2938.
  • ODBC: Fixed SQLNumResultCols and SQLDescribeCol to return useful information after a call to SQLPrepare and before any SQLExecute.
  • mclient: The exact interpretation of the -i (--interactive) option and the - filename argument have changed. The - filename argument means read from standard input and no longer implies that no \ interpretation is done on the input. Instead, \ interpretation is done if either standard input is a terminal, or if the -i option is given. The -i option no longer causes mclient to read from standard input. It only means to do \ interpretation when reading from standard input. Use the - filename argument to read from standard input. Note that if no -s option is specified and no filename arguments are present, mclient still reads from standard input.
  • ODBC: Improved internal query for SQLSpecialColumns. Before, the query returned all columns taking part in a PRIMARY KEY *and* all columns taking part in a UNIQUE constraint. Now it returns only one or the other set.
  • ODBC: The database name is now used as the catalog name throughout. Functions that return a catalog name return the database name, and functions that match on catalog name match it with the database name.
  • ODBC: Implemented an easier way to create a log file of interactions with the ODBC driver. You can now add a connection attribute "LOGFILE=filename" to the connection string parameter of SQLBrowseConnect and SQLDriverConnect, and to the relevant part of the Windows registry or odbc.ini file. This value is only used if there is no environment variable ODBCDEBUG.
  • Quoting of object names for mclient's \d command is now more flexible and consistent with standard SQL quoting rules, bug #2846.
  • ODBC: Implemented SQLColumnPrivileges function.
  • Install new program, mnc, which provides netcat functionality, but based on MonetDB's communication libraries
  • Report full URI to database instead of just database when available in mclient.
  • The time format of the timer output can now be controlled with an optional argument to the -i option. ms, s and m force the time to be formatted as milliseconds, seconds or minutes + seconds respectively.
  • ODBC: Implemented a workaround in SQLTables for bug 2908.
  • Small improvement to mclient's table rendering for tables without any rows. Previously, the column names in the header could be squeezed to very small widths, degrading readability.
  • Python DB API connect() function now supports PEP 249-style arguments user and host, bug #2901
  • mclient now checks the result of encoding conversions using the iconv library.
  • Fixed a source of crashes in mclient when a query on the command line using the -s option is combined with input on standard input (e.g. in the construct mclient -s 'COPY INTO t FROM STDIN ...' < file.csv).
  • Fixed bug 2897 where slow (network) reads could cause blocks to not be fully read in one go, causing errors in the subsequent use of those blocks. With thanks to Rémy Chibois.
  • Fixed a bug in mclient where processing queries from files could result in ghost empty results to be reported in the output
  • Fixed Perl DBD rowcount for larger results, bug #2889
  • Perl: We now distinguish properly between TABLE and GLOBAL TEMPORARY (the latter are recognized by being in the "tmp" schema).
  • Perl: fixed a bunch of syntax errors. This fixes bug 2884. With thanks to Rémy Chibois.
  • Perl: Fixed DBD::monetdb table_info and tabletype_info. This fixes bug 2885. With thanks to Rémy Chibois.
  • mclient: fix display of varchar columns with only NULL values.
  • Fixed a bug in mclient/msqldump where an internal error occurred during dump when there are GLOBAL TEMPORARY tables.
MonetDB5 Server
  • A couple of memory leaks have been plugged.
  • Introduced new variable max_clients that allows to define how many concurrent connections are allowed to be made against the database.
  • mal_interpreter.mx Protect against concurrent exceptions If multiple parallel blocks access the exception variables and perform GDKfree on old messages, then we may enter a case that one thread attempts a second free. Simple lock-based protection is the first line of defense.
  • bpm and partitions have been moved to the attic. It is replaced by the partition optimizer, still under construction.
  • The MAL debugger next/step operations semantics have been fixed. The profiler also now clearly shows entrance/exit of a MAL function.
  • The dataflow optimizer uses a less strict side-effect test for BAT new operations and better recognizes trivial plans.
  • Fixed a problem where MAL variables weren't properly cleared before reuse of the data strucutre. This problem could cause the data flow scheduler to generate dependencies between instructions that didn't actually exist, which in turn could cause circular dependencies among instructions with deadlock as a result. Bugs 2865 and 2888.
Merovingian
  • Fixed crash in monetdb when an invalid property was retrieved using the get command, bug #2953.
  • monetdb get output is now grouped by database instead of by property
  • Unlike in previous releases, 'monetdbd get mserver' now returns the path to the mserver5 binary only for dbfarms that have a running monetdbd, instead of only for those that are not served by a monetdbd.
  • Add nclients property that controls the maximum number of concurrent clients allowed to access the database
  • Starting monetdbd without any arguments or without dbfarm is no longer supported. A dbfarm now must be provided for each command.
  • The control passphrase has been turned into a hash of the password, for a more stronger authorisation model as used by mclients. On upgrade, any existing passphrase is converted to the hashed version of the passphrase.
  • The monetdbd controlport option has been removed in favour of a boolean control option. On upgrade, when controlport was set to a non-zero value and a passphrase was set, control is set to true, or false otherwise.
  • The monetdbd discoveryport option has been removed in favour of a boolean discovery option. On upgrade, when discoveryport was set to a non-zero value, discovery is set to true, or false otherwise.
  • Fixed a bug where monetdbd's socket files from /tmp were removed when a second monetdbd was attempted to be started using the same port.
  • Resolved problem where monetdbd would terminate abnormally when databases named 'control', 'discovery' or 'merovingian' were stopped.
  • monetdbd get status now also reports the version of the running monetdbd
  • Improved response time of 'monetdb start' when the database fails to start.
  • Fixed a bug in monetdbd where starting a failing database could incorrectly be reported as a 'running but dead' database.
  • To avoid confusion, all occurrences of merovingian were changed into monetdbd for error messages sent to a client.
MonetDB Common
  • Fixed memory detection on 32-bits Solaris systems with more memory than can be addressed in 32-bits mode, bug #2935
Bug Fixes
  • 2740: Segmentation fault using a column inside of a functioncall
  • 2802: update JDBC driver to JDBC4
  • 2846: \d command fails if the schema is provided explicitly
  • 2873: SQL: time with precision
  • 2891: Make MapiPP.pm fetch results progressively
  • 2910: SQLSetStmtOption(..., SQL_CONCURRENCY, SQL_CONCUR_READ_ONLY) should be allowed
  • 2914: mserver5 crash for huge_nr_of_columns.Bug-2609 on ppc-macos
  • 2934: Database upgrade with SQL catalog upgrade messes up privileges
  • 2935: mserver5 doesn't detect memory on Solaris/Sparc
  • 2936: bug into mapi_ping()
  • 2937: minor bug into monetdb_insert_id() (php_monetdb.php)
  • 2938: minor bug into auto_commit() (php_monetdb.php)
  • 2939: "PHP Notice" in mapi_execute()
  • 2940: PHP split() function deprecated
  • 2944: stethoscope on can't connect to database without monetdbd (discovery enabled)
  • 2950: MonetResultSet use of SimpleDateFormat not threadsafe
  • 2952: utf-8 characters in long string
  • 2953: segfault for monetdb get with invalid property
  • 2959: Nested SELECT with EXISTS causes the server to crash
  • 2963: invalid tuple received from server, got 17 columns, expected 24, ignoring
  • 2965: joinPath runs indeterminately
  • 2975: CSV file is kept write-locked after mclient failed to COPY
  • 2981: Different answer from querys in the ODBC compared to mcliente or other DBMSs

August 2011

August 2011 sjoerd Thu, 01/26/2012 - 15:43

The Aug2011 release is the first release where the default size of OIDs on 64-bit WIndows systems is 64 bits.  Before, the default on Windows was 32 bits.  On all other 64-bit platforms, the default size of OIDs was also 64 bits.  This means that, on 64-bit Windows systems, the database needs to be converted.  This is done automatically when the database is first opened, but as always with automatic conversions, it is a good idea to make a backup first.

The Aug2011 uses a slightly different catalog from earlier releases.  The database is automatically updated when first opened.

This release is the last release that comes with a JDBC driver that can be used with Java 1.4 and 1.5, also known as JDBCv3.

 

 

 

Aug 2011-SP3 bugfix release

SQL
  • Added a fix for bug #2834, which caused weird (failing) behaviour with PreparedStatements.
Merovingian
  • Fixed a bug where monetdbd's socket files from /tmp were removed when a second monetdbd was attempted to be started using the same port.
Bug Fixes
  • 2875: incomplete database initialization
  • 2912: Function body disappears after loading data - error message: SQLException:sql.resultSet:Cannot access descriptor
  • 2919: Add column and select in same transaction
  • 2920: Updated database from Aug2011 to Dec2011 make queries fail with "Subquery result missing".
  • 2921: complex where clause broken
  • 2923: Multiplication and cast from decimal to integer leads to 0 value
  • 2924: use LIKE with ESCAPE ignores OR operator
  • 2928: Cannot update a field with a count query based on an other table

 

 

 

Aug 2011-SP2 bugfix release

Client Package
  • ODBC: Implemented a workaround in SQLTables for bug 2908.
  • Small improvement to mclient's table rendering for tables without any rows. Previously, the column names in the header could be squeezed to very small widths, degrading readability.
  • Python DB API connect() function now supports PEP 249-style arguments user and host, bug #2901
  • mclient now checks the result of encoding conversions using the iconv library.
  • Fixed a source of crashes in mclient when a query on the command line using the -s option is combined with input on standard input (e.g. in the construct mclient -s 'COPY INTO t FROM STDIN ...' < file.csv).
  • Fixed bug 2897 where slow (network) reads could cause blocks to not be fully read in one go, causing errors in the subsequent use of those blocks. With thanks to Rémy Chibois.
  • Fixed a bug in mclient where processing queries from files could result in ghost empty results to be reported in the output
Merovingian
  • Resolved problem where monetdbd would terminate abnormally when databases named 'control', 'discovery' or 'merovingian' were stopped.
  • monetdbd get status now also reports the version of the running monetdbd
  • Improved response time of 'monetdb start' when the database fails to start.
  • Fixed a bug in monetdbd where starting a failing database could incorrectly be reported as a 'running but dead' database.
  • To avoid confusion, all occurrences of merovingian were changed into monetdbd for error messages sent to a client.
Bug Fixes
  • 2346: mkey.bulk_rotate_xor_hash MAL exception
  • 2712: un-rolled-back savepoints cause crash upon scenario exit
  • 2774: COPY INTO of double (~1.0e-310) cannot be imported.
  • 2890: Slow execution of COUNT in combination with DISTINCT
  • 2894: No consistent behavior of implicit cast of integer to double
  • 2897: Mapi.pm: short reads not handled when reading blocks from server
  • 2898: order by for multiple (two) columns fails for large tables when second ordering is 'desc': "mat.refine_reverse' undefined"
  • 2900: rand(int) should not return NULL
  • 2901: Make keyword arguments in Python DB API connect() function more standard
  • 2904: TypeException after calling UDF in other than sys schema (when it is initialised)
  • 2906: "ALTER TABLE mytable SET READ ONLY" results in server segfault
  • 2908: SQL query produces error with MAL statements
  • 2909: query result is not displayed when history = true

 

Aug 2011-SP1 bugfix release

SQL
  • Fixed a bug when using default values for interval columns. Bug 2877.
  • A bug was fixed where deleted rows weren't properly accounted for in all operations. This was bug 2882.
  • A bug was fixed which caused an update to an internal table to happen too soon. The bug could be observed on a multicore system with a query INSERT INTO t (SELECT * FROM t) when the table t is "large enough". This was bug 2883.
Client Package
  • Fixed Perl DBD rowcount for larger results, bug #2889
  • Perl: We now distinguish properly between TABLE and GLOBAL TEMPORARY (the latter are recognized by being in the "tmp" schema).
  • Perl: fixed a bunch of syntax errors. This fixes bug 2884. With thanks to Rémy Chibois.
  • Perl: Fixed DBD::monetdb table_info and tabletype_info. This fixes bug 2885. With thanks to Rémy Chibois.
  • mclient: fix display of varchar columns with only NULL values.
  • Fixed a bug in mclient/msqldump where an internal error occurred during dump when there are GLOBAL TEMPORARY tables.
MonetDB5 Server
  • Fixed a problem where MAL variables weren't properly cleared before reuse of the data strucutre. This problem could cause the data flow scheduler to generate dependencies between instructions that didn't actually exist, which in turn could cause circular dependencies among instructions with deadlock as a result. Bugs 2865 and 2888.
Bug Fixes
  • 2865: mserver5 --set gdk_nr_threads=2 --forcemito: deadlock during first SQL client connect on virgin (empty) DB
  • 2877: assertion failed when creating table with default interval
  • 2879: A SELECT query fails to produce some results
  • 2882: Delete of some records don't work on Monetdb
  • 2883: insert into t (select * from t) bogus in multithreaded execution
  • 2884: Mapi.pm and GetInfo.pm syntax fixes for array usage
  • 2885: DBD::monetdb table_info error
  • 2887: Division by zero error, when should not divide at 0
  • 2888: test/null/null_in_case.sql hangs server
  • 2889: Mapi::getBlock is erasing query count on second call

 

 

Aug 2011 feature release

Build Environment
  • Fixed Fedora 15 (and presumably later) configuration that uses a tmpfs file system for /var/run. This fixes bug 2850.
  • The default OID size for 64-bit Windows is now 64 bits. Databases with 32 bit OIDs are converted automatically.
Java Module
  • Add so_timeout Driver property to specify a SO_TIMEOUT value for the socket in use to the database. Setting this property to a value in milliseconds defines the timeout for read calls, which may 'unlock' the driver if the server hangs, bug #2828
  • Added a naive implementation for PreparedStatement.setCharacterStream
  • The obsolete Java-based implementation for PreparedStatements (formerly activated using the java_prepared_statements property) has been dropped
Client Package
  • msqldump now also accepts the database name as last argument on the command line (i.e. without -d).
  • Made error messages from the server in mclient go to stderr, instead of stdout.
  • Implemented SQL_ATTR_METADATA_ID attribute. The attribute is used in SQLColumns, SQLProcedures, and SQLTablePrivileges.
  • Implemented SQLTablePrivileges in the ODBC driver.
  • mclient now automatically sets the SQL `TIME ZONE' variable to its (the client's) time zone.
  • Removed perl/Cimpl, MonetDB-CLI-MapiLib and MonetDB-CLI-MapiXS
  • Switched implementation of MonetDB::CLI::MapiPP to Mapi.pm, and made it the default MonetDB::CLI provider.
  • Made Mapi.pm usable with current versions of MonetDB again
  • When the first non-option argument of mclient does not refer to an exising file, it now is taken as database name. This allows to simply do `mclient mydb`.
MonetDB5 Server
  • Make crackers optional and disable by default, since it wasn't used normally
  • Authorisation no longer takes scenarios into account. Access for only sql or mal is no longer possible. Any credentials now mean access to all scenarios that the server has available.
Geom Module
  • Implemented NULL checks in the geom module. Now when given NULL as input, the module functions return NULL instead of an exception. This fixes bug 2814.
Merovingian
  • Fixed confusing 'Success' error message for monetdb commands where an invalid hostname was given
  • The path to the mserver5 binary is no longer returned for the mserver property with monetdbd get for a dbfarm which is currently served by a monetdbd. Since the called monetdbd needs not to be the same as the running monetdbd, the reported mserver5 binary may be incorrect, and obviously lead to confusing situations. Refer to the running monetdbd's logfile to determine the mserver5 binary location instead.
  • Added -n option to monetdbd start command, which prevents monetdbd from forking into the background.
  • Fix incorrect (misleading) path for pidfile in pidfile error message, bug #2851
  • added status property to get command
MonetDB Common
  • Removed conversion code for databases that still used the (more than two year) old format of "string heaps".
  • On Windows and Linux/Unix we can now read databases built on the other O/S, as long as the hardware-related architecture (bit size, floating point format, etc.) is identical.
  • Implemented automatic conversion of a 64-bit database with 32-bit OIDs to one with 64-bit OIDs.
Bug Fixes
  • 2336: ANSI Client clobbered data ::SQLTables
  • 2579: SQL: incorrect cast from double to int
  • 2701: Divide by zero in OPTmitosis with select name from tables
  • 2814: Throws exception if a SQL/GIS takes argument null
  • 2817: Failed to find a table producing function
  • 2828: No Socket Timeout: MonetDB Java client hangs forever on reads and consume threads when the database is crashed
  • 2836: compilation fails when --with-valgrind in effect
  • 2838: Nested case results in message: "Predicate CASE 42: time to implement some more"
  • 2839: test monetdb5/optimizer/Tests/inlineFunction2 fails
  • 2840: Test monetdb5/optimizer/Tests/reorder01 fails
  • 2841: Tests monetdb5/optimizer/Tests/inline0{2,3,6} fail
  • 2842: Tests monetdb5/optimizer/Tests/dataflow{,3,4} lack stable output
  • 2845: INSERT INTO fails (hangs) when values are derived from SELECT, in combination with a FOREIGN KEY constraint
  • 2847: SELECT statement throws str_cast exception
  • 2848: Problem when reusing column name as CASE... WHEN alias in subquery
  • 2849: Multi-column constraints cause data corruption
  • 2850: monetdbd cannot open pidfile on Fedora 15
  • 2851: Incorrect error message when pidfile cannot be opened
  • 2852: Query with CASE and nested functions results in very long execution
  • 2855: test monetdb5/modules/mal/Tests/inspect00 fails
  • 2856: test monetdb5/optimizer/Tests/dataflow fails
  • 2857: obsolete conditional HAVE_LIBXML2 still used on Windows
  • 2859: prepare statement tests fail with number of arguments mismatch
  • 2860: /etc/init.d/monetdb5-sql uses incorrect PIDFILE to test if monetdb is running
  • 2862: foreign key self references cause violation upon alter table add constraint...
  • 2866: test monetdb5/optimizer/Tests/dataflow.mal fails on all Windows & Solaris (only)
  • 2868: tests tst285 & pithole fail on some Windows platforms
  • 2869: SQL tests with "LIKE" crash on Windows2008 & WindowsXP while working fine on Windows7

 

April 2011

April 2011 mk Sun, 06/26/2011 - 11:12
This is the first release of the MonetDB suite that does not contain MonetDB4 and MonetDB/XQuery. As already mentioned in the Mar2011 release notes, work on the Pathfinder compiler, which forms the basis of MonetDB/XQuery, continues at The University of Tübingen.

An upgrade from the Mar2011 release should be smooth, even if MonetDB/XQuery is installed. The Windows installers for MonetDB/SQL and MonetDB/XQuery have always been independent, and the Fedora, Debian, and Ubuntu versions have been made independent in the Mar2011 release.

The Apr2011 release uses a slightly changed SQL catalog than previous releases. The catalog will be updated automatically the first time an SQL database is opened. Note that because of the incompatibility, once updated, the database cannot be read by older versions. Also note that a database from the Feb2010 release or before cannot be read by this release. First update the database with the Mar2011 release.

This is also the first release where we have created FreeBSD binaries. These binaries are highly experimental. The software should run as well as on any other system, but the packaging is obviously different from other systems. As before, we also provide binaries for Windows, Fedora (13 and 14), Debian (5.0 and 6.0), Ubuntu (9.10, 10.04, and 10.10), Mac OS X, and Solaris.

 

 

Apr 2011-SP2 bugfix release

Build Environment
  • We can now build RPMs on CentOS 6.0. Since there is no geos library on CentOS, we do not support the geom modules there.
  • We can now also build on Fedora 15. This required some very minor changes.
  • Changed configure check for OpenSSL so that we can also build on CentOS 5.6. We now no longer demand that OpenSSL is at least version 0.9.8f, but instead we require that the hash functions we need are supported.
Client Package
  • Fix stethoscope's mod.fcn filter when using multiple targets, bug #2827
  • The separate Python distribution now uses the same version number as the main package.
MonetDB Common
  • Fixed a problem where appending string BATs could cause enormous growth of the string heap. This fixes bug 2820.
  • Fixes to memory detection on FreeBSD.
Java Module
  • Return false from Statement.getMoreResults() instead of a NullPointerException when no query has been performed on the Statement yet, bug #2833
  • Fixed read-only interpretation. Connection.isReadOnly now always returns false, setReadOnly now generates a warning when called with true. Partly from bug #2818
  • Allow readonly to be set when autocommit is disabled as well. Bug #2818
SQL
  • Fixed regression where the superuser password could no longer be changed, bug #2844
  • Fixed incorrect insert counts.
  • Fixed bug 2823: MAL exeption on SQL query with subquery in the where part.
  • Redirect error from create scripts back to the first client. This fixes bug 2813.
  • Added joinidx based semijoin; push join through union (using joinidx).
  • Fixed pushing select down.
Bug Fixes
  • 2058: scoping issue of functions and with
  • 2734: MALException:mkey.bulk_rotate_xor_hash on complex SQL query
  • 2767: `make uninstall` fails in clients/rubyadapter
  • 2797: mserver crashes at initialization when the disk is full
  • 2811: mserver crashes
  • 2812: Subqueries in FROM clause not accepted
  • 2813: SQL: catalog is built even error has occurred
  • 2818: JDBC: exception on calling Connection.setReadOnly
  • 2819: make text readable
  • 2820: SIGSEGV in BATins_kdiff triggered by a DELETE statement
  • 2821: segmentation fault when a select query (with group by) is run on server with "--single-user" option
  • 2822: A query with CASE and division results in ArithmeticException
  • 2823: MAL exeption on SQL query with subquery in the where part
  • 2825: "Insert into ... select distinct ..." fails with MALException:mkey.bulk_rotate_xor_hash:operation
  • 2826: sql function iterates instead of bulk processing
  • 2827: Incorrect filter setting when listening to several servers
  • 2829: logging issue
  • 2830: select 30.0 - 0.025; conversion to decimal type gives error
  • 2831: Crash in query analysis
  • 2832: Server terminates upon UPDATE
  • 2833: NullPointerException when calling getMoreResults() on Statement without a query executed
  • 2835: UNIQUE Constraint is not enforced
  • 2837: Server terminates after CREATE FUNCTION in multi-user scenario
  • 2844: ALTER USER SET PASSWORD doesn't recognise username

 

Apr 2011-SP1 bugfix release

MonetDB Common Fixed a bug where large files (> 2GB) didn't always get deleted on Windows.
Java Module Insertion via PreparedStatement and retrieval via ResultSet of timestamp and time fields with and without timezones was improved to better respect timezones, as partly indicated in bug #2781.
MonetDB5 Server
Fixed a bug in conversion from string to the URL type.  The bug was an incorrect call to free().
Bug Fixes
  • 2784: configure does not (properly) check for yacc/bison
  • 2798: Query did not produce a result set
  • 2806: Wrong logic in predicates over views defined with an open-interval predicate
  • 2807: wrong results for group by queries in the presence of primary keys
  • 2808: wrong results for count()/count(distinct) on the same attribute
  • 2810: Copy into should rollback correctly (i.e. restore the files) on a failure

 

Apr 2011 feature release

Client Package

Plugged a small memory leak occurring upon redirects by the server (e.g. via monetdbd)
Remove XQuery related code from Ruby adapter, PHP driver and Perl Mapi library
Make SQL the default language for mclient, e.g. to use when --language= or -l is omitted

MonetDB Common Fixed bugs in antiselect which gave the incorrect result when upper and lower bound were equal. This bug could be triggered by the SQL query SELECT * FROM t WHERE x NOT BETWEEN y AND y
Java Module Clarify exception messages for unsupported methods
Removed XQuery related XRPC wrapper and XML:DB code, removed support for language=xquery and language=mil from JDBC.
MonetDB5 Server
Mserver5 no longer reads monetdb5.conf upon startup by default. Use --config=file to have mserver5 read a configuration on startup
SQL Some names in the SQL catalog were changed. This means that the database in the Apr2011 release is not compatible with pre-Apr2011 databases. The database is converted automatically when opened the first time. This database can then no longer be read by an older release.
Merovingian Fix monetdb return code upon failure to start/stop a database.
The forward property for databases has been removed. Instead, only a global proxy or redirect mode can be set using monetdbd.
Monetdbd can no longer log error and normal messages to separate logfiles, logging to stdout and stderr is no longer possible either.
The .merovingian_pass file is no longer in use, and replaced by the .merovingian_properties file. Use monetdbd (get|set) passphrase to view/edit the control passphrase. Existing .merovingian_pass files will automatically be migrated upon startup of monetdbd.
Monetdbd now understands commands that allow to create, start, stop, get and set properties on a given dbfarm. This behaviour is intended as primary way to start a MonetDB Database Server, on a given location of choice. monetdbd get and set are the replacement of editing the monetdb5.conf file (which is no longer in use as of the Apr2011 release). See monetdbd(1).
Bug Fixes
  • 1956: Geom: relation and spatial tests fail
  • 2547: create schema leads to sql_stmt symbol(x)->token = CREATE_SCHEMA
  • 2588: test sql/src/test/BugTracker/Tests/authorization.SF-1430616.SQL.sh shows different error message
  • 2590: test sql/src/test/BugTracker/Tests/mdb_starts_with_sql_debug_64.SF-1999354.py fails with "too many digits"
  • 2639: Readline auto-complete reads from 'tables' instead of 'sys.tables'
  • 2766: SQL: Compute modulo of rand() in an UPDATE stmt crashes mserver5
  • 2779: Make building GDK optional
  • 2793: EXTRACT(SECOND FROM timestamp) always returns 0 integer seconds part.
  • 2795: SQL: ALTER TABLE with incorrect syntax crashes mserver5
  • 2796: latest antiselect changes cause ~20 SQL tests to fail
  • 2800: Test_PStypes fails due to ambiguous identifier
  • 2801: Nested SELECT expression with JOIN causes outer WHERE to be disregarded sometimes

 

 

 

March 2011

March 2011 mk Sun, 05/15/2011 - 12:00

MonetDB Mar2011 release

This is the last planned release for MonetDB4 and MonetDB/XQuery. MonetDB4 has served us well, but we need to move on. MonetDB/XQuery, being implemented on top of MonetDB4, is a casualty of this decision. The work on the Pathfinder compiler, which forms the basis of MonetDB/XQuery, continues at The University of Tübingen.

As of this "Mar2011" release (version 11.1.1), all components of MonetDB have been gathered into one single package, mainly to simplify the compilation process. If you're using the Mercurial clone, use the Mar2011 branch and the Mar2011_1 tag.

Contents

Mar 2011 feature release


Packaging

The packaging of MonetDB has been completely overhauled. In the past, there were various subpackages that had to be configured and built individually, but that had to be done in the correct order. From this release onward, the process of building MonetDB has been greatly simplified. There is only a single configuration/build/install cycle for the whole suite. On Linux/Unix systems this means one call to configure (when building from the Mercurial sources this has to be preceded by one call to the bootstrap script), one call to make, and one call to make install. On Windows, the bootstrap and configure stages can be skipped (and you need to use nmake instead of make).

The directory structure of the source tree was also completely overhauled. The MonetDB4 and MonetDB5 directories were renamed monetdb4 and monetdb5 respectively, the MonetDB directory was renamed gdk, src subdirectories were removed, and several subsystems were moved to the new common directory.

The version numbers have changed since we now have a single package. The version number of this release is 11.1.1.

Linux

We now package for Fedora 13 and 14, for Debian 5.0 (Lenny) and 6.0 (Squeeze), and for Ubuntu 9.10 (Karmic Koala), 10.04 (Lucid Lynx), and 10.10 (Maverick Meerkat).

On Debian and Ubuntu, updating to this release is a bit more involved than simply using apt-get upgrade or the "Mark All Upgrades" button in the Synaptic Package Manager. When using these commands, they will "keep back" the monetdb packages, so in order to actually update them you will need to give the apt-get install command and specify the relevant monetdb packages, or in Synaptic first select the relevant monetdb packages and then select Package->Mark for Upgrade.

Windows

Binaries are available.

Mac OS X

Binaries are available.

Solaris

Binaries are available.


MonetDB Common

  • Fixed memory detection on Darwin (Mac OS X) systems not to return bogus values
  • MonetDB/src/gdk was moved to gdk


MonetDB Server 4


MonetDB Server 5

  • sphinx module: update, adding limit/max_results support
  • Improve the performance of remote.put for BAT arguments. The put speed is now roughly equal to the speed of get on a BAT.


SQL

  • Fix bug #2648, do not allow restarting a sequence with NULL via the result of a sub-query.
  • Allow clients to release prepared handles using Xrelease commands


XQuery


Client Packages

  • Fixed bug 2677: SQL_DESC_OCTET_LENGTH should give the size in bytes required to copy the data.
  • Disable timer functionality for non-XQuery languages since it is incorrect, bug #2705
  • Added mapi_get_uri function to retrieve mapi URI for the connection
  • Allow to dump table data using INSERT INTO statements, rather than COPY INTO + CSV data using the -N/--inserts flag of mclient and msqldump. Bug #2727
  • Added support for \dn to list schemas or describe a specific one
  • Added support for \df to list functions or describe a specific one
  • Added support for \ds to list sequences or describe a specific one
  • Added support for wildcards * and ? in object names given to \d commands, such that pattern matching is possible, e.g. \d my*
  • Added support for \dS that lists also system tables
  • object names given to \d are now lowercased, unless quoted by either single or double quotes
  • Strip any trailing whitespace with the \d command


Geom Module

  • Set endianness for wkb en/decoding.


Java Module

  • PreparedStatements now free the server-side resources attached to them when closed. This implements bug #2720


Bug fixes

The following bug tracker items were fixed since the previous "Oct 2010" release:

  • 104: SQL: extremely long and complex query causes SEGFAULT
  • 2351: SQL: transaction aborts on prepared statements in autocommit
  • 2358: Unrelevant error message when copying bad length field
  • 2586: times.sql: procedure times() crashes the server
  • 2593: test sql/src/test/Tests/load_dec_as_int.sql.src fails a file sql/src/test/load_dec_as_int.sql is missing
  • 2609: insert into large number of columns
  • 2634: !ERROR: CTrefine: both BATs must have the same cardinality ...
  • 2640: VOC test fails multi-threaded
  • 2647: M5 breaks with huge SQL query.
  • 2648: ALTER SEQUENCE ... RESTART WITH (SELECT NULL) NO CYCLE is accepted
  • 2673: select a string column from a view over a table with primary key crashes
  • 2677: SQLColAttribute using SQL_DESC_OCTET_LENGTH doesn't return the maximum length
  • 2679: timestamp selection based on now() takes too long
  • 2680: make clean too eager
  • 2698: test sql/src/test/bugs/Tests/except-union-intersect-bug-sf-1146079.SQL.sh produces wrong result with multi-threaded execution
  • 2704: temporary tables on readonly mode
  • 2705: Timer information problems
  • 2708: function with different value for arguments crashes mserver
  • 2710: implement \d{t,v,s,S,...} to describe other (or only) objects in the database
  • 2713: wrong error message when dropping an un-existing view with the same as an existent table.
  • 2715: Join with 2 AND-ed conditions where one is LIKE/ILIKE returns too many results
  • 2717: function call in readonly mode.
  • 2720: Allow a client to close a prepared handle
  • 2725: mclient/mapilib aborts during connect
  • 2726: join on two literal tables leads to
  • 2727: allow dumping database with INSERT INTO statements
  • 2729: mserver5: ../../../src/server/rel_bin.mx:537: exp_bin: Assertion `0' failed.
  • 2730: select * from sequence crashes mserver5
  • 2733: MAL parser accepts 1 + 1 + 1, MAL interpreter only computes 1 + 1
  • 2735: complex SQL query leads to multiple TypeExceptions
  • 2737: Sphinx module update, adding limit/max_results support
  • 2738: MonetDB 'allows' binding and connecting to ports above 65535 (has patch)
  • 2743: mapi_next_result skips over results (losing output)
  • 2744: UNLOCKED causes mserver to work indefinitely
  • 2745: foreign key type mismatch causes MAL stacktrace
  • 2748: COPY INTO does not work correctly with backslashes in VARCHARs
  • 2750: Function arguments of type string described not completely in SQL catalog
  • 2752: CALL function without warning
  • 2753: check existence of external names for SQL functions/procedures at creation time
  • 2757: ORDER BY DESC + LIMIT incorrectly yields empty result
  • 2768: "no such column 'u.name'" in users system view
  • 2773: starting database 'test'... FAILED
  • 2786: crash when first connection is with another user than monetdb
  • 2788: stop -a complains about already stopped databases
  • 2790: View returns erroneous result for squared average function as compared to query

The detailed list of changes can be found in the source code Mercurial logs.

June 2010

June 2010 sjoerd Thu, 07/01/2010 - 16:54

MonetDB Jun2010 release

Release "Jun2010" consists of the following updated components. If you're using the Mercurial clone, use the Jun2010 branch and the Jun2010_1 tag.

 

Name Version Subdirectory Note/Comment
MonetDB Common 1.38.1 MonetDB
MonetDB Clients 1.38.1 clients
MonetDB4 Server 4.38.1 MonetDB4
MonetDB4/XQuery 0.38.1 pathfinder
MonetDB5 Server 5.20.1 MonetDB5
MonetDB5/SQL 2.38.1 sql
MonetDB5/SQL/GIS 0.18.1 geom
MonetDB Java 1.38.1 java
MonetDB Testing 1.38.1 testing (developers only)

NOTE:
Before upgrading to the latest "Jun2010" release from a release of MonetDB older than the Nov2008 release, please make a dump (SQL, XQuery) of your database, and rename/move/backup the database directories (dbfarm, sql_logs, xquery_logs). After the upgrade, restore (SQL, XQuery) your database, again. Once you have checked and confirmed the correctness of the restore, you can delete the above renamed/moved/backed-up old database directories.

When upgrading from a release of MonetDB older than the May2009 release, you need to move your write-ahead log following the "recommended" instructions for SQL and/or XQuery.

Contents

Jun 2010 feature release

Packaging

 

Linux

We now package for Fedora 11, 12, and 13, for Debian 5.0 (Lenny), and for Ubuntu 9.10 (Karmic Koala) and 10.04 (Lucid Lynx).

MonetDB Common

 

  • Updated Vendor information.
  • Made compilation of "testing" (and "java") independent of MonetDB. This is mainly for Windows, but also on other systems, "testing" can now be built independently of (and hence before) "MonetDB". Files that mimic configure functionality on Windows were moved from "MonetDB" to "buildtools"; hence, this affects all packages on Windows, requiring a complete rebuild from scratch on Windows. getopt() support in testing has changed; hence, (most probably) requiring a rebuild from scratch of testing on other systems.
  • Implemented build directory support for Windows, i.e., like on Unix/Linux also on Windows we can now build in a separate build directory as alternative to ...<package>NT, and thus keep the latter clean from files generated during the build. On Windows, the build directory must be a sibling of ...<package>NT .

MonetDB Server 4

 

  • Updated Vendor information.
  • Made compilation of "testing" (and "java") independent of MonetDB. This is mainly for Windows, but also on other systems, "testing" can now be built independently of (and hence before) "MonetDB". Files that mimic configure functionality on Windows were moved from "MonetDB" to "buildtools"; hence, this affects all packages on Windows, requiring a complete rebuild from scratch on Windows. getopt() support in testing has changed; hence, (most probably) requiring a rebuild from scratch of testing on other systems.
  • Implemented build directory support for Windows, i.e., like on Unix/Linux also on Windows we can now build in a separate build directory as alternative to ...<package>NT, and thus keep the latter clean from files generated during the build. On Windows, the build directory must be a sibling of ...<package>NT .

MonetDB Server 5

 

  • When libxml2 is available, the XML module is automatically loaded.
  • Updated Vendor information.
  • Created a new RPM MonetDB5-server-rdf for the optional MonetDB/RDF module.
  • Added the Z-order module to simplify future manipulation of arrays.
  • fixed BUG #2994521 "mat.slice unable to cope with only empty BAT arguments" by making MATpackSliceInternal() handle empty input BATs correctly
  • Select <col> from <t> limit <n> has been improved by introducing mat.slice().
  • Made compilation of "testing" (and "java") independent of MonetDB. This is mainly for Windows, but also on other systems, "testing" can now be built independently of (and hence before) "MonetDB". Files that mimic configure functionality on Windows were moved from "MonetDB" to "buildtools"; hence, this affects all packages on Windows, requiring a complete rebuild from scratch on Windows. getopt() support in testing has changed; hence, (most probably) requiring a rebuild from scratch of testing on other systems.
  • Implemented build directory support for Windows, i.e., like on Unix/Linux also on Windows we can now build in a separate build directory as alternative to ...<package>NT, and thus keep the latter clean from files generated during the build. On Windows, the build directory must be a sibling of ...<package>NT .
  • The MAL debugger list command has been extended with an optional hash '#', which produces line numbers for each of reference and analysis of variable span.
  • The dataflow scheduler has been revamped to allow for more parallelism to be exploited.
  • The garbage collection administration has been changed. Every variable record now comes with an end-of-life field (eolife), which denotes the instruction after which the BAT variable reference counter can be decremented. The garbage collector is never called automagically on MAL functions, because there may be other pressing needs to retain them. This means that MAL functions defined and used in the context of SQL, and which are not inlined, may cause a leakage. The garbage collection has become part of each interpreter step. The new approach makes the SQL/MAL plans half the size as before.
  • Removed stethoscope from MonetDB5 sources. New location is in the clients repository.
  • Added the compression optimizer as an example of how to gain access to foreign file formats deep down in the kernel and transfer them just in time into a temporary BAT.
  • Renamed configure argument --with-console to --enable-console. Default remains console being enabled.
  • Fixed cleaning the user module context upon session end. Fixes bug #2956664
  • The config variable mapi_usock can now be used to instruct the server to listen for connections on a local UNIX domain socket on UNIX-like systems.
  • The MAL interpreter has been extended with an operation admission policy to control the memory claims of all concurrent running interpreters. Instructions are hold up unto there is sufficient resource or the query plan can not avoid its execution anymore.
  • The join path optimizer has been extended with searching for join*, semijoin*, and leftjoin* paths. Furthermore, it avoids duplicate work by factoring out all common simple join paths.

SQL

 

  • make it possible to use '?' in offset and limit
  • Added include and .lib files to Windows installers that are needed to compile client programs.
  • Updated Vendor information.
  • Made compilation of "testing" (and "java") independent of MonetDB. This is mainly for Windows, but also on other systems, "testing" can now be built independently of (and hence before) "MonetDB". Files that mimic configure functionality on Windows were moved from "MonetDB" to "buildtools"; hence, this affects all packages on Windows, requiring a complete rebuild from scratch on Windows. getopt() support in testing has changed; hence, (most probably) requiring a rebuild from scratch of testing on other systems.
  • Implemented build directory support for Windows, i.e., like on Unix/Linux also on Windows we can now build in a separate build directory as alternative to ...<package>NT, and thus keep the latter clean from files generated during the build. On Windows, the build directory must be a sibling of ...<package>NT .

XQuery

 

  • Updated Vendor information.
  • Made compilation of "testing" (and "java") independent of MonetDB. This is mainly for Windows, but also on other systems, "testing" can now be built independently of (and hence before) "MonetDB". Files that mimic configure functionality on Windows were moved from "MonetDB" to "buildtools"; hence, this affects all packages on Windows, requiring a complete rebuild from scratch on Windows. getopt() support in testing has changed; hence, (most probably) requiring a rebuild from scratch of testing on other systems.
  • Implemented build directory support for Windows, i.e., like on Unix/Linux also on Windows we can now build in a separate build directory as alternative to ...<package>NT, and thus keep the latter clean from files generated during the build. On Windows, the build directory must be a sibling of ...<package>NT .

Client Packages

 

  • Updated Vendor information.
  • The MonetDB ODBC driver now no longer depends on its own copy of the standard ODBC include files but instead depends on system include files (unixODBC on Linux and Microsoft SDK on Windows).
  • Made compilation of "testing" (and "java") independent of MonetDB. This is mainly for Windows, but also on other systems, "testing" can now be built independently of (and hence before) "MonetDB". Files that mimic configure functionality on Windows were moved from "MonetDB" to "buildtools"; hence, this affects all packages on Windows, requiring a complete rebuild from scratch on Windows. getopt() support in testing has changed; hence, (most probably) requiring a rebuild from scratch of testing on other systems.
  • Implemented build directory support for Windows, i.e., like on Unix/Linux also on Windows we can now build in a separate build directory as alternative to ...<package>NT, and thus keep the latter clean from files generated during the build. On Windows, the build directory must be a sibling of ...<package>NT .
  • Changed d output of mclient no longer to list (internal use only) system tables. Administrator users who like to inspect these tables can use SELECT * FROM tables; instead
  • Implemented the changes required for properly running ODBC on a 64-bit platform.
  • Added stethoscope, an utility to profile MonetDB5 instances, originally from the MonetDB5 repository.

Geom Module

 

  • Updated Vendor information.
  • Made compilation of "testing" (and "java") independent of MonetDB. This is mainly for Windows, but also on other systems, "testing" can now be built independently of (and hence before) "MonetDB". Files that mimic configure functionality on Windows were moved from "MonetDB" to "buildtools"; hence, this affects all packages on Windows, requiring a complete rebuild from scratch on Windows. getopt() support in testing has changed; hence, (most probably) requiring a rebuild from scratch of testing on other systems.
  • Implemented build directory support for Windows, i.e., like on Unix/Linux also on Windows we can now build in a separate build directory as alternative to ...<package>NT, and thus keep the latter clean from files generated during the build. On Windows, the build directory must be a sibling of ...<package>NT .

February 2010

February 2010 mk Wed, 03/24/2010 - 11:27

Release "Feb2010" consists of the following updated components

Name Version CVS module CVS branch Note/Comment
MonetDB Common 1.36.1 MonetDB Feb2010
MonetDB Clients 1.36.1 clients Feb2010
MonetDB4 Server 4.36.1 MonetDB4 Feb2010
MonetDB4/XQuery 0.36.1 pathfinder Feb2010
MonetDB5 Server 5.18.1 MonetDB5 Feb2010
MonetDB5/SQL 2.36.1 sql Feb2010
MonetDB5/SQL/GIS 0.16.1 geom Feb2010
MonetDB Java 1.36.1 java Feb2010
MonetDB Testing 1.36.1 testing Feb2010 (developers only)

NOTE:
Before upgrading to the latest "Feb2010" release from a release of MonetDB older than the Nov2008 release, please make a dump (SQL, XQuery) of your database, and rename/move/backup the database directories (dbfarm, sql_logs, xquery_logs). After the upgrade, restore (SQL, XQuery) your database, again. Once you have checked and confirmed the correctness of the restore, you can delete the above renamed/moved/backed-up old database directories.

When upgrading from a release of MonetDB older than the May2009 release, you need to move your write-ahead log following the "recommended" instructions for SQL and/or XQuery.

Contents

Feb 2010 feature release


Packaging

We now package for Fedora 11 and 12.


MonetDB Common

  • Various concurrency bugs were fixed.
  • Various changes were made to run better on systems that don't have enough memory to keep everything in core that was touched during query processing. This is done by having the higher layers giving hints to the database kernel about future use, and the database kernel giving hings to the operating system kernel about how (virtual) memory is going to be used.
  • Fixed bug in mergejoin implementation. This fixes bug #2952191.
  • Added support for compiling on Windows using the Cygwin-provided version of flex.
  • Fix compilation issue when configured with --with-curl. This fixes bug #2924999.
  • Added implementation of MT_getrss() for Solaris. This yields in the kernel knowing about its (approximate) memory usage to try and help the operating system to free that memory that is best to free, instead of a random page, e.g. the work of the vmtrim thread.
  • Implemented a "fast" string BAT append: Under certain conditions, instead of inserting values one-by-one, we now concatenate the string heap wholesale and just manipulate the offsets. This works both for BATins and BATappend.
  • Changed the string heap implementation to also contain the hashes of strings.
  • Changed the implementation of the string offset columns to be dynamically sized.


MonetDB Server 5

  • The MonetDB 5 code base underwent a series of small/medium changes:
    • Solving name conflicts with external libraries in mal_box.
    • Dependency graph generation in MAL debugger fixed.
    • Code hardening against out of memory errors and based on Coverity checks.
    • Recycler improved to deal with SQL plans from different sessions.
    • Profiler extended to report the argument types, user, and thread id.
    • MAL interpreter reports an event before and after the instruction.
    • Dataflow also allowed for updates on temporary BATs.
    • Reorder optimizer better respects the dataflow.
    • All update instructions return their target to mark the dataflow.
    • Clean up of (bat)calc module.
    • Packing pieces together simplified and sped up.
  • The MAL interpreter has been extended with an operator admission scheme, which is active during parallel execution. It blocks threads if the total amount of memory needed for the operator can not be claimed. Only if there is one operation left to execute, it won't block. The admission level is controlled by a threshold, which is set to 90% of the physical memory.
  • The ilike[u]select operations has been included in the repertoire recognized by the mergetable optimizer for push through of selections.
  • The joinpath optimizer has been extended with recognition of the leftjoin. This way, series of leftjoin operations can be optimized by looking at the smallest starting point.
  • Introduced a dictionary encoding option to the optimizer pipeline. Encoding is initiated with a SQL call compress(tablename).
  • Removed the ability to redirect to other running databases in the same dbfarm. This functionality has been taken over completely by merovingian, and only results in confusement these days, bug #2891191.
  • Added --enable-console configure argument, defaulting to 'yes' for now. Disabling the server console increases security by avoiding local access exploits. This is not the default since our Testing setup cannot deal with a console-less server yet.
  • Report detected amount of main memory and cpu cores in output of `mserver5 --version`.


SQL

Nothing to report.


XQuery

Nothing to report


Client Packages

  • Fixed a bug in the Mapi libary when sending very large queries to the server.
  • Implemented BLOB and CLOB support in ODBC driver.
  • Fixed crash upon connect to a server that is under maintenance but has the requested language (scenario) not loaded.
  • Fixed bug in the ODBC driver where a non-ASCII character at the end of a string caused an error when the string was converted to wide characters.
  • mclient timer information (-t option) is now written to standard error instead of standard output. This makes getting timer information easier when there is a large amout of regular output (which can be redirected to a file or /dev/null).
  • Implemented new function mapi_fetch_field_len() which returns the length (excluding trailing NULL byte) of the field returned by mapi_fetch_field().


Bug fixes

The following bug tracker items were fixed since the previous "Nov 2009" release: The detailed list of changes can be found in the source code CVS logs.

November 2009

November 2009 mk Tue, 03/30/2010 - 20:27

MonetDB Nov2009-SP2 release

Release "Nov2009-SP2" consists of the following updated components

Name Version CVS module CVS branch Note/Comment
MonetDB Common 1.34.4 MonetDB Nov2009
MonetDB Clients 1.34.4 clients Nov2009
MonetDB4 Server 4.34.4 MonetDB4 Nov2009
MonetDB4/XQuery 0.34.4 pathfinder Nov2009
MonetDB5 Server 5.16.4 MonetDB5 Nov2009
MonetDB5/SQL 2.34.4 sql Nov2009
MonetDB5/SQL/GIS 0.14.4 geom Nov2009
MonetDB Java 1.34.4 java Nov2009
MonetDB Testing 1.34.4 testing Nov2009 (developers only)

NOTE:
Before upgrading to the latest "Nov2009-SP2" release from a release of MonetDB older than the Nov2008 release, please make a dump (SQL, XQuery) of your database, and rename/move/backup the database directories (dbfarm, sql_logs, xquery_logs). After the upgrade, restore (SQL, XQuery) your database, again. Once you have checked and confirmed the correctness of the restore, you can delete the above renamed/moved/backed-up old database directories.

When upgrading from a release of MonetDB older than the May2009 release, you need to move your write-ahead log following the "recommended" instructions for SQL and/or XQuery.

Contents


Nov 2009 SP2 bug-fix release

This is the second bug fix release of the Nov2009 version. This version can be installed "on top of" the Nov2009 release. The database format is fully compatible, so there is no need to do a dump and restore. (Although it is *always* a good idea to make backups.)

Some of the bugs that were fixed:

  • No need to require bash for running the *-config scripts. (SF bug #2914563)
  • MonetDB5 development package now contains all development include files. (SF RFE #2929558)
  • Added support for compiling on OpenBSD (contributed, i.e. not tested by us).
  • General code improvements, partly inspired by running Coverity Scan.
  • mclient -D will exit with a non-zero exit code if dumping failed. (SF bug #2925674)
  • The Python interface now uses the same socket options as the C version, giving a boost in performance. (SF bug #2925750)
  • Fixed a bug in the Python interface to deal with fields with embedded newline characters. (SF bug #2917219)
  • Added initial support to the Python client interface for specifying the encoding.
  • Fixed a bug in execute argument parsing in Python interface.
  • Added missing .sql files to the Windows installer.
  • In MonetDB/SQL, fixed bug in not(invalidtable). (SF bug #2927174)

The following bug tracker items were fixed since the Nov2009-SP1 release:

  • [ 2911008 ] Cannot start because lib_replication.dll was not found
  • [ 2914558 ] Modify MonetDB.m4 to support OpenBSD
  • [ 2917219 ] Python client doesn't handle fields with newline characters
  • [ 2925674 ] mclient dump fails on UTF-8 char and gives no error
  • [ 2925750 ] setsockopt in Python client greatly improves performace
  • [ 2927174 ] SQL: not(invalidtable.bla) segfaults


Nov 2009 SP1 bug-fix release

This is a bug fix release of the Nov2009 version. This version can be installed "on top of" the Nov2009 release. The database format is fully compatible, so there is no need to do a dump and restore. (Although it is *always* a good idea to make backups.)

Some of the bugs that were fixed:

  • The Windows installers for MonetDB/SQL in the Nov2009 release were incomplete. The missing files have been added.
  • Fixed performance issue with loading complex schemas in readonly mode in MonetDB/SQL.
  • Fixed performance issue with grouping in MonetDB/SQL when one of the columns is sorted.
  • Fixed problems with LIKE joins in MonetDB/SQL.
  • Load all files in a multi-file COPY INTO query.
  • Fix error reporting of unexpected input in XQuery queries to be Unicode aware.
  • Correctly handle circular dependencies in FD property check. (SF bug #2908615.)
  • Fixes throughout the code base to better cope with unexpected situations (such as memory shortage).

The following bug tracker items were fixed since the Nov2009 release:

  • [ 1681244 ] Mapi ports and patfhinder/pftijah
  • [ 1893448 ] PF: xrpcConvert2 fails on Windows
  • [ 2686008 ] SQL: multi-attribute GROUP BY may ignore sortedness
  • [ 2879008 ] SQL: WHERE A.x LIKE A.y misses results
  • [ 2879011 ] SQL: leftjoin undefined
  • [ 2884050 ] Broken link to VOC tutorial
  • [ 2902320 ] SQL: COPY multiple files
  • [ 2903111 ] Wrong error / no check for new database dir
  • [ 2908615 ] Not clause breaks result


Nov 2009 feature release

Ubuntu

While the Ubuntu version of the Aug2009 release was built for Ubuntu 9.04, the Jaunty Jackalope, the Ubuntu version of the Nov2009 release has now been built for Ubuntu 9.10, the Karmic Koala.


Packaging


Fedora

In addition to Fedora 10 & 11, we now also provide RPMs for Fedora 12.

Python

Packaging of some Python support files which are only used for testing was changed. The MonetDB-python RPM has been removed, the files are now packaged in the MonetDB-testing RPM. Equally, the python-monetdb Debian/Ubuntu package has been removed.

Windows

On Windows, the bzip2 library is now also used. The server (e.g., using SQL's COPY INTO) can read files that are compressed using gzip and bzip2.


MonetDB Common

The Coverity service has been used to uncover potential errors, such as dereference through NULL pointers and leakage. The warnings produced in the first run have been dealt with for >90%.


MonetDB Server 5

 

  • The M5 code underwent code hardening to better deal with rare events that would materialize mostly on 32bit systems. Mostly minor changes, e.g. display of debugging output has been improved to be more informative.


SQL

 

  • There is a new implementation of the SQL COPY INTO statement which uses multiple CPUs/cores when available. This version is a bit stricter when parsing the CSV file than the old version. E.g., when strings are quoted in the CSV file, the string delimiter must be specified in the COPY INTO statement, and to let empty field be treated as NULL values, the COPY INTO statement must specify NULL AS ''.
  • The TRACE statement modifier has been changed to collect the events in BATs first and returning them after the result set. The support library has been adjusted accordingly. See the documentation: SQL runtime features.
  • Several experimental optimizer pipelines are researched and being worked upon. They are not (yet) meant as the default pipeline for production applications. See the documentation: SQL runtime features.
  • Changing usernames is now possible from SQL, including the monetdb adminstrator user. Feature Request #2830257. The SQL syntax is: ALTER USER "monetdb" RENAME TO "newname";.
Merovingian

The following features were implemented in the merovingian and monetdb programs:

  • monetdb: Created databases are now always put under maintenance and as such they have to be explicitly released. This allows the DBA to initialise the new database without any client interference.
  • merovingian: Neighbour discovery now no longer announces databases under maintenance, such that they are not shown as being available to peers. This allows to pull a database out of the cluster easily.
  • monetdb: No operations can be performed with monetdb any more without a merovingian instance running.
  • merovingian: Implemented Feature Request #2830754. monetdb can now control a remote merovingian using the -h -p and -P global options to monetdb. The password used is stored at the merovingian site in the .merovingian_pass file in the controlled dbfarm. An example of how this can be used can be found in the Assets section under the VOC loader script.
  • merovingian: Remote controllable merovingians now announce their controlport to peers, which are marked with an asterisk in the output of monetdb discover.
  • monetdb: Implemented Feature Request #2855846. The global -q option to monetdb makes it suppress any status messages, causing only error messages to be printed when they occurr.
  • merovingian: The outside URI that Merovingian is available on is now announced to each mserver5 started by merovingian via the merovingian_uri GDK environment variable.
  • monetdb: For each database now the property optpipe can be set which allows per database control of the SQL optimiser pipeline. Valid values can be found in the monetdb5.conf configuration file under the sql_optimizer section.
  • monetdb: Fixed bug where the uptime of databases with an uptime longer than 7 days was reported incorrectly.


XQuery

 

  • Use of value indices on comparisions between a value and an XPath ending in text() or attribute access (e.g., doc(x)//x/y[z/text() = "foo"] and for $x in doc//x/y where $x/@z = "bar" return $x); such queries do not need scanning data, but start at the index and navigate backwards from there.

    NOTE: This required a change in the physical database storage. While the Nov2009 release of MonetDB/XQuery can still read older (pre-Nov2009) databases, it is recommended to re-shred all documents to benefit from the new indices and achieve optimal performance.

  • A session expression cache, where you can run multiple queries in the same session and within that session keep intermediate results present (somewhat like a database cursor). One use is quickly paging in a GUI through a large result, another is acceleration of series of subsequent queries that overlap (e.g. zoom in to a result set). See: Session Expression Cache


Client Packages


mclient

 

  • Result rendering has been changed to try harder to find an optimal rendering of results wider than the terminal allows. Unlike previous releases, mclient now emits results wider than the terminal width if it is impossible or unreasonable to fit the results, thereby aiming to avoid wrapping columns.
  • EXPLAIN, TRACE and PROFILE output are now correctly rendered.
  • Timer support for single statements was implemented, Bug #2872884.
  • Added support for .monetdb file with defaults for mclient. The DOTMONETDBFILE environment file can be used to point to a different location or file name. The file layout of a .monetdb file is a key=value file, where keys include user, password and language.
  • Data fed into mclient using a pipe is no longer read line by line, but per block, like files given as arguments are. This improves performance considerably, but changes the behaviour regarding error messages and understood commands. The internal \x commands of mclient are no longer respected when used with a pipe. To obtain the previous behaviour, add the -i option to mclient when using data fed from a pipe.
  • The default username (the name of the currently logged in user) is no longer used without asking. The -u option now always requires an argument, when this option is absent, mclient will ask for a username, unless a default was set in the .monetdb file. Note: mclient no longer logs in as monetdb/monetdb by default.
  • The start_with_help option was removed. --help now always gives the general help of mclient and exits afterwards. Instead, mclient now gives a welcome message with some diagnostic information and a hint how to get (language specific) help.
  • The \d command now describes all tables by reporting whether it is a TABLE or VIEW. Rendering mode of database object describe mode has been improved.
  • Autocommit status and changes therein are now reported by mclient. Empty responses are no longer reported as "0 tuples", instead proper status messages are returned, giving e.g. information about a generated identity key.
  • Error messages no longer include the leading !.
Python & Ruby

 

  • The native implementations have been matured further.
PHP

 

  • The SWIG-based PHP interface (leveraging the C MAPI library) has been replaced by a native PHP implementation. For the time being, the old SWIG-based interface is still in the code base, but only the new native interface is activated and distributed in the installers.
JDBC

 

  • JDBC received a workaround for a longstanding problem when running the driver using Java 6.
  • A bug was fixed which caused Connection warning messages to disappear when a connection to Merovingian was made.


Bug fixes

The following bug tracker items were fixed since the previous "Aug 2009 SP2" release:

  • ID: 2903111: Wrong error / no check for new database dir
  • ID: 2902320: SQL: COPY multiple files
  • ID: 2898944: pf takes 'forever' (> 1 hour) to compile particular query
  • ID: 2898378: M4: tests/by_Peter fail
  • ID: 2897126: pf fatal error: duplicate column in equi-join
  • ID: 2896012: Filter using same field in OR and AND throws typeexception
  • ID: 2895791: ORDER BY with Row_number() return wrong results
  • ID: 2895290: M5: segfault for tst019 when run with mclient
  • ID: 2894927: Mx: @include does not write any "#line" directives
  • ID: 2894150: test fail due to error message format/protocol changes
  • ID: 2893878: SQL: more tests fail with Nov2009 that worked with Aug2009
  • ID: 2893875: PF: 82 tests fail on Auf2009_NFI/Nov2009 but work on Aug2009
  • ID: 2893859: SQL: various VIA-M tests fail
  • ID: 2893846: SQL: set_history_and_drop_table.SF-2607045 fails
  • ID: 2893823: SQL: src/backends/monet5/Tests/optimizers.sql fails
  • ID: 2893821: SQL: src/test//Tests/median_stdev.sql fails
  • ID: 2893815: PF: xrpcConvert tests segfault since 2009/10/08
  • ID: 2893484: All CASE branches are evaluated
  • ID: 2893247: SQL: QOTupdateStatistics segfaults
  • ID: 2891718: M5/MAL: io.print() yields no output
  • ID: 2891191: unconscious redirect
  • ID: 2890914: StartOfHeaderParser.java#getNextAsString() Err: IndexOutOfB
  • ID: 2890702: M5: pma.mx fails to compile on various platforms
  • ID: 2890035: SQL: optimizer confused by concurrency
  • ID: 2887282: union bug
  • ID: 2879011: SQL: leftjoin undefined
  • ID: 2879008: SQL: WHERE A.x LIKE A.y misses results
  • ID: 2879005: SQL: duplicate column name
  • ID: 2878994: SQL: WHERE A.x LIKE B.x does not work
  • ID: 2877044: PF: binding sequence in a for clause has static type `empty'
  • ID: 2876948: PF: error "We do not support the value NaN"
  • ID: 2874045: SQL: segv for unknown identifier in subquery
  • ID: 2873564: Cannot create unique constraint on char column
  • ID: 2872884: mclient -t -s... does not show "Timer" info
  • ID: 2864179: function call not handled
  • ID: 2863458: PF/TIJAH: tokenize in a for-loop
  • ID: 2837050: SQL: src/test/xquery/Tests/q01.sql fails
  • ID: 2834654: PHP+XQUERY: multiple results not returned correctly
  • ID: 2817414: XQ: MonetDB allocates >5GB mem while querying small document
  • ID: 2791361: SQL: Division by zero error
  • ID: 2791356: SQL: Floating point exception
  • ID: 2771052: M5/SQL: parallel bulk-load (copy into) incorrect
  • ID: 2716723: PF: child-steps + replace() fails to compile
  • ID: 2686008: SQL: multi-attribute GROUP BY may ignore sortedness
  • ID: 2627137: PF: tests fail with BATpropcheck error
  • ID: 2211565: PF: type disjointness incorrect
  • ID: 2015135: PF: pf:mil() can't deal with nil values in MIL results
  • ID: 1991738: XQ: compilation or runtime choice
  • ID: 1981735: PF/alg: some error messages differ from PF/mps
  • ID: 1981735: PF/alg: some error messages differ from PF/mps
  • ID: 1976341: XQ: leftovers after deleting document
  • ID: 1964365: PF/Alg: insert..into inserts in wrong order
  • ID: 1959269: XQ+JDBC: protocol violation
  • ID: 1911209: XQ: MonetDB 'hangs' after requesting non-existing doc
  • ID: 1893448: PF: xrpcConvert2 fails on Windows
  • ID: 1809586: PF: busy port crashes Mserver
  • ID: 1747068: XQ: windows de-install should warn and empty bat/ dirs
  • ID: 1739353: EXPLAIN support from JDBC
  • ID: 1730547: XQ: Mserver crashes on concatenated query
  • ID: 1726599: XQuery: collection management broken
  • ID: 1681244: Mapi ports and patfhinder/pftijah
  • ID: 1637867: XQ: seqcast int? to numeric unsupported
  • ID: 1607210: XQ: server-side compilation crash (member benchmark)
  • ID: 1595129: XQ: property error on 64-bit Darwin - big-endian related?
  • ID: 987304: SQL: div by zero results in ascii art
  • ID: 921173: SQL: extremely long and complex query causes SEGFAULT

The detailed list of changes can be found in the source code CVS logs.

 

Augustus 2009

Augustus 2009 mk Tue, 03/30/2010 - 20:30

MonetDB Aug2009-SP2 release

Release "Aug2009-SP2" consists of the following updated components

Name Version CVS module CVS branch Note/Comment
MonetDB Common 1.32.4 MonetDB Aug2009
MonetDB Clients 1.32.4 clients Aug2009
MonetDB4 Server 4.32.4 MonetDB4 Aug2009
MonetDB4/XQuery 0.32.4 pathfinder Aug2009
MonetDB5 Server 5.14.4 MonetDB5 Aug2009
MonetDB5/SQL 2.32.4 sql Aug2009
MonetDB5/SQL/GIS 0.12.4 geom Aug2009
MonetDB Java 1.32.4 java Aug2009
MonetDB Testing 1.32.4 testing Aug2009 (developers only)

NOTE:
Before upgrading to the latest "Aug2009-SP2" release from a release of MonetDB older than the Nov2008 release, please make a dump (SQL, XQuery) of your database, and rename/move/backup the database directories (dbfarm, sql_logs, xquery_logs). After the upgrade, restore (SQL, XQuery) your database, again. Once you have checked and confirmed the correctness of the restore, you can delete the above renamed/moved/backed-up old database directories.

When upgrading from a release of MonetDB older than the May2009 release, you need to move your write-ahead log following the "recommended" instructions for SQL and/or XQuery.

Contents

Aug 2009 SP2 bug-fix release

This is the second bug fix release of the Aug2009 version. This version can be installed "on top of" the Aug2009 release. The database format is fully compatible, so there is no need to do a dump and restore. (Although it is *always* a good idea to make backups.)

Some of the bugs that were fixed:

  • Changes in the memory mapping code to less zealously allocate space.
  • Changes to the use of the way the operating system is informed about how memory is used, resulting in a better fit between the predictions the operating system makes and the reality of what MonetDB actually does.
  • Fix to exit code.
  • Performance fix releated to anonymous virtual memory.
  • Performance fix for when a single item is joined to a table. (MonetDB4/XQuery)
  • When extracting fails, don't leave a partial output file (Mx, SF bug 2840225).
  • Added support for GCC version 4.4.* by disabling the strict-aiasing optimization in the compiler.
  • Various improvements to the Ruby interface.
  • Fix warning messages to disappear when a connection to merovingian is made.
  • Fix physical plan for merge-adjacent-text-nodes. (MonetDB/XQuery)
  • For some plans with aggregates, lax checking resulted in incorrect rewrites. (MonetDB/XQuery)
  • Fix bug in order lists merging. In case the replaced column is sorted in descending order we need to reverse the order of all replacement order criteria. (MonetDB/XQuery)
  • Fixes so that larger plans can be handled. (MonetDB/XQuery, SF bug 2860037)
  • Fixes so that less stack space is used during processing, and so that running out of stack space is recognized in more places. (MonetDB/SQL)
  • Fixed bug in IN handling. (MonetDB/SQL, SF bug 2869818)
  • Fixed crash when attempting to create a function in read-only mode by disallowing this. (MonetDB/SQL, SF bug 2872421)
  • Added partial support for compiling with bison++. Bison++ is not supported and can't be used out of the box, but only minor tweaks are needed to use it. (SF bug 2872565)
  • Fix warning messages to disappear when a connection to merovingian is made. (JDBC driver)

The following bug tracker items were fixed since the Aug2009-SP1 release:

  • [ 2812963 ] JDBC connection hangs if result is too long
  • [ 2839063 ] M5: property error in src/modules/mal/Tests/chopper03.mal
  • [ 2840225 ] Mx creates output files even on errors
  • [ 2847738 ] SuperBall build fails with PHP 5.3.0
  • [ 2860037 ] PF: fatal error: aborted too deep recursion on linux
  • [ 2860574 ] PF: fatal error: column not found / BATappend access denied
  • [ 2865631 ] SQL: incref assertion
  • [ 2869818 ] SQL: Program contains errors
  • [ 2871426 ] SQL: mclient-lsql-D fails everywhere but on 64-bit Fedora
  • [ 2872421 ] SQL: Crash when trying to create function to read only db
  • [ 2872880 ] mclient shows prompt with "-s" option without "-i&qu ot; option
  • [ 2872884 ] mclient -t -s... does not show "Timer" info
  • [ 2876948 ] PF: error "We do not support the value NaN"
  • [ 2879833 ] SQL: SIMILAR keyword does not exist

Aug 2009 SP1 bug-fix release

This is a bug fix release of the Aug2009 version. This version can be installed "on top of" the Aug2009 release. The database format is fully compatible, so there is no need to do a dump and restore. (Although it is *always* a good idea to make backups.)

Some of the bugs that were fixed:

  • On Windows, we switched to newer versions of the iconv and geos libraries (iconv 1.11.1 and geos 3.1.1, from iconv 1.11 and geos 3.0).
  • On Windows, MonetDB4/XQuery was compiled with a newer version of the Intel compiler (version 11.1.046 from version 10.1.022).
  • Better error handling if BATs are growing too large. (SF bug #2850513)
  • The MAL parser was made more robust against syntax errors.
  • A recursion depth check was added to the MAL interpreter. (SF bug #2852707)
  • The PHP client is not compatible with PHP 5.3.0 or later. We now check for that. (Note that in a future release, the PHP client code will be replaced by a pure PHP implementation where this restriction does not apply.) (SF bug #284773)
  • The SQL dump code (msqldump and mclient --dump) now specify all delimiters and the quote character on the generated COPY INTO query. This is needed to be compatible with the next feature release.
  • The SQL dump code (msqldump and mclient --dump) now also knows about CLOBs (Character Large Objects) and properly quotes the values.
  • The problem that occurred under certain circumstances that the JDBC driver would throw an SQLException when large result sets were retrieved is fixed. (SF bug #2812963)
  • Fixed a bug in MonetDB/XQuery where the comparison between values of types strings and untypedAtomic was not done correctly. (SF bug #2853441)
  • Fixed a bug in MonetDB/XQuery where the dead code elimination was a bit too eager. (SF bug #2852928)
  • Fixed a bug in MonetDB/XQuery where a warning was generated that a pin count was below zero. (SF bug #2863458)
  • Fixed a bug in PF/Tijah where OR queries were treated as AND queries. (SF bug #2836870)
  • Fixed a bug in PF/Tijah where the nexi scanner would split strings on '_'.
  • Fixed overflow checks in MonetDB/SQL. (SF bug #2853458)
  • Fixes to get the embedded SQL example working again.
  • Fixed startup of MonetDB/SQL server so that a client cannot connect before the server is ready. (SF bug #2855021)
  • Fixed a bug in MonetDB/SQL where old and dropped view definitions were sometimes incorrectly reused. (SF bug #2863804)
  • Fixed a bug in merovingian where the mero_port in the config file was not properly recognized. (SF bug #2844202)
  • Fixed a bug in MonetDB/SQL where it was not possible to add month intervals to a timestamp. (SF bug #2854949)
  • Fixed various bugs in the SQL engine. (SF bugs #2847106, #2862146)
  • Use bigint for RESTART value in MonetDB/SQL. (SF bug #2846238)
  • Fixed a bug in MonetDB/SQL with parsing hex values.
  • Fixed a bug in MonetDB/SQL when attempting to make changes to the schema when the database is in read-only mode. (SF bug #2830238)
  • Fixed a bug in MonetDB/SQL where adding a primary key constraint to a column containing a NULL value was allowed. (SF bug #2075241)
  • Fixed a bug in MonetDB/SQL where updating values in one column could affect the values in another column. (SF bug #2864313)

The following bug tracker items were fixed since the Aug2009 release:

  • [ 2808453 ] SQL: Trying a copy into in read only database segfault
  • [ 2830238 ] SQL: Crash when trying to write on read-only DB
  • [ 2831472 ] tijah:queryall crashes Mserver
  • [ 2836870 ] PFtijah: Use of 'or' in phrase search yelds incorrect result
  • [ 2840310 ] SQL: "Cannot access descriptor" on big endian
  • [ 2844202 ] Merovingian: mero_port configuration option ignored
  • [ 2846238 ] alter sequence restart on large integer
  • [ 2847106 ] SQL: crash upon join with advanced between
  • [ 2848525 ] mac10.6 and monetdb installation
  • [ 2850341 ] SQL: bit_and returns invalid values
  • [ 2852928 ] PF: updates broken using algebra front-end?
  • [ 2853441 ] PF: distinct values on two sequences
  • [ 2853458 ] SQL: cast exceeding maximum value, returns null
  • [ 2854949 ] SQL: + interval '1' year; Program contains errors
  • [ 2855021 ] SQL: mserver accepts sql connections before scenario load
  • [ 2855152 ] mac10.6 and monetdb installation
  • [ 2862146 ] Two COUNT(*) with LIMIT doesn't execute
  • [ 2863458 ] PF/TIJAH: tokenize in a for-loop
  • [ 2863804 ] SQL: old views incorrectly reused
  • [ 2864313 ] SQL: Update changes all columns with same type and default

Aug 2009 feature release


Packaging

Fedora

We now provide RPMs for Fedora 10 & 11, but no longer for Fedora 9. However, a first check indicates that the Fedora 10 RPMs also install and work fine on Fedora 9.


MonetDB Common

 

  • A bug-fix in the May2009 release had unnoticedly introduced a potential performance degradation, caused by non-optimal use of memory-mapped files, with workload that exceed physical memory, in particular with bulk-loading ("COPY INTO") large tables in MonetDB/SQL. This performance problem has been fixed.
  • Various problems related to sloppy type usage and type casting had surfaced with testing on big-endian hardware as well as on 64-bit systems when using 32-bit OIDs. These bugs have been fixed.
  • The compilation process has been improved by splitting the "infamous" large (generated) "gdk_scanselect_defs.c" into multiple files.


MonetDB Server 5

 

  • auto-sensing is no longer implicit based on the port number being 0, but can now be enabled and disabled via mapi_autosense in monetdb5.conf. When set to enabled, it will try to find a free port starting from the port given by mapi_port.


SQL

 

  • fix crash with select * from 'whatever' = bla, select * where bla = "whatever", Bug #2825974
  • Ignore escapes for non-escape sequences, instead of inserting a literal backslash. Bug #2830900
  • Decimals now are properly detected in PreparedStatements, allowing to use them. Bug #2831994
  • Fixed hang after invalid statement + COMMIT outside auto-commit mode. Bug #2814622
  • Fixed NUMERIC type detection. Bug #2818176
  • No longer allow SELECT in value lists, causing confusing error messages. Bug #2807609
  • Allow for NULL assignment and NULL as argument of a function. Bug #2812729
  • Added missing power (^) operator. Bug #2812721
  • DISTINCT was not properly recognised, causing incorrect count results. Bug #2826015
  • When encountering ambigious columns, fail instead of crashing. Bug #2807336
  • GROUP BY on a view with constants no longer yields in a crash. Bug #2837561
  • PLAN output is fixed to no longer violate the protocol and to be tuples instead of comments.
  • Optimisating pipelines can now be set in monetdb5.conf and changed without compiling SQL.
  • sql_logdir can no longer be set and defaults to a location close to the actual data. This approach was chosen since the logs are useless without the data.
  • The ILIKE operator was added. This operator is a case-insensitive LIKE operator, as also found in PostgreSQL.
Merovingian

The following features were implemented in the merovingian and monetdb programs:

  • merovingian: Implemented a configurable port for the neighbour discovery service. This also allows to entirely disable neighbour discovery by setting mero_discoveryport to 0. Feature request #2800964
  • merovingian: Implemented full per-database property support. monetdb now can manage the following per-database properties: forward (whether to proxy or redirect), shared (if it will be available in the network and how) and nthreads (number of working threads the mserver process uses). Feature Request #1837591.
  • merovingian: Increased the default mero_exittimeout from 7 to 60 seconds. This makes merovingian by default wait longer for mservers to exit before forcefully terminating them.
  • merovingian: Implemented sharing of databases with tags. Using these tags, extra information is available on a remote merovingian to select a database during connect. For this, a client can now connect to a database using a glob expression, which is matched on the database plus tag. When multiple databases match such expression, a load-balancing technique is applied. See the merovingian man-page for details.
  • monetdb: Implemented dynamic width of monetdb status output. If a database name is wider than 14 chars, the name column is made wider as long as the terminal window allows. If a column cannot be made any wider this way, it is abbreviated as before.
  • monetdb: Upon monetdb destroy, avoid multiple questions by merging them into a single confirmation for the destroy command. This accidently also fixed a bug where a response such as 'yes' would abort the next database removal.
  • monetdb: Databases created by monetdb now can only have names restricted to the pattern [A-Za-z0-9-_]+.
  • monetdb: Allow setting the name of a database (a.k.a. renaming it).
  • monetdb: Allow an expression to be used to match against the list of discovered or local databases as argument in the commands discover and status.
  • monetdb: destroy now allows to remove a database under maintenance. This way an administrator can prevent users connecting to the database and hence starting it on demand, which in turn prevents removal of the database.


XQuery

 

  • xquery_logdir can no longer be set and defaults to a location close to the actual data. This approach was chosen since the logs are useless without the data.


Client Packages

mclient

The rendering of the mclient utility for SQL has been improved to better utilise the available space on the screen. Main improvement is that columns are no longer wrapped if they contain only NULLs, even though enough space on the terminal is availble.

Python

 

  • The python API now supports connections to Merovingian.
  • Transaction support was added.
  • Added compatibility with python3k.
  • Full Unicode/UTF-8 support.
  • autocommit, commit and revert functions were added.
Ruby

 

  • Fixed bugs that prevented the correct working of activerecords' migration; The activerecord connector now supports insertion, update and alter table operations; Type casting is working in activerecord; Added a rubygem and rakefile for activerecord-monetdb-adapter; Added a new usage example for activerecord to the README file; Added an example directory to the cvs tree; The driver now correctly works with Merovingian.
  • Support for ruby 1.9.
  • Both the standalone driver and activerecord now support transactions, nested transactions are simulated via savepoints in activerecord.
  • Type conversion in the standalone driver is now performed after data has been retrieved and can be executed on single fields.
JDBC

 

  • blockmode_blocksize option, and the -Xblksize option from JdbcClient were removed, as they are meaningless and inimplemented in the current JDBC driver.
  • Full redirection support was added to support Merovingian's tag-based connections for localisation and loadbalancing.

May 2009

May 2009 mk Tue, 03/30/2010 - 20:31

MonetDB May2009-SP2 release

Release "May2009-SP2" consists of the following updated components

Name Version CVS module CVS branch Note/Comment
MonetDB Common 1.30.4 MonetDB May2009
MonetDB Clients 1.30.4 clients May2009
MonetDB4 Server 4.30.4 MonetDB4 May2009
MonetDB4/XQuery 0.30.4 pathfinder May2009
MonetDB5 Server 5.12.4 MonetDB5 May2009
MonetDB5/SQL 2.30.4 sql May2009
MonetDB5/SQL/GIS 0.10.4 geom May2009
MonetDB Java 1.30.4 java May2009
MonetDB Testing 1.30.4 testing May2009 (developers only)

IMPORTANT NOTE:
Before upgrading to the latest "May2009-SP2" release from a release of MonetDB older than the Nov2008 release, please make a dump (SQL, XQuery) of your database, and rename/move/backup the database directories (dbfarm, sql_logs, xquery_logs). After the upgrade, restore (SQL, XQuery) your database, again. Once you have checked and confirmed the correctness of the restore, you can delete the above renamed/moved/backed-up old database directories.

Contents

May 2009 SP2 bug-fix release

This is the second bug fix release of the May2009 version. This version can be installed "on top of" the May2009 release. The database format is fully compatible, so there is no need to do a dump and restore. (Although it is *always* a good idea to make backups.)

Some of the bugs that were fixed:

  • Fixed a problem where in certain condition you could get an error on Windows saying "GDKremovedir: rmdir(bat\DELETE_ME) failed".
  • Various improvements to the Python and Ruby clients.
  • Fix mserver5 crash when started on demand by merovingian (SF bug #2805340).
  • Check for write errors during dump and abort when they occur.
  • Some innocuous looking XQuery queries could produce large incorrect results and take a relatively long time to do so. SF bug #2815820.
  • Fixed some potential buffer overflow problems in the Mapi library.
  • Improved help message from mclient (SF bug 2818536).
  • Fixed certain SQL LIKE queries on some O/S.
  • Fixed SQL dump in mclient and msqldump to dump values of some less used types correctly.
  • The Debian/Ubuntu installation now also contains the MonetDB PHP client support files.
  • Fixed several performance issues (among others: SF bug 2821251).
  • The various client libraries now also recognize the wrd type which may be emitted by the SQL server.
  • In SQL, the char and varchar types now use Unicode code points (characters to you and me) as the unit for counting instead of bytes in the UTF-8 representation (SF bug 2822855).
  • Fixed some UI issues with the XQuery administrator console.
  • Fixed a bug in SQL when handling IN and OR lists with equal values.
  • Fixed a stack overflow condition in SQL with certain types of quite complex queries (SD bug 2827784).

The following bug tracker items were fixed since the May2009 SP1 release:

  • [ 1840239 ] BAT capacity checking needed
  • [ 2805340 ] Merovingian (?): mserver5 segfaults if started by a client
  • [ 2815820 ] PF: Endless loop in innocent xquery
  • [ 2818536 ] improve mclient -u / -p help
  • [ 2818931 ] MAL: segv on mortal soul's syntax error
  • [ 2819972 ] function problom ----return and elseif
  • [ 2821251 ] PF: Complex optimization overly complex
  • [ 2822855 ] utf8 real size upon insert

May 2009 SP1 bug-fix release

This is a bug fix release of the May2009 version. This version can be installed "on top of" the May2009 release. The database format is fully compatible, so there is no need to do a dump and restore. (Although it is *always* a good idea to make backups.)

Some of the bugs that were fixed:

  • mclient and msqldump would cache all results. This is fixed.
  • The Fedora RPM for the Ruby clients package is now included.
  • Some files were missing in the Python client directory. They are now included in the client source tar ball.
  • mclient now shows the results of all subqueries in a query batch that was passed on the command line using the -s option if there was an error in one of them.
  • mclient \d and \D interactive commands now obey the \| command.
  • Fixed naming of the document and collection when both the --input and --collection arguments are given to mclient (XQuery).
  • Various fixes to the low level memory mapped file support.
  • Fixes to the geom module.

The following bug tracker items were fixed since the May2009 release:

  • [ 2603876 ] XQ: fn:put() tests segfault with ALG back-end
  • [ 2687549 ] SQL: test type_dump_test.SF-989257.SQL.sh fails since Feb 28
  • [ 2737889 ] rand() is not executed for every row when using alias
  • [ 2782202 ] SQL: dump&restore fully usable only by admin
  • [ 2791356 ] Floating point exception
  • [ 2797767 ] mclient --dump caches dump
  • [ 2803477 ] alter table drop contraint error
  • [ 2804823 ] The Mapi Client Utility
  • [ 2805513 ] fn:trace() produces output _and_ error
  • [ 2807038 ] tab + ) + mal function = seg fault

May 2009 feature release


Packaging

Windows

The Windows binaries for SQL and ODBC were built using Microsoft Visual Studio 2008, unlike previous releases which were built using Microsoft Visual Studio 2005. The main implication of this change is that the ODBC driver should now also install on Windows Vista.

The Perl, PHP, and Python clients were removed from the Windows installers. The clients are still available in the source distribution (although the Python client has changed significantly).

Fedora

There is no longer an RPM for MonetDB-client-ruby. In order to update to this release, you will first have to remove MonetDB-client-ruby after which you can update to this version. (A new version of the RPM will probably appear in a future release.)

Ubuntu

The Ubuntu version was built for Ubuntu 9.04, the Jaunty Jackalope. The Feb2009 release was built for Ubuntu 8.10, the Intrepid Ibex.


MonetDB Common

The way string data is stored in the database has changed for systems with a 64-bit word size and 32-bit OIDs. When the server first opens a database that was created using the old format, the database is updated automatically. This change affect only the 64-bit Windows build. The other 64-bit builds that are provided by us use 64-bit OIDs and are unaffected by the change.


MonetDB Server 5

Parallel loading has been disabled since there was an inherent problem with the implementation. This also affects SQL.


SQL

Write-ahead log

The default location of the write-ahead log has changed. At startup, when the server sees there is an SQL catalog but no write-ahead log, the server will exit immediately with an error message. You can do one of two things (not counting deleting your whole database):

  • move the log files to the new location (recommended): create the directory that is mentioned by the error message and move the log files (two files, log and log.n where n is some integer) there;
  • start the server with the --set sql_logdir=... option (where you replace ... with the appropriate path) to point the server to the old location.

In either case you need to know the old location of the write-ahead logs. This old location was $prefix/var/MonetDB5/sql_logs/<dbname>/sql/ where $prefix is the prefix under which MonetDB was installed and <dbname> the database name. The error message mentioned above gives the expected location of the write-ahead logs.

Note that when you use the Windows installer and you start the server using the Start menu, the write-ahead log is moved to the new location automatically.

Authentication

The authentication mechanism has changed somewhat so that it is now possible to dump and restore password information. Note that the passwords themselves cannot be retrieved: the dump/restore mechanism uses a hash of the password.

Merovingian

The following features were implemented in the merovingian and monetdb programs:

  • The logging of merovingian has changed. TME records are no longer issued, instead each line is prefixed with a timestamp, syslog style. Feature Request #2782776
  • merovingian now reopens its logfiles when being sent the SIGHUP signal. This allows utilities like e.g. logrotate to rotate the merovingian.log file. Feature Request #2782776
  • The merovingian default log file location has changed from /var/log/merovingian.log into /var/log/MonetDB/merovingian.log.


XQuery

Like the previous "Feb 2009 SP2" release, the default Algebra-based Pathfinder compiler ("ALG") still does not support a few XQuery features and extensions. A detailed list and temporary work-arounds by resorting to the deprecated "milprint_summer" ("MPS") version of the Pathfinder compiler can be found here.

Write-ahead log

The default location of the write-ahead log has changed. At startup, when the server sees there is an XQuery catalog but no write-ahead log, the server will exit immediately with an error message. You can do one of two things (not counting deleting your whole database):

  • move the log files to the new location (recommended): create the directory that is mentioned by the error message and move the log files (two files, log and log.n where n is some integer) there;
  • start the server with the --set xquery_logdir=... option (where you replace ... with the appropriate path) to point the server to the old location.

In either case you need to know the old location of the write-ahead logs. This old location was $prefix/var/MonetDB4/xquery_logs/<dbname>/xquery/ where $prefix is the prefix under which MonetDB was installed and <dbname> the database name. The error message mentioned above gives the expected location of the write-ahead logs.

Note that when you use the Windows installer and you start the server using the Start menu, the write-ahead log is moved to the new location automatically.


Client Packages

Mapi

Support for SSL-encrypted connections with the server has been discontinued.

The protocol between client and server has changed slightly. This means you cannot use an old client with a new server.

Python

The Python client modules were rewritten and are now in pure Python and so do not depend on any other part of MonetDB. The new module has support for Python 2.5, 2.6 and 3.0, and it is Python DBAPI 2.0 compatible.

Ruby

The Ruby client modules were rewritten and are now in pure Ruby and so do not depend on any other part of MonetDB.

February 2009

February 2009 mk Tue, 03/30/2010 - 20:33

MonetDB Feb2009-SP2 release

Release "Feb2009-SP2" consists of the following updated components

Name Version CVS module CVS branch Note/Comment
MonetDB Common 1.28.4 MonetDB Feb2009
MonetDB Clients 1.28.4 clients Feb2009
MonetDB4 Server 4.28.4 MonetDB4 Feb2009
MonetDB4/XQuery 0.28.4 pathfinder Feb2009
MonetDB5 Server 5.10.4 MonetDB5 Feb2009
MonetDB5/SQL 2.28.4 sql Feb2009
MonetDB5/SQL/GIS 0.8.4 geom Feb2009
MonetDB Java 1.28.4 java Feb2009
MonetDB Testing 1.28.4 testing Feb2009 (developers, only)

IMPORTANT NOTE:
Before upgrading to the latest "Feb2009-SP2" release from a release of MonetDB older than the Nov2008 release, please make a dump (SQL, XQuery) of your database, and rename/move/backup the database directories (dbfarm, sql_logs, xquery_logs). After the upgrade, restore (SQL, XQuery) your database, again. Once you have checked and confirmed the correctness of the restore, you can delete the above renamed/moved/backed-up old database directories.

Contents


Feb 2009 SP2 bug-fix release

One set of fixes that affects all components is that stack overflow checking has been improved. It is now much less likely that the server crashes when queries need too much stack space. Instead, the server will return an error.

MonetDB/XQuery (tijah): fixes to UTF-8 handling; fixes towards being able to index more than 2**31 words on a 64 bit architecture.

MonetDB5: fixed a crash on exit of the server (this crash was not harmful, just annoying).

MonetDB: avoid integer overflow on very large tables.

mclient/msqldump: fixes to the order in which tables, views, and functions are dumped; also dump triggers; fixed the Ubuntu and Debian installers to include mclient and msqldump.

MonetDB/XQuery: fixed a bug where documents were not available when another document in the collection was loaded.

merovingian/monetdb: fixes to the way the server is stopped.

MonetDB/SQL: fixed a bad concurrency bug where updates were lost or data was incorrectly inserted.

The following bug tracker items were fixed since the Feb2009 SP1 release:

  • [ 1811229 ] [ADT] Adding large document, with update support
  • [ 1836980 ] XQ: Error with declared variable in module
  • [ 1894477 ] PF: "fatal error: aborted too deep recursion" on Windows
  • [ 2255453 ] mclient terminates connection unexpectedly
  • [ 2418722 ] pf:docname() problem
  • [ 2549582 ] M4: leftthetajoin estimate way to small
  • [ 2550429 ] nlthetajoin asks for 128 bytes too much memory
  • [ 2567316 ] Segfault after long period of use
  • [ 2636285 ] ancestor-or-self: illegal data in size column or context lis
  • [ 2637051 ] Wrong error message for invalid identifier in group by
  • [ 2664405 ] fn:collection("DOES-NOT-EXIST") does not fail
  • [ 2668437 ] PF runtime: parent step produces not/wrongly sorted result
  • [ 2683759 ] PF:memory leak when fireing multiple simultaneous clients
  • [ 2687705 ] Performance degradation due to memory management
  • [ 2693776 ] xquery fails: memory allocation
  • [ 2698327 ] XQ/GDK: strPut: Assertion failed
  • [ 2704016 ] 'null' from copy into gets wrong
  • [ 2710426 ] PF/TIJAH: dbat_extend fails
  • [ 2711258 ] Mtest.py fails with Python <=2.3.?
  • [ 2716585 ] Java fails to run make dist with latest autotools
  • [ 2722117 ] PF/TIJAH: dbat_finalize fails
  • [ 2722174 ] XQ: shredding 70GB XML fails.
  • [ 2726337 ] M4: Mserver startup errors after changes on 2009/04/01
  • [ 2747661 ] Merovingian: wait for stopping MonetDB
  • [ 2752158 ] M4: test output differs due to reduced error buffer size
  • [ 2756553 ] stethoscope -p parameter doesn't work
  • [ 2757113 ] crash in la_cse
  • [ 2757642 ] Inaccurate AVG of SQRT with 51M records
  • [ 2761480 ] compilation error
  • [ 2776100 ] SQL: "CREATE VIEW AS SELECT 1;" fails
  • [ 2777473 ] PF: incremental shredding broken


Feb 2009 SP1 bug-fix release

The main fix that was done in this release that affects all parts of the code, but most especially MonetDB/SQL is that we have plugged a large number of memory leaks. MonetDB/SQL can now handle larger databases and more demanding queries. This is especially noticeable on 32 bit Windows systems where the memory management system (the system malloc library) is not as sophisticated as the one in Linux.

mclient/msqldump: Fix to the order in which tables and views are dumped: tables must be dumped before views, since views may reference tables and the view creation will only work if the referenced tables already exist.

mclient: Fixed potential deadlock situation when there is a large input file and several queries early on produce large results.

MonetDB/XQuery: Fixed a bug where a replace operation could corrupt the document (bug #2642003).

The following bug tracker items were fixed since the Feb2009 release:

  • [ 2581675 ] mal variable type is persistent over sessions
  • [ 2627138 ] SQL: test fails with BATpropcheck error
  • [ 2642003 ] XQUF replace insert cycle corrupts database
  • [ 2651397 ] M5: manual.summary() produces empty result
  • [ 2654133 ] simple join fails on latest stable
  • [ 2654300 ] problem with views reusing aliases ?
  • [ 2654603 ] assertion failed in store.mx
  • [ 2656762 ] Views with select on top of join - attribute lookup problem
  • [ 2658243 ] SQL: INSERT INTO + ORDER BY
  • [ 2671445 ] Mapi read error when rows exceed maximum
  • [ 2678502 ] Scary: (not)equal evaluated wrong
  • [ 2686045 ] M5 fails to compile with optimization enabled (gcc & icc)
  • [ 2686121 ] XQ: oids in DocmgmTape
  • [ 2687040 ] M4: test src/modules/plain/Tests/bat.py fails
  • [ 2705666 ] Merovingian does not remove pid-file on clean shutdown

The following things were fixed or changed in merovingian and monetdb:

  • merovingian.mx: Remove the pid file upon shutdown, fixes bug #2705666
  • monetdb.mx: Small cosmetic fix not to overflow the uptime in monetdb status display
  • merovingian.mx: Fix hang on Linux when shutting down. select() calls are not interrupted on Linux, so make sure they time out. Fixes bug #2623838
  • merovingian.mx: Fixed crash of merovingian when a to be started database terminates immediately after startup (such as when it crashes)
  • merovingian.mx: Protect shared linked list structures from being accessed and/or modified by two or more threads concurrently. This resolves random crashes. Fixes bug #2605706
  • monetdb.mx: Fixed crash when using monetdb stop on a non-started database. Bug #2605105
  • merovingian.mx: Create logfiles when they do not exist. Bug #2556973

The following was changed in the JDBC interface:

  • Connection.setReadOnly() now always succeeds but sets a warning when called with true. This allows Mondrian to be used with MonetDB/SQL.
  • Holdability invocation methods were implemented, because the JDBC specs state that they need to be implemented and just be warned upon when unavailable (15.1.3.1).
  • Implemented retrieved keys, works only for the first auto-generated column, which is always of int-type.


Feb 2009 feature release


Packaging

We now provide Debian (Lenny, i.e. 5.0) and Ubuntu (Intrepid, i.e. 8.10) packages. These packages have only been very lightly tested (the packaged software is tested extensively, though). For now we provide 32 and 64 bit versions for Debian and only a 64 bit version for Ubuntu. This of course in addition to the Fedora packages we already provided.


MonetDB Server 5

The code base has been tested for memory leaks reducing the change of running out of memory too quickly.

The Merovingian has undergone more thorough stress testing making it much more stable.


SQL

The optimizer of SQL has been overhauled. Rather than working on the binary plans, it performs optimization on the relational algebra tree before it produces the binary plans.


XQuery

Like the previous "Nov 2008 SP2" release, the default Algebra-based Pathfinder compiler ("ALG") still does not support a few XQuery features and extensions. A detailed list and temporary work-arounds by resorting to the deprecated "milprint_summer" ("MPS") version of the Pathfinder compiler can be found here.

Time and date related types and the comparison and extraction functions on top of them have been implemented in MonetDB/XQuery.


Mtest

Mainly for MonetDB developers:

The MonetDB testing tools (Mtest.py & friends) from MonetDB/src/testing/ have been moved into their own "testing" CVS module / MonetDB component.

IMPORTANT NOTE:

In case you plan to upgrade your existing CVS-based "MonetDB" installation, please run `make uninstall` in your "MonetDB" build directory, first, i.e., before running `cvs update` in your "MonetDB" source directory.

Once you update your "MonetDB" to the latest CVS development trunk version, and plan to continue (or start) using Mtest.py, you also need to check out the new "testing" CVS module (from SF) and compile and install it any time after you compiled and installed "MonetDB", and before you plan to use Mtest.py.

The new "testing" CVS module depends only on "MonetDB" (and hence requires that to be installed), but on no other MonetDB component (except from the "buildtools", in case your are compiling from CVS sources). No other MonetDB component depends on the new "testing" CVS module other than that you obviously need to have "testing" installed once you want/need to run tests using Mtest.py.

Details:

The MonetDB testing tools (Mtest.py & friends) are not part of the actual MonetDB functionality, but mainly for MonetDB development purposes (and hence only distributed in the source tarballs, but not in any (official) binary packages).

To make them less dependent on release cycles (and vice versa), as well as to prepare making them usable without MonetDB --- for now, they actually still depend on MonetDB; see below) --- we decided to separate the MonetDB testing tools from the "MonetDB" CVS module into their own "testing" CVS module.

For the time being (i.e., until further notice), "testing" depends on "MonetDB" --- more precisely only on some utils from MonetDB/src/common/, but not on the GDK kernel from MonetDB/src/gdk/, as

  1. Mtest.py (mainly for "convenience") still uses monet_options.py; and
  2. (on Windows only), Mdiff.c uses "our" getopt code.

Python modules "subprocess26.py" & "trace.py" have been moved from <PYTHON_LIBDIR>/MonetDB/ to <PYTHON_LIBDIR>/MonetDBtesting/ .


Bug fixes

A detailed list of the 65 bug reports that have been closed since the previous "Nov 2008 SP2" release can be found in the MonetDB Bug Tracker at SourceForge: page 1/2, page 2/2.
The detailed list of changes can be found in the source code CVS logs.


Known Issues in this Release

  • Parallel loading is known to be buggy. Actual workaround is to start the server using gdk_nr_threads=1 set in your monetdb5.conf file (or given via --set gdk_nr_threads=1 on your mserver5's command line) during the bulkload using COPY INTO FROM FILE with mclient.
    Details: (no known bug tracker reference)
  • ...
 

November 2008

November 2008 mk Tue, 03/30/2010 - 20:35

MonetDB Nov2008-SP2 release

Release "Nov2008-SP2" consists of the following updated components

Name Version CVS module CVS branch
MonetDB Common 1.26.2 MonetDB Nov2008
MonetDB Clients 1.26.2 clients Nov2008
MonetDB4 Server 4.26.2 MonetDB4 Nov2008
MonetDB4/XQuery 0.26.2 pathfinder Nov2008
MonetDB5 Server 5.8.2 MonetDB5 Nov2008
MonetDB5/SQL 2.26.2 sql Nov2008
MonetDB5/SQL/GIS 0.6.2 geom Nov2008
MonetDB Java 20081118 java (none; CVS trunk)

IMPORTANT NOTE:
Before upgrading to the latest "Nov2008-SP2" release from a release of MonetDB older than the Nov2008 release, please make a dump (SQL, XQuery) of your database, and rename/move/backup the database directories (dbfarm, sql_logs, xquery_logs). After the upgrade, restore (SQL, XQuery) your database, again. Once you have checked and confirmed the correctness of the restore, you can delete the above renamed/moved/backed-up old database directories.

Contents

Nov 2008 SP2 bug-fix release

This is the second bug fix release of the Nov2008 version. This version can be installed "on top of" the Nov2008 and Nov2008-SP1 releases. The database format is fully compatible, so there is no need to do a dump and restore. (Although it is *always* a good idea to make backups.)

Some of the bugs that were fixed:

  • MonetDB/XQuery: There was a bug when shredding documents into an existing updatable collection when the existing collection happened to end at a page boundary. (Bug [ 2141012 ] XQ: unclear error).
  • General: Fix to configure to not complain when the raptor library isn't found, unless it was specifically asked for.
  • MonetDB/SQL: significantly speeded up loading of data when lines are extremely long (e.g. when loading BLOBs).
  • XRPC: Since the change of the XRPC protocol (by June 2008), the XRPC server also sends response messages (with empty nodes) for updating function calls. However, the XRPC serever failed to send response messages for calls to the Pathfinder document management functions, because they are handled differently than normal updating functions. This is now fixed.
  • XRPC: The function XRPCHTTPConnection.sendReceive() did not check if HttpURLConnection.getResponseCode() returns -1 or not. If the return value is indeed -1, subsequent use of the stream returned by HttpURLConnection.getInputStream() or HttpURLConnection.getErrorStream() might cause a NullPointerException. This is now fixed. The return value and the stream returned by those two functions are now checked and a proper error message is given.

The following bug tracker items were fixed since the Nov2008-SP1 release:

  • [ 1964658 ] Problem with names of reverse()'d BATs.
  • [ 2141012 ] XQ: unclear error
  • [ 2233581 ] MonetDB does not like '\0' in its input
  • [ 2388542 ] SQL bad blob insert causes crash
  • [ 2405952 ] Merovingian takes too much CPU resources
  • [ 2430201 ] Is distinct broken?
  • [ 2457195 ] Broken query results in OOM, working one has no output
  • [ 2460935 ] Pathfinder fails to compile with icc
  • [ 2460953 ] `make dist` (and hence `make rpm`) fails for pathfinder
  • [ 2466725 ] SubSelect with Outer Reference, result is wrong
  • [ 2468513 ] SQL: copy into does not detect syntax errors
  • [ 2471371 ] SQL: varchar bounds should be obeyed

Nov 2008 SP1 bug-fix release

This is the first bug fix release of the Nov2008 version. This version can be installed "on top of" the Nov2008 release. The database format is fully compatible, so there is no need to do a dump and restore. (Although it is *always* a good idea to make backups.)

Some of the bugs that were fixed:

  • Fixed formatted printing from MAL.
  • mclient now exits with an error code if one of the queries in a non-interactive session failed. [2388047]
  • Fixed a deadlock condition when multiple queries simultaneously update the same part of a document (XQuery).
  • When writing to a file, mclient now flushes the output when the query is done instead of waiting until the file is closed. [2344251]
  • Fixed a race condition in document management (XQuery). [2338542]

The following bug tracker items were fixed since the Nov2008 release:

  • [ 1911209 ] XQ: MonetDB 'hangs' after requesting non-existing doc
  • [ 1981762 ] PF/alg: "Mapping unique name to an original name failed."
  • [ 2049320 ] pf:collection doc count wrong (again?)
  • [ 2075180 ] the unique column can't insert more than 1 row with null
  • [ 2075241 ] When the NULL is in a column,it shouldn't add primary key
  • [ 2103348 ] unique problem with nulls
  • [ 2142522 ] XQ: Duplicates when not using intermediate result
  • [ 2144639 ] XQ: fn:collection() in algebra version
  • [ 2209871 ] most GEOM test fail after error handling changes
  • [ 2224233 ] M4 test ID.914935 fails after warning changes
  • [ 2233677 ] SQL Command kills DB
  • [ 2276427 ] CMDmulti_merged_union: operation failed
  • [ 2277120 ] XQ: following(-sibling)/preceding cause Mserver to crash
  • [ 2305746 ] merovingian crashes when starting a vaultkeyless database
  • [ 2338524 ] Deadlock on barrier
  • [ 2344251 ] Output isn't complete
  • [ 2388047 ] Mclient exit code
  • [ 2388424 ] ERROR: XML Generation: NULL BAT has a 24 head, but tail is N
  • [ 2420768 ] col from another schema inside function body

Nov 2008 feature release

Copyright & License

The copyright for the MonetDB suite was transferred to a newly formed company MonetDB B.V. One of the most important conditions for this transfer was that the software remains open source, and so the license under which this software is being distributed has not changed (apart from the copyright notice).

Windows Installers

The default installation location has been changed from C:\Program Files\CWI to C:\Program Files\MonetDB.

MonetDB Common (GDK)

The kernel administration for BATs has been changed to prepare for future enhancements. All descriptors are now initialized and kept in memory, which reduces the number of files and related IO accessing the descriptors.

Memory management for large database has been improved, with the scavenging thread becoming more aggressive in moving dirty pages to disk when we reach 70% of the physical memory.

Code cleaning and simplification continues to provide ease of compilation on many platforms.

MonetDB Server 5

The MonetDB 5 kernel underwent a code clean up too, e.g. datatypes have been more strictly applied and all kernel errors are propagated to MAL exceptions.

The number of experimental modules have been developed. The 'recycler' provides support for automatic re-use of intermediate results. It improves query throughput for long sequences with partially overlapping queries.

The system supports intra-query parallelism using the multi-core systems capabilities using the 'dataflow' optimizer.

The 'mergetable' optimizer has been extended to perform symbolic reasoning over oid-reasoning. Together with the MAT and dataflow optimizers it provides a basis to improve multi-core performance.

Several bugs were fixed in Merovingian, the MonetDB Database Server daemon, including a file descriptor leak, which caused Merovingian to terminate after a certain amount of connections using the proxy method. Merovingian has been extended with neighbour discovery features, allowing to redirect to discovered neighbours. The monetdb tool has been improved to give better feedback on starting and stopping databases. Most notably it now waits for completion of the start operation, and gives error feedback when starting failed for some reason.

SQL

This release comes with a few more optimizers enable by default. The most important one is the inter-query parallelism, which leads to better use of a multi-core platforms. The effects can be seen in our TPCH experiments.

The dataloading code has been improved during our effort to load a 2.6TB astronomical database. Upon request we can provide a detailed report on this experiment.

A query log can be captured with the history.sql script. It provides a basis for post-session analysis of execution times.

Several bugs reported have been solved and functionality in many other areas has been improved.

XQuery

Like the previous "Jun 2008" release, the default Algebra-based Pathfinder compiler ("ALG") still does not support a few XQuery features and extensions. A detailed list and temporary work-arounds by resorting to the deprecated "milprint_summer" ("MPS") version of the Pathfinder compiler can be found here.

Bug fixes

A detailed list of the 116 bug reports that have been closed since the previous "Jun 2008" release can be found in the MonetDB Bug Tracker at SourceForge: page 1/3, page 2/3, page 3/3.
The detailed list of changes can be found in the source code CVS logs.

Known Issues in this Release

  • Merovingian crashes when starting a vaultkeyless database. Actual workaround is not to try and start databases not created using monetdb create.
    Details: SF bug tracker #2305746
  • Parallel loading is known to be buggy. Actual workaround is to start the server using gdk_nr_threads=1 set in your monetdb5.conf file (or given via --set gdk_nr_threads=1 on your mserver5's command line) during the bulkload using COPY INTO FROM FILE with mclient.
    Details: (no known bug tracker reference)

June 2008

June 2008 mk Tue, 03/30/2010 - 20:35

Release "Jun2008" consists of the following updated components

Name Version CVS module CVS branch
MonetDB Common 1.24 MonetDB MonetDB_1-24
MonetDB Clients 1.24 clients Clients_1-24
MonetDB4 Server 4.24 MonetDB4 MonetDB_4-24
MonetDB4/XQuery 0.24 pathfinder XQuery_0-24
MonetDB5 Server 5.6 MonetDB5 MonetDB_5-6
MonetDB5/SQL 2.24 sql SQL_2-24
MonetDB5/SQL/GIS 0.4 geom Geom_0-4


and the jars for JDBC and XRPC.

IMPORTANT NOTE:
Before upgrading to the latest "Jun 2008" release from any previous release of MonetDB, please make a dump (SQL, XQuery) of your database, and rename/move/backup the database directories (dbfarm, sql_logs, xquery_logs). After the upgrade, restore (SQL, XQuery) your database, again. Once you have checked and confirmed the correctness of the restore, you can delete the above renamed/moved/backed-up old database directories.

Contents

 

Compilation from sources

For convenience of both developers and users as well as to comply even more with open source standards, we now set/use the following defaults for the configure options

	--enable-strict, --enable-assert, --enable-debug, --enable-optimize

When compiling from CVS sources (as mainly done by developers):

	strict=yes  assert=yes  debug=yes  optimize=no (*)

When compiling from packaged/distributed sources (i.e., tarballs) (as mainly done by users):

	strict=no   assert=no   debug=no   optimize=no (*)

For building binary distributions (RPMs, MSIs):

	strict=no   assert=no   debug=no   optimize=yes

With nightly testing for the "Stable" release branches:

	strict=yes  assert=no   debug=no   optimize=yes

With nightly testing for the "Current" development trunk:

	strict=yes  assert=yes  debug=no   optimize=yes

(*)

IMPORTANT NOTE:

 

Since

--enable-optimize=yes

is no longer the default for any case except binary packages, it is

strongly recommended

to (re)compile everything from scratch,

explicitly configured

with

	--enable-debug=no --enable-assert=no --enable-optimize=yes

in case you want/need to run any performance experiments with MonetDB!

Please note:
--enable-X=yes is equivalent to --enable-X, and
--enable-X=no is equivalent to --disable-X.

Windows Installers

There are some changes to the Windows installers since previous releases which are described in this section.

Backup and restore

It is recommended to dump the old database and remove it before installing the new version. After installation, the dump can be restored.

In order to dump the SQL database, start the MonetDB SQL Client program and type the commands

\>...\databasedump.sql
\D
\>

The path after

\>

should be an absolute path name (i.e. start with a drive letter) and be in a save location.

In order to dump the XQuery database, start the server and then connect to the administrative console http://127.0.0.1:50001/admin and use that to dump the database.

By default the database is located in %APPDATA%\MonetDB4 for the MonetDB4/XQuery server and %APPDATA%\MonetDB5 for the MonetDB5/SQL server. After having made a database dump, the latter folder can be removed, but the database dump for MonetDB4/XQuery is made inside the database folder, so remove anything except the folder backup and its contents. This folder is located inside the dbfarm\demo folder.

Restoring the SQL database can be done using the MonetDB SQL Client program with the following command

\<...\databasedump.sql

Restoring the XQuery database can be done from the administrative console.

Installation

The 32 bit Windows installers can be installed "over" the old installation. The installer will automatically remove the old version and install the new. The 64 bit Windows installer will not do this (this time), since the code identifying the package was changed. So on 64 bit Windows, first uninstall the previous version and then install the new. The reason for this change is so that the 32 bit version and 64 bit version can both be installed on a 64 bit platform.

The ODBC driver was removed from the MonetDB5-SQL installer. You now need to install the ODBC driver separately. On 64 bit Windows you should install both 32 and 64 bit versions of the ODBC driver.

Also, make sure that after you have uninstalled the old version or upgraded to the new, the files C:\WINDOWS\system32\lib*.dll are all gone. If not, remove them. The new version does not install any files in the system folder. (To be precise, the files are libstream.dll, libmutils.dll, libMonetODBC.dll, libMonetODBCs.dll, and libMapi.dll.)

The JDBC driver was removed from the Windows installers. The driver is now only available separately.

A new MonetDB5-Geom-Module installer is available which can get installed after the MonetDB5-SQL-Installer has been installed. This module provides GIS support to MonetDB/SQL.

MonetDB Common (GDK)

TODO: mmap, property checks, ...

SQL

TODO: ...

Geometry (SQL/GIS)

This release includes (partial) support for Simple Feature Specification of OpenGIS. Follow the GEOM link above for a small tutorial on it.

XQuery

This release marks the beginning of a new era. The default for MonetDB/XQuery has been changed from using the original Pathfinder compiler (nick-named "milprint_summer" or simply "MPS") to using the new improved and enhanced Algebra-based Pathfinder compiler ("ALG").

Compared to the "preview" in the previous "Feb 2008" release, the functionality of the Algebra-based Pathfinder compiler has been significantly extended to (almost) match that of the original MPS-based compiler. Only the following XQuery features and extensions are not yet supported with the new Algebra-based Pathfinder compiler:

  • fn:put(), fn:collection()
    Please use the old MPS version as described below.
  • recursive functions
    If the recursion depth is known at compile time, adding the option
    	declare option pf:recursion-depth "<number>";
    

    to the query text will ensure that the recursive function is unfolded <number> times. If the recursion is deeper than <number> invocations a runtime error is triggered.
    For arbitrary recursive functions, please use the old MPS version as described below.

  • Arithmetic Functions
    Please use the old MPS version as described below.
  • Prepared Queries
    When using prepared queries, MonetDB/XQuery will automatically switch back to the old MPS version.
  • XRPC Extension
    In case MonetDB/XQuery's XRPC Server is contacted via XRPC (e.g., by the Administrative GUI), the MonetDB/XQuery server will automatically switch back to the old MPS version to handle the XRPC request.
    To use the execute at XRPC Syntax, please use the old MPS version as described below.
  • StandOff Extension
    Pathfinder automatically falls back to the old MPS version, in case StandOff extensions are enabled at runtime via
    Mserver --set standoff=enabled and/or pf --enable-standoff

While support for the old MPS version will not be continued, it remains available for backward compatibility without recompilation, at least for the time being.
For the stand-alone XQuery compiler, the MonetDB/XQuery server, and/or the MonetDB/XQuery client, simply use the following command line options to revert the default to the old MPS version:

  • pf -M
  • Mserver --dbinit='module(pathfinder);' --set xquery_backend=milprint_summer
  • mclient --language=xquery --no-algebra
  • mclient -lxquery -g

In an interactive `

mclient -lxquery

` session, commands "

\g

" / "

\G

" enable / disable the old

MPS

-based compiler in the MonetDB/XQuery server.

Bug fixes

A detailed list of the 139 bug reports that have been closed since the previous "Feb2008" release can be found in the MonetDB Bug Tracker at SourceForge: page 1/3, page 2/3, page 3/3.
The detailed list of changes can be found in the source code CVS logs.

February 2008

February 2008 mk Tue, 03/30/2010 - 20:36

Release "Feb2008" consists of the following updated components MonetDB Common 1.22, MonetDB Clients 1.22, MonetDB4 Server 4.22, MonetDB4/XQuery 0.22, MonetDB5 Server 5.4, MonetDB5/SQL 2.22, and the jars for JDBC and XRPC.

IMPORTANT NOTE:
The database format of this release is not compatible with that of any previous release. Before installing this release, make a dump of your database, and do a restore afterwards.

MonetDB Server This release marks the roll forward of the binary storage layout to a single column storage layout. This involved a major overhaul of the MonetDB Server code. Memory leaks have been addressed.

JDBC & XRPCwrapper The Java code (JDBC & XRPCwrapper) has been moved from the clients and pathfinder source trees to a dedicated java source tree in CVS, which can simply be compiled using "ant". Pre-compiled ".jar" are available for Download.

SQL The functional extensions to SQL are limited. The mostly concern minor bugs and some of the optimizers have been improved to better exploit re-occurring joins. The core cracker code has been cleaned up for better inspection and sharing in the community.

XQuery Next to the original Pathfinder compiler (nick-named "milprint_summer" or simply "MPS"), the development of the new improved and enhanced Algebra-based Pathfinder compiler ("ALG") is progressing quickly. While the original MPS version remains the default in this release of MonetDB/XQuery, a preview of the new ALG version is now available without recompilation. For the stand-alone XQuery compiler, the MonetDB/XQuery server, and/or the MonetDB/XQuery client, simply use the following command line options to select the Algebra version as default:

  • pf -A
  • Mserver --dbinit='module(pathfinder);' --set xquery_backend=algebra
  • mclient --language=xquery --algebra
  • mclient -lxquery -G

In an interactive `

mclient -lxquery

` session, commands "

\G

" / "

\g

" enable / disable the

Algebra

-based compiler in the MonetDB/XQuery server.

 

The

Algebra

-based version of Pathfinder will become the default in the next release of MonetDB/XQuery.

Additionally, the following features have been added:

Algebra & MPS:
  • New Feature: (Restricted) optimistic type checking for functions.
    For every function argument whose type is not promotable to the type in the function definition we check whether the type would be promotable if we would apply fn:zero-or-one to the function argument. If that makes the types compatible we introduce an fn:zero-or-one function call for the argument.
    The effect is that a query writer now can quite often ignore adding fn:zero-or-one function calls (just to make the compiler happy :)).
Algebra:
  • XML import/export for logical algebra plans (pf -I / pf -X). There is however no official description of the used dialect.
  • New Feature: (Restricted) support for recursive functions in the algebra.
    If at query compile time the recursion depth is known the query can be extended with the following option:

    declare option pf:recursion-depth "<number>";

    which triggers the compiler to unfold recursive functions <number> times -- thus supporting recursive functions until depth <number>. If the recursion is deeper than <number> invocations a runtime error is triggered.

  • Implementations of the following functions have been added:
    op:to (e.g., "1 to 10"), fn:concat, fn:string-join, fn:substring, fn:string-length, fn:normalize-space, fn:upper-case, fn:lower-case, fn:translate, fn:contains, fn:starts-with, fn:ends-with, fn:substring-before, fn:substring-after, fn:matches, fn:replace, fn:name, fn:local-name, fn:namespace-uri, fn:error, pf:add-doc, pf:del-doc.

Bug fixes

A summary with the highlights is given below. A detailed list of

closed bug reports

can be found in the MonetDB Bug Tracker at SourceForge. The detailed list of changes can be found in the source code CVS change logs.

SQL

  • 1884382 table dependency inside union...
  • 1865953 local temp table
  • 1855673 bug in xml attribute parsing
  • 1850809 create the same temp table after its drop...
  • 1850789 add and then drop the same constraint...
  • 1850781 alter column to set the same constraint
  • 1850187 length function is not working properly
  • 1848541 select sign...
  • 1848511 select insert....
  • 1848500 octet_length...
  • 1848499 difference function crashes
  • 1842550 Better error reports on bulk load (include line num)...
  • 1841759 insert text with repeat...
  • 1841754 select with full join...
  • 1841749 creation of exception for select crash...
  • 1841734 select with coalesce
  • 1841727 pkey in temp table
  • 1838816 SQL: can't use variable name staring with _digit in SQL func
  • 1838813 SQL: select 5 % 3 crashes
  • 1838785 column with date type
  • 1765919 API:Incomplete block read from stream (Mserver still alive?)

XQuery

  • 1883569 XQ: Algebra
  • 1881018 PF: value clash in compiler/include/algebra.h
  • 1877300 XQ: reduce: Assertion `rule' failed
  • 1872806 PF: weird results of fn:id()
  • 1871790 XQ: uncompilable comparison
  • 1871789 XQ: substitution fails
  • 1866944 Failed to compile current pathfinder

MonetDB Server

  • 1856038 Memory leaks when querying
  • 1856034 Memory leaks when bulk loading
  • 1852518 GDK/M4/PF: GDK assertion in PF/XQ tests since
  • 1842019 BULK load failing for NUMERIC columns

Miscellaneous

  • 1845266 Python converters don't quote Unicode strings...

November 2007

November 2007 mk Tue, 03/30/2010 - 20:37

Release "2007-11-22" (released on Nov 22 2007) consists of the following updatedparts MonetDB Clients 1.20.2, MonetDB5 Server 5.2.2, MonetDB5/SQL 2.20.2.

This release is a bug fix release, covering the following items:

  1. Clients: The APIs for python, perl and php have been synchronized by the latest state of the Mapi protocol and implementation. JDBC contains a non-blocking IO fix for slow network connections.
  2. A performance fix of the group module. Improved grouping functionality significantly through a better estimate of the intermediate sizes.
  3. The MonetDB/SQL embedded version has been made to work again.
  4. Memory leakage in both the Server and SQL module have been addressed.

Bug fixes

List of fixed bugs

 

  • 1834941 cursor.executemany (python) fails
  • 1834442 Compiling clients (Stable) failed on GNU.32.32.d-Gentoo1.12.
  • 1833800 Python errors not getting passed through MAPI
  • 1831630 Clients: Debian gcj/ant doesn't work.
  • 1828314 make failed on MonetDB5
  • 1827627 --enable-bits doesn't work with icc
  • 1826708 Merovingian: cannot start out of the box, as dbfarm doesn't
  • 1823387 Cascade operations in different sessions...
  • 1823378 Incorrect return type of "with - seeded by - recurse"
  • 1819168 sql/src/test/mapi/python_dbapi test fails
  • 1814768 Python Client problems

Release "2007-10-17" (released on Oct 22 2007) consists of MonetDB Common 1.20, MonetDB Clients 1.20, MonetDB4 Server 4.20, MonetDB4/XQuery 0.20, MonetDB5 Server 5.2, MonetDB5/SQL 2.20.

This release marks a long summer of mostly core software development and maintenance. The restructuring of the source tree was partly successful. Semantic dependencies amongst packages, however, forced us to release more components together than originally planned. Close interaction with some large users provided valuable input for bug fixes and performance improvements.

SQL

We added a way to bulk request sequence numbers. When loading (copying) data into tables which have an auto_increment (or alike) column lots of time was spend in the many calls to get all the sequence numbers (observed a performance in crease of about 10x when loading pieces of 100K rows) TINYINT was added to the standard repertoire. We changed how selections over multiple columns of a single table are handled. Furthermore, several bugs stemming from large-scale deployment have been resolved.

XQuery In the XQuery area many issues have been dealt related to the transaction safe update functionality. Furthermore, a value based indexing scheme has been added to speedup the search.

Clients

The mclient utility has undergone a major overhaul. It moved from a Spartan rendering of SQL results towards a more user friendly tabular rendering on ascii terminals. In passing, the option structure and command options has been stratified and extended with a few use full functions, e.g. piping the tabular result as a CSV to an external program. Use the --help for an overview.

MonetDB 5

Merovingian has been added as the gatekeeper of all SQL database services in a particular farm. It redirect clients to the proper connection. The accompanying Monetdb toolkit provides a textual frontend for assessing the status of a collection of MonetDB servers.

Bug fixes

A summary with the highlights is given below. The detailed list of changes can be found in the source code CVS change logs.

SQL

  • 1806869 SQL: prepare output has unqualified table_name header
  • 1790101 SQL: parser warnings
  • 1786908 SQL+PF: loading both "sql_server" and "pathfinder" fails
  • 1775550 SQL: implied conversion from numeric to varchar disconnect
  • 1774312 SQL: conversion from decimal to varchar error
  • 1774217 SQL: order by timestamps column with limit crash
  • 1757923 SQL: Prepared statement type mismatch
  • 1737777 select id not in another select...
  • 1737668 Alter table to add Fkey....
  • 1734143 Update fkey with a values not in the pkey...
  • 1733712 Update pKey with the same value for different rows...

XQuery

  • 1786397 XQ: SAX serialization not working properly
  • 1771532 MIL-optimizer fails sometimes for big queries
  • 1766414 XQ: third pf-add into collection fails
  • 1763575 XQ: insert attribute gives ERROR in merged_union
  • 1763495 XQ: illegal offsets in swizzle (2)
  • 1760811 XQ: illegal offsets in swizzle
  • 1758902 XQUF replace function seems to corrupt the database
  • 1757327 XQ: attributes in 2nd doc don't show
  • 1755466 XQ: printing special characters in comment nodes
  • 1751684 XQ: merged_union error with fn:sum()
  • 1747017 XQ: admin console images not installed
  • 1743442 XQ: illegal XML syntax in serialized attributes
  • 1743433 XQ: problem with ampersand in attribute value
  • 1766173 Double for / position / sub-sequence bug
  • 1760445 PF: test attribute_access_mposjoin_error.SF-1726928 fails
  • 1760168 PF: wrong use of leftfetchjoin causes segfault in bat_cast
  • 1741459 PF: ws_create inconsistently adds a collection node
  • 1740502 PF: live-variable-limit-handling in mil_opt.c buggy
  • 1737735 128MB RAM needed to compile '1'?

Miscellaneous

  • 1774575 Single page manual damaged
  • 1754452 NullPointerException in MonetConnection
  • 1739356 TRACE,PROFILE,DEBUG should be impossible for JDBC and ODBC
  • 1739353 EXPLAIN support from JDBC
  • 1733744 Cannot connect via ODBC (from Powerbuilder 10)

June 2007

June 2007 mk Tue, 03/30/2010 - 20:38

MonetDB 4.18, MonetDB/XQuery 0.18, MonetDB 5.0, MonetDB 5.0/SQL 2.18. The Mars release marks the official release of the MonetDB server 5 line as the basis for future investments in application development. This affects first and foremost the maintenance of SQL code base, where the MIL version is not being maintained actively anymore by the MonetDB team. A synopsis of the major improvements

  1. implementation of delete/update with cascading
  2. PSM procedure call implementation
  3. support for polymorphic functions
  4. autoloading of sql scripts upon server restart

WarningThe SQL catalog structure has been changed, which requires a database dump before you install the new version.WarningThe old Window installers do not always remove MonetDB DLLs. You may have to manually remove them from system32 before (re-)installing the new version. It concerns:

  • libstream.dll
  • libmutils.dll
  • libMonetODBCs.dll
  • libMonetODBC.dll
  • libMapi.dll

MonetDB XQuery XQuery Update Facility has undergone a large series of stress tests to improve correctness, performance and robustness under concurrent updates. In many areas the functionality has been improved:

  1. a completely revised and enhanced type checking code
  2. the administrative console for XQuery
  3. automatic use of value indexing
  4. improved and extended XRpc functionality (javascript library)
  5. all XPath axes looplifted (siblings will come soon)
  6. Intel-based native Windows build (i.e., 64-bit)

This release containsPF/Tijah,, a text search extension for XQuery. It provides a flexible environment for setting up search systems by supporting out-of-the-box solutions for common tasks, like index creation and result ranking of XML elements. Work continuous on the algebra version compiler and its SQL backend code generator.Release model The months passed since the bug-fix release of February 2007 showed that the MonetDB product family has grown to a size that more frequent releases are only possible if we continue our decomposition process. The SQL and XQuery release dependencies have now been cut. Source distribution The Super Source Tarball has become a stable means to access and install the packages on Linux platforms. Bug fixes A detailed list of changes can be found in the source code CVS change logs. A summary with the highlights is given below.MonetDB/SQLBug fixes SQL/Core:

  1. 1734143 Update fkey with a values not in the pkey...
  2. 1733712 Update pKey with the same value for different rows...
  3. 1730987 cast of expression hangs
  4. 1725349 large COPY INTO with primary key
  5. 1724151 SQL: mixing auto-commit with explicit transactions
  6. 1723863 SQL Query Crashes Server - CRITICAL
  7. 1723791 rand() is not executed for every row
  8. 1720934 Bind Variable with CAST function Crashes Server
  9. 1720293 WITH queries works only once!
  10. 1718253 SQL: decimal handling errors
  11. 1715679 CREATE TABLE WITH DATA...
  12. 1715672 LOCAL AND GLOBAL temporary table
  13. 1715669 Temporary tables....
  14. 1715659 LIKE for time values...
  15. 1715553 Set default value.....
  16. 1714829 SQL: CHECK CONSTRAINT...
  17. 1714814 SQL: View creation using a function....
  18. 1714809 Infinite loop....
  19. 1711947 src/test/Update_Delete_action: missing stable output files
  20. 1711251 M5: mserver5 segfaults after the second query
  21. 1711242 M5/SQL: wrong result for range predicate
  22. 1710678 CREATE PROCEDURE...
  23. 1710677 DROP PROCEDURE.....
  24. 1709182 SQL: M4 and M5 crash on Solaris/x86 when SQL module is loaded
  25. 1703497 The mserve crash due the null control...
  26. 1701823 copy into from 'filename' not closed
  27. 1695290 SQL changing column default gives error
  28. 1691646 REVOKE authorization crash
  29. 1691344 SQL UNION with ORDER BY crash
  30. 1691098 Mserver/SQL crashes on empty window_specification
  31. 1688915 SQL: crash on nested query
  32. 1684798 LIKE clause
  33. 1670739 SQL: Corrupt database after restart
  34. 1659889 SQL: sequence not restored properly

Bug fixes XQuery:

  1. 1731850 Error in preceding axis
  2. 1730556 XQ/JDBC: 250 results
  3. 1730547 XQ: Mserver crashes on concatenated query
  4. 1726954 XQuery: inserting (large) doc into other doc
  5. 1726928 XQuery: Attribute access causes mposjoin error
  6. 1726599 XQuery: collection management broken
  7. 1726548 XQuery: Administrative Console
  8. 1725352 PFtijah: failing alloc() causes assertion failure (segfault)
  9. 1724118 PF: coreopt optimization eats all available memory and swap
  10. 1723931 XQuery compiler warning treated as error
  11. 1723217 PF: compiler/sql/lalg2sql.brg fails to compile w/o assertion
  12. 1722078 PF: WebSite tests books[123] fail since 2007.05.03
  13. 1721701 PF: "!ERROR: Division by zero" after latest checkins
  14. 1719164 PF: new loop-lifted following step still buggy?
  15. 1718709 PF: select deleted node results in error
  16. 1718622 PF: strange behaviour of "rename"
  17. 1718066 PF: put() to /tmp crashes Mserver
  18. 1716158 PF: compiler/algebra/opt/opt_thetajoin.c icc-compil. fails
  19. 1712578 Mix-up in nested element construction
  20. 1706520 XQuery: boolean every
  21. 1705898 "forgot to cope with type 'double' in cast to integer"
  22. 1682059 XQ: internal integer representation
  23. 1698498 id-function fails to locate some kinds of nodes
  24. 1684416 Pathfinder does not compile due to undefined LLFMT
  25. 1718709 PF: select deleted node results in error
  26. 1718635 PF: node's sibling not accessable after deletion of the node
  27. 1718622 PF: strange behaviour of "rename"
  28. 1718066 PF: put() to /tmp crashes Mserver
  29. 1714603 PF: XRpc tests fail on MacOS (Darwin) since
  30. 1714399 XQuery/XRPC: refresh admin-page causes segmentation fault
  31. 1714360 PF/runtime: find_qn_bulk() adds new qnames twice
  32. 1712595 Error mposjoin: missing matches
  33. 1711754 shredder msync() bug
  34. 1706438 PF: property error when adding a doc. to an existing coll.
  35. 1683730 XQuery: mserver (0.16.2) sometimes crashes on complex query
  36. 1681921 XQ: Character escaping is bogus
  37. 1678593 XQ: document added/deleted with error occured
  38. 1676443 XQ: Interleave of put and doc via mod. func. crashes
  39. 1719164 PF: new loop-lifted following step still buggy?
  40. 1712463 PF: various tests fail since 2007.05.04
  41. 1712461 PF: several tests fail since 2007.05.03
  42. 1712459 PF: several tests fail since 2007.04.28
  43. 1712458 PF: prop. err. in update tests since 2007.04.26 (64-bit OIDs
  44. 1712456 PF: various tests fail since 2007.04.04
  45. 1712455 PF: test fails since 2007.04.03
  46. 1712454 PF: tests/W3C_use_cases/XQ/XMP/Q04alg fails since 2007.04.03
  47. 1712449 PF: "bogus_text_step" test fails since 2007.03.08
  48. 1714133 PFtijah: "/tmp/TIJAH_LOGGER" never cleaned => fills /tmp/ !
  49. 1714129 PFtijah: hard-wired "/tmp/TIJAH_LOGGER" not portable

Bug fixes APIs:

  1. 1733744 Cannot connect via ODBC (from Powerbuilder 10)
  2. 1675795 Float to decimal conversion still broken in v5.0.0_beta1_2
  3. 1671737 JDBC: exception in getChallengeResponse
  4. 1670732 XQ/JDBC: escaping backslash
  5. 1693694 ODBC driver not honoring host, username, or password
  6. 1473288 SQLTables fails if previous statement failed
  7. 1702698 Mapi < problem

Bug fixes Kernel:

  • 1713584 delete on committed dense persistent BAT causes prop. error
  • 1709519 M5: Linux process killed
  • 1706713 Database corruption after out-of-memory condition
  • 1705726 crash in leftfetchjoin due to lack of range checking
  • 1704972 GDK change causes FATAL BATpropcheck with XQuery
  • 1686756 Implicit declaration of function .fdatasync.
  • 1679800 Mserver segfaults during recursive module loading
  • 1673410 STREAM_DEBUG causes compilation error in stream.mx

January 2007

January 2007 mk Tue, 03/30/2010 - 20:39

MonetDB 4.16, MonetDB/SQL 2.16, MonetDB/XQuery 0.16, MonetDB 5.0.0_beta1, MonetDB 5.0.0_beta1/SQL 2.16.

A bug fix release was made available on Feb 26, 2007. See the notes at the end of this paper.

The Venus release represents a major shift in functionality. First, we released the long awaited for XQuery Update Facility. As far as we know, it is the first W3C compliant XUF implementation available and provides full transaction control.

The next step was the

big bang

change in the packaging of the server layer. The successor, MonetDB Version 5 is moved to SourceForge. The MonetDB/SQL software line is available with both back-ends, but for Version 4 it is the final release. Finally, redesign over the website has lead to a better navigational structure and much more information on the product lines. A detailed list of changes can be found in the source code CVS change logs. A summary with the highlights is given below.

Source distribution MonetDB Version 5.0.0_beta1 was prepared and shipped to SourceForge.

SQL front-end The SQL functionality has been extended to support SQL triggers and fully functional Persistent Stored Modules. This extension was inspired by our ongoing port of the Skyserver application. Dependencies between SQL objects are stored now and dropping objectsrespect the 'strict' object dependency rule. Loading large tables have been significantly improved and the heuristic query optimizer covers a much larger space.

XQuery This release contains an implementation of XUF (XQuery Update Facility) based on the W3C working draft. In addition, the release adds a number of significant other features to XQuery:

  • "multi-document collections" allow much more efficient management of XML databases that consist of *many* (small) documents, a weakpoint of the previous releases.
  • XRPC: distributed query processing between multiple MonetDB servers; also useful as SOAP API.
  • document management interface: the XML document database can now be controlled fully from XQuery (no MIL needed anymore).
  • improvements in query performance.

Kernel improvements The memory mapped file functionality has been signficantly changed to accomodate efficient processing of XQuery/XUF.

Documentation The revamped website became alive.

Bugfix release Feb 26,2007

New versions of all components of the MonetDB suite are released.

  1. MonetDB/XQuery
    The handling of updates was incomplete. Several memory mapping and concurrency problems have been fixed. Moreover, empty pages that resulted from deletion of nodes are now properly handed back to the system for reuse.
  2. MonetDB/XRPC
    Security and concurrency handling has been improved.
  3. MonetDB/SQL
    Foreign key constraint checking fixed; View in view dependencies bug fixed; Resolved crashes caused by large views; Prepared statements with more than 30 arguments now work.
  4. MonetDB/API
    Haskell and TCL libraries were broken and have been removed; Perl and Python libraries were broken and have been fixed. The current version of the protocol is not compatible with the previous version.
  5. Bugs fixed:
    [ 1579510 ] XQ: update error for "(1,2)/."
    [ 1590523 ] XQ: substring-after
    [ 1636828 ] fork() behaves strangely
    [ 1637867 ] XQ: seqcast int? to numeric unsupported
    [ 1639108 ] mserver crashes on delete query
    [ 1640248 ] PF: ~500 tests randomly fail on Solaris x86
    [ 1644245 ] M5 & SQL5: various tests hang/timeout on Darwin
    [ 1648303 ] database dump does not save/restore auto-increment sequences
    [ 1650469 ] SQL: Armada sinks MonetDB ships
    [ 1651567 ] yet another delete bug, foreign key related
    [ 1651599 ] caching of inserts causes extreme memory consumption
    [ 1655818 ] SQL:Error inserting float in numeric column using JDBC
    [ 1655857 ] XQ: call to builtin function crashes XRPC client
    [ 1659889 ] SQL: sequence not restored properly
    [ 1644610 ] Conflict between normal query and XRPC query
    (and many more that hadn't been reported)

November 2006

November 2006 mk Tue, 03/30/2010 - 20:39

MonetDB 4.14, MonetDB/SQL 2.14, MonetDB/XQuery 0.14, MonetDB 5.0_beta1

This release was created in November 2006, but abandoned a few weeks later as the code base was too unstable to warrant a public release.

June 2006

June 2006 mk Tue, 03/30/2010 - 20:40

MonetDB 4.12, MonetDB/SQL 2.12, MonetDB/XQuery 0.12, and MonetDB/Five Alpha Release 1 were released on June 23, 2006.

This MonetDB release marks the end of a period where much attention was given to extend the XQuery/Xupdate functionality and efficiency. Although not finished, the many code improvements are valuable to the extended client base. Furthermore, this release marks a feature freeze for XUpdate functionality, claiming enough time to asses its stability and performance before an official release is announced. You may encounter some loose ends and rough edges. A detailed list of changes can be found in the change log of the CVS repository. A summary with the highlights is given below.

Source distribution The MonetDB Version 5 Alpha Release was prepared. It shares the SQL front-end with version 4.

SQL front-end Preliminary support for triggers was added. Now local and global temporary tables are supported. They caused a large rewrite of the session/transaction management. Next to these improvements, many small and large bugs have been fixed. Most notably are fixes to key constraint handling and concurrent transactions.

XQuery All open bugs of XQuery have been addressed, significantly improving its functionality. Serialization, a critical component in any XQuery implementation, has been revamped to obtain a 5-10x speedup in this area. Special attention has been given to parent/ancestor steps. Now they work many times faster inside for-loops (have been loop-lifted) Furthermore, we added XPath extensions for XML standoff annotations (--set standoff=enabled as runtime option to Mserver)

The development of XUpdate was set back by the necessary changes to the underlying MonetDB engine to support BATs with holes in the void sequences. More testing is needed to remove the rough edges.

APIs
The PHP library on Linux has been revamped. It is now based on the PostgreSQL PHP library code.

Kernel improvements The BAT recycle cache has been completed. This greatly improve the performance for applications that create/destroy an abundance of small BATs.

The transaction semantics of the kernel has been extended to support a sub-commit of BATs. This prepares the road to combine SQL and XQuery in one Mserver in the future.

Windows port Cross- and native- compilation for 64-bit Windows platforms versions has somewhat be improved, but is not complete yet. The distribution section will shortly contain a 64-bit version for download.

Documentation improvements Some minor changes, but still an area with a potential great return-on-investment.

January 2006

January 2006 mk Tue, 03/30/2010 - 20:40

MonetDB 4.10, MonetDB/SQL 2.10 and MonetDB/XQuery 0.10 were released on Jan 24, 2006. This MonetDB release marks the end of a period where much attention was given to support simple SQL and XQuery applications efficiently. These improvements came from careful analysis of student programs and participating in the c't Magazine DVDstore benchmark.
A detailed list of changes can be found in the change log of the CVS repository. A summary with the highlights is given below.

Source distribution The MonetDB software base has been split to ease source distribution. The tools to prepare the source compilation (Mx), source code generators (MEL, Burg) and makefile generators (autogen) are assembled in the buildtools package. They are not needed anymore to recompile the source code distributed. It involved major changes to the configuarion scripts. A complete clean rebuild is advised.

SQL front end The SQL catalog management has been completely overhauled. It provides much better performance for lightweight queries and provides a cleaner interface for transaction management. All changes to a base table are assembled in insertion/deletion BATs, which can be merged at transaction commit with the base tables.

Sequences support has been added along the specification of SQL 2003. This allows for identity columns, serials (PostgreSQL) or auto_increments (MySQL) in MonetDB. Several minor changes to the build-in operators and syntax recognition, e.g. uchr has been dropped and temporal support has been improved.

SQL server's default transaction mode is now AUTO COMMIT. Support for SET AUTO_COMMIT=true/false has been removed and has been replaced with the lower level Xauto_commit statement issued from the communication libraries. AUTO COMMIT can be switched on and off using the SQL statements START TRANSACTION and COMMIT or ROLLBACK.

There were many small changes to the SQL back-end, such as private name space for cached queries and a large cleanup of the SQL result set code. The protocol for sending results set has been optimized to reduce the communication overhead.

Most BAT related code has been moved to their own directory to prepare for future different storage back-ends. This directory also contains the transaction and logger code. The transaction management has been inproved to allow for reusing the structures which store the transactions catalog information.

XQuery MonetDB/XQuery performance has been boosted using an index node and 'prepared' query/result caching. It now also supports XQuery module definitions. The performance of small XQueries has noticeably improved.

APIs
The experiments with the c't Magazine benchmark led to a re-design of parts of the MAPI protocol, aiming for less overhead in header analysis and falling back to a single exchange mode (blocked) for JDBC. It proofed to increase performance with at least a factor two.

JDBC has been largely revamped for additional speed improvements. Support for retrieval of more metadata has been added as well. The stage has been prepared to also support server-side redirections for future use.

The Perl DBI library has greatly be improved. It uses the MAPI functionality directly now. The module has been registered at the official Perl archive (CPAN)

ODBC has been upgraded to align with the protocol changes and also supports native (server side) prepared statements.

PHP, Python: No changes.

MIL The MIL internal representation (YYTREE) has been overhauled to improve execution of small XQuery operations. Minor language changes, e.g. the module statement is limited to a single argument.

Kernel improvements The BAT buffer pool re-uses the file names of BAT locations and the default temporary names. This improves handling of small BAT creations. The code has been prepared for re-use-garbage in the near future.

A constant module has been added. It supports BAT operations with a column designated by a scalar value.

Windows port A new windows installer preparation scheme under Linux has been deployed.

Documentation improvements Some minor changes, but still an area with a potential great return-on-investment.

May 2005

May 2005 mk Tue, 03/30/2010 - 20:41

MonetDB 4.8 & MonetDB/SQL 2.8 were released on May 29, 2005. They were accompanied by the first MonetDB/XQuery 0.8 release (the Mercury release). A detailed list of changes can be found in the change log of the CVS repository. A summary with the highlights is given below.

SQL front end Added SQL statement modifiers EXPLAIN, PROFILE and PREPARE/EXEC. Explain returns the query execution plan as a MIL program and the query is not executed. PROFILE executes the query and stores profiling information in the 'history' table. The prepare and execute work together and can be used by the client programs to implement PREPARE and EXECUTE functionality. The SQL module functionality has been included . They can be used to introduce a module written in C or a plain SQL module. Modules can add new types, functions and tables to the system. [TODO] See for example the 'url' example module.

JDBC driver

  • JdbcClient
    • support for reading (gzipped) files from http web locations
    • better dumping support for time zones on temporal types
    • improved warning feedback system
  • Connection
    • client time zone support for support of INSERT INTO queries which deal with time zones
    • update to intermediate Mapi authorisation scheme with support for multiple catalogs if the server supports it
    • sliding window fetch size which improves speed on large result sets
    • various small changes to improve overall processing speed (not necessarily simplifying the code)
  • DatabaseMetaData
    • updated actual limits of the database measured in maximum column lengths
  • Driver
    • added support for multiple catalogs
  • PreparedStatement
    • fixed bug with Float processing (returned null)
  • ResultSet
    • various small fixes to improve performance
    • preliminary support for retrieving Clobs
    • fixed bug with Float processing
    • fixed time zone behaviour (also made it more efficient)
  • MonetSocketBlockMode
    • made small optimalisation in internal buffer management
  • Statement
    • if fetchSize is not set, a sliding window fetch size is now being used
  • Clob
    • preliminary implementation added

Application interfaces The PHP interface has been finalized. The Perl DBI library interface has reached a mature level.

Kernel improvements The kernel underwent some renovation to improve its usefulness in future releases. The BAT storage structures have been trimmed to consume just 1/3th of the previous space, which can be used to reduce the overhead of short queries by caching all descriptors in main memory. The BAT structure also underwent a revision to accomodate a more flexible property maintenance list. The changes have be shown to improve processing of 'compiled' small MIL queries to run up to 30\% faster.

Windows port The windows port can now be build both using Microsofts Compiler suites and using MinGW.

Documentation improvements The documentation for PHP5 installation and interaction has been added.

February 2005

February 2005 mk Tue, 03/30/2010 - 20:42

MonetDB 4.6.0 & MonetDB/SQL 2.6.0 was released on February 14th, 2005, (the Valentijn release). Since the previous release, many improvements have been made. A detailed list of changes can be found in the change log of the CVS repository. A summary with the highlights is given below.

SQL front end MonetDB/SQL now survives the famous 'crashme' program used by MySQL to analyze the features of a DBMS implementation.

Several improvements with regard to the SQL integrity and compliance with the standard were made, as well as new supporting features:

  • The LIKE operator has been overhauled for better support for search expressions
  • Closer conformance to the SQL standard for types
    • INT(x) is no longer allowed/supported
    • SMALLINT, INT and BIGINT are now supported 'native' types
    • convenience aliases for TINYINT (mapped on SMALLINT) and MEDIUMINT & INTEGER (mapped on INT)
    • CHARACTER VARYING and VARCHAR now need a mandatory length specification, ie. VARCHAR(x)
    • added support for CHARACTER LARGE OBJECT next to CLOB
  • Constraints that are not given a name explicitly now get an explanatory name, ie. mytable_mycolumn_pkey instead of c102773
  • Improved dumping support for MapiClient: now constraints and indexes are also dumped
  • Various fixes and optimizations in JDBC and JdbcClient
    • resolved a bug where the JDBC driver would hang if the server would stop sending results prematurely
    • improved support for types, ie. better and more consistent mapping from server to JDBC types
    • Newly implemented java.sql.ResultSet methods: getCatalogName(), isCaseSensitive(), isSearchable(), isCurrency(), isSigned() and getColumnClassName()
    • JdbcClient now correctly escapes identifier names in database dumps and describe actions
    • added -e of --echo flag to JdbcClient to echo input while reading from a file: results in a clear dialog, instead of answers only
    • small optimizations in the socket code, amongst others: avoidance of unnecessary memory copy, skipping of empty blocks.
    • SQLWarnings are now added to the warning lists of the Connection, Statement, and ResultSet objects for various problems or inconsistencies encountered by the driver, that are not fatal.
  • INTERVAL works with TIME and TIMESTAMP
  • Improvement of SQL parser errors: can lead to exact specification of what is expected in certain cases

Mbedded The code base has been reduced and the configuration parameterized to allow MonetDB to be linked with your application. The smallest system we have been able to run MonetDB was a single-board Linux computer, called the Gumstixs.

Application interfaces The MapiClient command line tool has been extended to enable automatic timing. Reading scripts as part of the interaction is now supported.

Steffen Goeldner joined our team and completely revamped the Perl DBI library interface.

JDBC: see above (section "SQL front end").

MIL cleanup The MIL syntax cleanup, started in version 4.4, was finished. The parse now denies all "legacy" (i.e., pre 4.4.0_rc0) syntax alternatives.

Kernel improvements MonetDB was known to crash on for example large cross-product calculations. This has been prevented by better boundary checking code at several places.

Documentation improvements The MonetDB source code has been largely moved to texi as a preparation step to improve documentation generation.

Code base improvements

Source code indentation has been normalized throughout the code base.

September 2004

September 2004 mk Tue, 03/30/2010 - 20:42

Version 4.4.2 was released on October 26, 2004. It is primarily a bug-fix release.

SQL front end Several bugs affecting the SQL integrity were fixed:

  • Unique constraints were not checked in auto-commit mode
  • The LIKE operator returned wrong matches on patterns starting with %
  • Deleted rows reappeared on conditional selections
  • Removal of legacy variables in system tables
  • ALTER TABLE in auto-commit mode generated error "Error in result"
  • Leakage of BATs caused crashes after heavy load with updates

ODBC and JDBC JDBC suffered from a bug when non-ASCII characters were used in queries. Due to an incorrect charset handling the driver locked up. Another bug returned the incorrect username when being asked for the current logged in user using JDBC's DatabaseMetaData.getUser().

Synopsis of features available in Version 4.4.0

Version 4.4 was released on September 30, 2004 and the first official release. It was preceded by version 3.14, 3.16 and 3.18 delivered in 2004, which were all focused on reaching the point where researchers and casual users would benefit from the MonetDB solution. The feature list below summarizes the efforts spent.

SQL front end A fairly complete SQL-99 front end has been constructed. Amongst others, it provides schema management, user authorization, view definitions, referential constraint enforcement, time zone support, sub-queries, and query caching to speedup processing.

ODBC and JDBC The predominant application programming languages are supported using the MonetDB Application Programming Interface (MAPI) library. Amongst others, it is used to construct a functional rich ODBC implementation, which allows you to retrieve data from a MonetDB server from inside an Excel spreadsheet. The JDBC code is a class 4 implementation. It has been used to link MonetDB with other database application development environments.

MIL cleanup The MonetDB Interface Language (MIL) was originally designed as an intermediate and debugging language. Its definition has been brought more in line of a real programming language, without loosing its flexibility to write concise scripts.

Kernel improvements Heavy internal use revealed bugs and performance issues. In combination with the lengthly bug list compiled by our commercial partner over the years, we have reached the point that serious crashes have become a rarity.

Documentation improvements Documentation in a research setting often comes last, but we are confident that the information provided is sufficient for both a casual end-user, building an application like the VOC, and the demanding researcher, interested in our research on cache-conscious code.

Testing MonetDB is tested daily on 17 different platforms, configured with widely different hardware profiles, but also a variety of compilers. The MonetDB code base was cleaned up to the point that all compiler warnings were dealt with. Currently, it compiles with the strongest compilation options available. Testing has reached the point that a day with 24 hours is not enough to run and analyze the test suite.

Packaging Distribution of a software package as rich and complex as MonetDB calls for major (human) investments in software engineering tasks. The software heavily relies on open-source packages such as automake, auto-config. For the Windows platform the necessary files are included as well.