January 2006January 2006 mk Tue, 03/30/2010 - 20:40
MonetDB 4.10, MonetDB/SQL 2.10 and MonetDB/XQuery 0.10 were released on Jan 24, 2006. This MonetDB release marks the end of a period where much attention was given to support simple SQL and XQuery applications efficiently. These improvements came from careful analysis of student programs and participating in the c't Magazine DVDstore benchmark.
A detailed list of changes can be found in the change log of the CVS repository. A summary with the highlights is given below.
Source distribution The MonetDB software base has been split to ease source distribution. The tools to prepare the source compilation (Mx), source code generators (MEL, Burg) and makefile generators (autogen) are assembled in the buildtools package. They are not needed anymore to recompile the source code distributed. It involved major changes to the configuarion scripts. A complete clean rebuild is advised.
SQL front end The SQL catalog management has been completely overhauled. It provides much better performance for lightweight queries and provides a cleaner interface for transaction management. All changes to a base table are assembled in insertion/deletion BATs, which can be merged at transaction commit with the base tables.
Sequences support has been added along the specification of SQL 2003. This allows for identity columns, serials (PostgreSQL) or auto_increments (MySQL) in MonetDB. Several minor changes to the build-in operators and syntax recognition, e.g. uchr has been dropped and temporal support has been improved.
SQL server's default transaction mode is now AUTO COMMIT. Support for SET AUTO_COMMIT=true/false has been removed and has been replaced with the lower level Xauto_commit statement issued from the communication libraries. AUTO COMMIT can be switched on and off using the SQL statements START TRANSACTION and COMMIT or ROLLBACK.
There were many small changes to the SQL back-end, such as private name space for cached queries and a large cleanup of the SQL result set code. The protocol for sending results set has been optimized to reduce the communication overhead.
Most BAT related code has been moved to their own directory to prepare for future different storage back-ends. This directory also contains the transaction and logger code. The transaction management has been inproved to allow for reusing the structures which store the transactions catalog information.
XQuery MonetDB/XQuery performance has been boosted using an index node and 'prepared' query/result caching. It now also supports XQuery module definitions. The performance of small XQueries has noticeably improved.
The experiments with the c't Magazine benchmark led to a re-design of parts of the MAPI protocol, aiming for less overhead in header analysis and falling back to a single exchange mode (blocked) for JDBC. It proofed to increase performance with at least a factor two.
JDBC has been largely revamped for additional speed improvements. Support for retrieval of more metadata has been added as well. The stage has been prepared to also support server-side redirections for future use.
The Perl DBI library has greatly be improved. It uses the MAPI functionality directly now. The module has been registered at the official Perl archive (CPAN)
ODBC has been upgraded to align with the protocol changes and also supports native (server side) prepared statements.
PHP, Python: No changes.
MIL The MIL internal representation (YYTREE) has been overhauled to improve execution of small XQuery operations. Minor language changes, e.g. the module statement is limited to a single argument.
Kernel improvements The BAT buffer pool re-uses the file names of BAT locations and the default temporary names. This improves handling of small BAT creations. The code has been prepared for re-use-garbage in the near future.
A constant module has been added. It supports BAT operations with a column designated by a scalar value.
Windows port A new windows installer preparation scheme under Linux has been deployed.
Documentation improvements Some minor changes, but still an area with a potential great return-on-investment.