developers-list

Download

developers-list@monetdb.org

January 2006

11 participants
12 discussions

Re: [Monetdb-developers] RFC: Upcoming release of MonetDB 4.10: Burkowski Code
by Stefan Manegold 19 Jan '06

19 Jan '06

On Tue, Jan 17, 2006 at 10:47:33PM +0100, Martin Kersten wrote: > > > Wouter Alink wrote: > >My opinion, for what it is worth... > > >I would like to ask to leave the code in, because it (probably) would not > >harm anybody, and I would be happy. > But are you planning to stick to the stable release then? or > do you switch to the development head directly? > Does your users work with that stuff from the release? > > I think that it can be removed from the release and stay > in the development branch. This would make you happy and > does not distract outsiders to use it too early. Exactly that was my proposal: leave the Burkowski code in the development runk, but keep it out of the release. Since the release will not be configure with `--enable-burk` (most; see * below) of the respective code will not be compiled and the features will not be usable with the (binary) release, anyway. Moreover, if I understand the comments of Jan & Jens (separate mail discussion between Wouter, Jens, Jan & me) correctly, the current burK implementation/integration does cause some significant(?) performance penalty in the burG generated code, which will harm the performance of XQuery translation significantly(?). Hence, unless Wouter has some clients that need to use the release with Burkowski features (i.e., self-compiled from the sources), I make a strong point to keep the Burkowski code out of the release --- the next release (then with Burkowski code) will (must ;-)) be ready within 2 months from now. (*) "Even" for the development trunk, I'd suggest to try an use the macro BURKOWSKI (and an AC_CONDITIONAL that we should add), that is/are set with the --enable-burk configure switch, to remove ALL Burkowski related code from the compilation; i.e., (if possible) also the declarative parts in compiler/algebra/core2alg.brg, compiler/core/coreopt.brg, compiler/core/simplify.brg, compiler/semantics/typecheck.brg, as well as the changes in compiler/parser/parser.y, compiler/parser/scanner.l, & runtime/pathfinder.mx. In fact, Jan & Jens suggest to even reduce the changes in (at least) compiler/algebra/core2alg.brg to a minimum. Maybe, Jan & Jens can help with the *.brg file (also optimizing the indices), I could help with runtime/pathfinder.mx, and the AC_CONDITIONAL to avoid compilation of pf_burk unless configure with --enable-burk. If we agree on keeping Burk out of the release, we have almost tow month to implement these improvements in the development trunk ;-) Kind Regards, Stefan ps: the release must be ready by Sunday Jan 22, 23:59:59, hence a quick reply is appreciated; sorry for the hurry, but "external constraints" forced us on short notice to finish the release a few days earlier than planned ... -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 2

[Monetdb-developers] RFC: Upcoming release of MonetDB 4.10
by Stefan Manegold 17 Jan '06

17 Jan '06

Dear Developers, Enclosed you find the draft cover letter for the upcoming MonetDB release. Please provide any comments that from your perspective are relevant to mention. In preparing the distribution content we have finally decided not to include anything of the Xupdate functionality, SOAP, or other XML modules. The primary reason is that implementation and validation of these components have not reached a point were we are confident of their stability. These issues will be moved forward to the next release planned for March 2006 when the system will be presented at EDBT. Nevertheless, we can be proud of the progress made. Especially interaction with the server using a large number of small SQL update queries has been boosted to the point where we are in the same arena as MySQL and PostgreSQL using JDBC connectivity. The code freeze date is Friday, Jan 13 2006. We plan to release MonetDB v4.10 on Tuesday, Jan 24 2006. Regards, The MonetDB Development Team ------------------- Cover release 4.10 ---------- The MonetDB release 4.10 marks the end of a period where much attention was given to improving the performance of simple SQL applications. These improvements came from careful analysis of student programs and participating in the "c't" Magazine's database contest featuring the DVDstore benchmark. It led to re-consideration of MAPI -- the communication protocol -- aiming for less overhead and complex structures to be parsed. To facilitate this, JDBC has been largely be rewritten at core level to streamline the communication flow even more. Not only JDBC has benefited from the changes made, all MAPI driven communication has been sped up. A second Perl library has been prepared, which more directly uses the MAPI library. It has been registered at the official Perl archive for distribution (ICAN). ODBC has been upgraded to support server side prepared statements like JDBC. More meta-data is now available for columns in JDBC. MonetDB/SQL has been extended with SQL:03's sequences support. This allows for identity columns, serials (PostgreSQL) or auto_increments (MySQL) in MonetDB. A complete rewrite of the catalog and transaction handling removed the overhead encountered in small interactions. The SQL transaction scheme is based on three BATs (stable,inserts,deletes) which are merged at the end of a transaction. MonetDB/XQuery performance has been boosted using query/result caching and a rewrite of the code to manage the internal representation of MIL programs. Further, MonetDB/XQuery no also supports XQuery module definitions. XQuery is currently actively used in large scale applications and is stable. The MonetDB software base has been split to ease source distribution. The tools to prepare the source compilation (Mx), source code generators (MEL, Burg) and makefile generators (autogen) are assembled in the buildtools package. They are not needed to recompile source code distributed. ! TODO: ! Actually make the latter work. ! For now also the source distributions (still) require the buildtools package. In the kernel area we have added the 'constant' module. It provides functions to work with BATs whose tail consist of a constant value. During this period we have solved numerous bugs reported by our users. In addition we organized 3 bug testing days to encapsulate the problems reported into scripts that became part of our nightly testing scheme. Several smaller performance improvements have been made to the inner core of the server. One semantic error has been the cause of confusion. Bats with signature [void,void] are now allowed. A final clean-up of the respective semantic is currently being implemented; a detailed description of the changes will follow soon. Lack of manpower, expiring access to external machines, ceasing system maintenance and decrepit hardware forced us to stop nightly testing on IBM AIX, SGI IRIX and Sun Solaris. Thus, all active support for these (aging) platforms had to be suspended, too, but could easily be revived on request, once we (re-)gain access to such systems. Higher on our wishlist are more modern systems, especially 64-bit Windows and 64-bit Mac OS X. We welcome any donation in this area: access to respective systems, hardware, and/or (especially in case of the pending 64-bit Windows port of MonetDB) development-expertise and -time. The next release of MonetDB version 4 is scheduled for March 2006. By that time we also plan to release the alpha version of the next generation of MonetDB, version 5. Regards, The MonetDB Development Team -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

5 12