[Monetdb-developers] Feature Freeze for upcoming MonetDB-4.14, MonetDB/SQL-2.14, MonetDB/XQuery-0.14

Stefan Manegold Stefan.Manegold at cwi.nl
Wed Nov 8 01:19:09 CET 2006


Dear all,

(almost) as announced I spent tonight on creating the CVS branches from the
upcoming MonetDB/* release.

"Almost", because I failed to start at 18:00 (sharp), but did only start at
19:22, a delay which I sincerely regret (see below).


*WARNING*:
==========
I strongly dissuade all users and all developers that are not involved in
debugging Pathfinder from checking out / upgrading to the latest version of
Pathfinder (neither this new release branch nor the development trunk).
Preliminary "sanity checks" indicate that the code is SEVERELY BROKEN, at
least *most* of the 665 tests in runtime// and tests// do fail since tonight
(I haven't checked the tests in benchmarks// and compilers//, yet).

See topic "Pathfinder" below.


*PLEA*:
=======
I kindly ask all developers that did change Pathfinder code today
(in particular later afternoon and evening) to check the test result
emails and testweb tomorrow morning (or even run Mtest themselves)
and fix the (new) BUGs A.S.A.P., i.e., by no later than tomorrow night.



General Details:
================

As usually, I first tagged the current development trunk versions as follows
	MonetDB, buildtools, template, TestTools (CWI internal):
		"MonetDB_4-14_root"
	sql:
		"SQL_2-14_root"
	pathfinder:
		"XQuery_0-14_root"
	monetweb (CWI internal):
		"MonetDB_4-14_old_root"

and then created the following release branches:
	MonetDB, buildtools, template, TestTools (CWI internal):
		"MonetDB_4-14"
	sql:
		"SQL_2-14"
	pathfinder:
		"XQuery_0-14"
	monetweb (CWI internal):
		"MonetDB_4-14_old"

Version numbers in both the release branches and the development trunks are
increased accordingly.
In MonetDB/conf/monet.m4, I set the configure defaults as follows
	MonetDB_4-14 release branch:
		dft_strict=no
		dft_assert=no
		dft_optimi=yes
		dft_warning=no
		dft_netcdf=no
	development trunk (unchanged):
		dft_strict=yes
		dft_assert=yes
		dft_optimi=no
		dft_warning=no
		dft_netcdf=auto

For ocnvenience of later reference, the code version after increasing the
version numbers and setting the defaults are tagged as
	MonetDB, buildtools, template, TestTools (CWI internal):
		"MonetDB_4-14_" (release branch)
		"MonetDB_4-15_" (development trunk)
	sql:
		"SQL_2-14_" (release branch)
		"SQL_2-15_" (development trunk)
	pathfinder:
		"XQuery_0-14_" (release branch)
		"XQuery_0-15_" (development trunk)

		
As usually, as of now
* all bug-fixes (and only these) must go to the new release branchs
* all new features must go to the development trunks (and only there).

Propagation of bug-fixes from the release branchs to the development trunks
will be handled by Sjoerd or me (only) on an irregular basis (and possible
on request or need).



Pathfinder:
===========

Since I was appointed the "release master", I felt the responsibility to not
only "mechanically" make the branches, but also to apply minimal "sanity
checks" to ensure that what ever I provide you with is at least not "worse"
than the it was before.

Having done that, I consider if my duty to inform you as neutrally as
possible about the problems I found, and could not solve, yet ---
I intend no blames or accuses at all!

Knowing that there were no critical changes on buildtools, MonetDB & sql
during the day (and last nights testing as well I my tests during the day
revealed no major problems with them), I focused my final "sanity checks"
on pathfinder. Compilations worked fine, but running the tests in 
runtime// and tests// resulted in (almost) all of the 665 tests failing,
while only 90 failed earlier this afternoon --- I haven't tested the
remaining ~1000 tests in benchmarks// & compiler//, yet.

To me, this suggests that the current code base is SEVERELY BROKEN.

A quick analysis revealed that the following checkins trigger the problems:
===================================================================
2006/11/07 - boncz: compiler/mil/milprint_summer.c,1.298
        compiler/semantics/xquery_fo.c,1.89 runtime/pathfinder.mx,1.268
        runtime/pf_support.mx,1.172 runtime/serialize_dflt.mx,1.19
        runtime/serialize_sax.mx,1.8
- remove some unnecessary void casts from serialize
- implemented information functions (extended query locking infrastructure)
===================================================================
2006/11/07 - boncz: runtime/pathfinder.mx,1.269
- streamlined startup (pfstart() is no longer needed)
  NOTE: you must start pathfinder (i.e. 'module(pathfinder);') now in the
Mserver always!!!
        doing this in MIL programs on each MapiClient session only (and not
beforehand in the server)
        will produce errors on multiple/concurrent sessions

- added a checkpointing thread
===================================================================

I shortly considered undoing these checkins or at least keeping them out of
the release branch, but mainly the additional work and severe SF problems
kept me from doing so --- moreover, even as "release master", I don't want
to be a "dictator".

For later reference, I wrapped the respective changes in CVS tags
"information-functions_and_checkpointing-thread_before" and
"information-functions_and_checkpointing-thread_after".

(Sjoerd knows how to use them, in case removing these changes from the
 release branch should be desired.) 

Postponing the creation of the branches was no option for me, basically,
because I didn't want to "compromise" the release --- we have merely more
than a week left...

That's all from my side.


As announced on Monday (at CWI), due to other obligations (that go postponed
due to my limited time management skills), I'll not be available for any
MonetDB related work or questions etc. (at least) until next Tuesday. 
(In urgent cases TestTools and Mtest might be an exception.)


All my work so far has been documented (here and in the bug reports).


Success with the release!

Stefan



ps: Automatic testing:
    ==================
    As of now I'll again test both the "Stable" release banches and the
    "Current" development trunks.
    For the time being "Stable" will be tested with (overruling defaults)
	--enable-assert
	--enable-strict
	make check (i.e., Mtest -d10)
    while "Current" will be tested with (overruling defaults)
	--disable-assert
	--disable-strict
	Mtest -d0

    I might consider swapping the settings, once we know whether pathfinder
    compiles with --disable-strict ...


-- 
| Dr. Stefan Manegold | mailto:Stefan.Manegold at cwi.nl |
| CWI,  P.O.Box 94079 | http://www.cwi.nl/~manegold/  |
| 1090 GB Amsterdam   | Tel.: +31 (20) 592-4212       |
| The Netherlands     | Fax : +31 (20) 592-4312       |




More information about the developers-list mailing list