[ I deliberately send this email to various related mailing lists; ] [ in case you receive this email more than once, it might be time ] [ to teach your email client to eliminate multiple copies of the ] [ very same email... ] Dear all, (admittedly on short notice --- sorry for any related inconvenience) you're all "invited" to join a community effort to significantly improve the the MonetDB quality control system: Let's (try to) make "Mtest.py-compliant" test scripts from all bug reports that have been filed on SF since July 2002, and thus ensure that these bugs do not re-occur, respectively ensure that we (via the "TestWeb"; cf. http://monetdb.cwi.nl/Development/TestWeb/) immediately notice whenever code changes do trigger an "old" bug. Summary: ======== When: Thursday, Oct 6, 2005, "all day" Where: Mainly at CWI, but you're also welcome to join "virtually" working from home or any other place in the world; for those who want to turn the Bug-Day into a Bug-"Party", I reserved our meeting room (C0,01) from 12:30 - 17:00 --- you need to bring your own "equipment" though ... Who: Primarily everyone who is in touch with MonetDB, and/or one of its companions/front ends (SQL, XQuery, etc.), has rights to check-in to the MonetDB CVS repository on SourceForge, and is reasonably familiar with "Mtest.py" (cf. http://monetdb.cwi.nl/monet/src/testing/README). But I will also (try to) provide "Testing HowTo" by Thursday, so others are welcome to join; further, if you cannot check-in to the MonetDB CVS repository yourself, a colleague of your's (or one of the core developers) might do this for you, once you prepared the complete test. What: I will prepare a list of all bug that have to test script, yet, and assign them round-robin to all participants (probably, I will try to classify them in say "MIL", "SQL", "XQuery", and "others", and you're welcome to choose your preferred class, given availability). More detailed instructions will follow by Thursday. Background: =========== Next to providing high performance, maintaining correctness and stability of the code base is the major goal of MonetDB and its companions. First of all, the bug tracking system on SourceForge (http://sourceforge.net/tracker/?group_id=56967&atid=482468) is used intensively to report any problems related to MonetDB --- since MonetDB moved to SourceForge in July 2002, almost 700 bug reports have been filed, most of which have been solved. Secondly, our automatic Nightly Multi-Platform Regression Test System (aka "TestWeb"; cf. http://monetdb.cwi.nl/Development/TestWeb/) helps us to ensure that bugs that sneak in with modified or newly added code are detected (and fixed) instantly. Obviously, the test system only work for that features/code that are indeed covered by the available test scripts... Given these two tools, the bug tracker and the test system, we can easily build a system that prevent the (unnoticed) re-occurrence of bug that have been reported and fixed before: We "simply" have to add a proper test script for each bug report. Unfortunately, so far there are test script only for a fraction of the ~700 bug reports. Given that many bugs are reported by users that are not allowed to add test script to the CVS repository, it is not convenient (not even possible) to require that each bug report includes an "Mtest.py-compliant" (cf. http://monetdb.cwi.nl/monet/src/testing/README) test script. Nevertheless, most bug reports come with a query, a script, or a detailed description how to reproduce the respective bug; hence, it should be rather simple to turn these queries / scripts / description into proper "Mtest.py-compliant" test scripts, and thus significantly improve our automatic quality control system. Since this task is quite a challenge (in terms of volume, not intellectual complexity ;-)), the idea popped-up to make it a "community effort", as sketched in the "Summary" above. The advantage of this coordinated community effort should be, that all of us dedicate our time to this concurrently, share the same "context" and hence can support each other efficiently when/where necessary. In advance, thank you very much for your cooperation! Regards, Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |