[Monetdb-developers] TTT goes MonetDB-Bug-Day

Stefan Manegold Stefan.Manegold at cwi.nl
Tue Oct 4 13:49:15 CEST 2005

[ I deliberately send this email to various related mailing lists; ]
[ in case you receive this email more than once, it might be time  ]
[ to teach your email client to eliminate multiple copies of the   ]
[ very same email...                                               ]

Dear all,

(admittedly on short notice --- sorry for any related inconvenience)

you're all "invited" to join a community effort to significantly improve the
the MonetDB quality control system: Let's (try to) make "Mtest.py-compliant"
test scripts from all bug reports that have been filed on SF since July 2002,
and thus ensure that these bugs do not re-occur, respectively ensure that we
(via the "TestWeb"; cf. http://monetdb.cwi.nl/Development/TestWeb/)
immediately notice whenever code changes do trigger an "old" bug.


When:	Thursday, Oct 6, 2005, "all day"

Where:	Mainly at CWI, but you're also welcome to join "virtually" working
	from home or any other place in the world;
	for those who want to turn the Bug-Day into a Bug-"Party", I
	reserved our meeting room (C0,01) from 12:30 - 17:00 --- you need to
	bring your own "equipment" though

Who:	Primarily everyone who is in touch with MonetDB, and/or one of its
	companions/front ends (SQL, XQuery, etc.), has rights to check-in to
	the MonetDB CVS repository on SourceForge, and is reasonably
	familiar with "Mtest.py" (cf. 
	But I will also (try to) provide "Testing HowTo" by Thursday, so
	others are welcome to join; further, if you cannot check-in to the
	MonetDB CVS repository yourself, a colleague of your's (or one of
	the core developers) might do this for you, once you prepared the
	complete test.

What:	I will prepare a list of all bug that have to test script, yet, and
	assign them round-robin to all participants (probably, I will try
	to classify them in say "MIL", "SQL", "XQuery", and "others", and
	you're welcome to choose your preferred class, given availability).

	More detailed instructions will follow by Thursday.


Next to providing high performance, maintaining correctness and stability of
the code base is the major goal of MonetDB and its companions.

First of all, the bug tracking system on SourceForge
(http://sourceforge.net/tracker/?group_id=56967&atid=482468) is used
intensively to report any problems related to MonetDB --- since MonetDB
moved to SourceForge in July 2002, almost 700 bug reports have been filed,
most of which have been solved.

Secondly, our automatic Nightly Multi-Platform Regression Test System (aka
"TestWeb"; cf. http://monetdb.cwi.nl/Development/TestWeb/) helps us to
ensure that bugs that sneak in with modified or newly added code are
detected (and fixed) instantly. Obviously, the test system only work for
that features/code that are indeed covered by the available test scripts...

Given these two tools, the bug tracker and the test system, we can easily
build a system that prevent the (unnoticed) re-occurrence of bug that have
been reported and fixed before: We "simply" have to add a proper test script
for each bug report.

Unfortunately, so far there are test script only for a fraction of the 
~700 bug reports. Given that many bugs are reported by users that are not
allowed to add test script to the CVS repository, it is not convenient (not
even possible) to require that each bug report includes an
"Mtest.py-compliant" (cf. http://monetdb.cwi.nl/monet/src/testing/README)
test script. Nevertheless, most bug reports come with a query, a script, or
a detailed description how to reproduce the respective bug; hence, it should
be rather simple to turn these queries / scripts / description into proper
"Mtest.py-compliant" test scripts, and thus significantly improve our
automatic quality control system.

Since this task is quite a challenge (in terms of volume, not intellectual
complexity ;-)), the idea popped-up to make it a "community effort", as
sketched in the "Summary" above. The advantage of this coordinated community
effort should be, that all of us dedicate our time to this concurrently,
share the same "context" and hence can support each other efficiently
when/where necessary.

In advance, thank you very much for your cooperation!



| Dr. Stefan Manegold | mailto:Stefan.Manegold at cwi.nl |
| CWI,  P.O.Box 94079 | http://www.cwi.nl/~manegold/  |
| 1090 GB Amsterdam   | Tel.: +31 (20) 592-4212       |
| The Netherlands     | Fax : +31 (20) 592-4312       |

More information about the developers-list mailing list