[Monetdb-developers] Re: TTT goes MonetDB-Bug-Day

Niels Nes Niels.Nes at cwi.nl
Wed Oct 5 19:49:18 CEST 2005


On Wed, Oct 05, 2005 at 06:43:17PM +0200, Stefan Manegold wrote:
> Dear all,
> 
> as promised, here are some more detailed informations for tomorrow.
> 
> General issues
> ==============
> 
> To "decentralize" processing and prevent me from becoming the
> "bottleneck"/"hot spot", I not to assign tests to you; rather, each of you
> claims the bug that he/she is going to prepare a test script for
> by adding a comment like
> 
> 	"BugDay_2005-10-06: Claimed [by <name>]."
> 
> to the respective bug report; adding a name is optional if you claim a bug
> for yourself; if you claim a bug for someone who does not have an account
> with SourceForge (and is not willing to request one) and hence cannot add
> comments to the bug tracker, adding his/her name is mandatory.
> See http://sourceforge.net/tracker/?group_id=56967&atid=482468 for a
> complete list of bug report, and don't for get to set your selection as
> follows:
> 
> 	Assignee:	Status:		Category:	Group:
> 	Any		Any		Any		Any
> 
> 	Sort By:	ID	Ascending
> 
> Of course, you're welcome to first claim multiple bugs, before actually
> starting to prepare and check-in the tests.
> 
> To avoid that everyone starts (trying) to claim the same bugs, here are some
> bug IDs as entry points. Each one chooses a different entry point, and from
> there on processes the bugs in in ascending order of bug ID.
> 
>  602271	http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=100&_category=100&_group=100&order=artifact_id&sort=ASC&offset=0
Okay I simply claim the first 50..

Niels
>  899586	http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=100&_category=100&_group=100&order=artifact_id&sort=ASC&offset=50
>  920585	http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=100&_category=100&_group=100&order=artifact_id&sort=ASC&offset=100
>  929158	http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=100&_category=100&_group=100&order=artifact_id&sort=ASC&offset=150
>  941693	http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=100&_category=100&_group=100&order=artifact_id&sort=ASC&offset=200
>  972833	http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=100&_category=100&_group=100&order=artifact_id&sort=ASC&offset=250
> 1010604	http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=100&_category=100&_group=100&order=artifact_id&sort=ASC&offset=300
> 1029099	http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=100&_category=100&_group=100&order=artifact_id&sort=ASC&offset=350
> 1076604	http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=100&_category=100&_group=100&order=artifact_id&sort=ASC&offset=400
> 1119199	http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=100&_category=100&_group=100&order=artifact_id&sort=ASC&offset=450
> 1166456	http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=100&_category=100&_group=100&order=artifact_id&sort=ASC&offset=500
> 1205565	http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=100&_category=100&_group=100&order=artifact_id&sort=ASC&offset=550
> 1214335	http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=100&_category=100&_group=100&order=artifact_id&sort=ASC&offset=600
> 1257463	http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=100&_category=100&_group=100&order=artifact_id&sort=ASC&offset=650
> 1300366	http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=100&_category=100&_group=100&order=artifact_id&sort=ASC&offset=700
> 
> Please indicate you choice by (group) replying to this mail. I will resolve
> clashes where necessary.  In case there are more than 15 people
> participating (concurrently), we can easily sub-divide the ranges.
> 
> 
> Requirements
> ============
> 
> Obviously, you need to have MonetDB (plus SQL or XQuery for those bugs)
> installed and running; preferably an up-to-date ssh-based (non anonymous)
> CVS check out of either the release branches (MonetDB_4-8, SQL_2-8,
> XQuery_0-8) or the respective CVS heads. Anonymous (pserver-based CVS check
> outs or source distributions should also work, but then you cannot check-in
> your tests directly (see below on how to participate and submit test script
> in this case).
> 
> Stable vs. Current
> ------------------
> 
> In principle, it does not matter, whether you're using the Stable release
> branch(es) or the Current development head(s). Use what ever you like best
> (or have already running).
> 
> Test directories
> ----------------
> 
> Your tests must be located (and hence check in) in different directories
> depending on the MonetDB version you're using:
> 
> Stable branch:
> 	MonetDB 4.8.*:    MonetDB/tests/BugDay_2005-10-06_4.8/Tests/
> 	SQL     2.8.*:        sql/src/test/BugDay_2005-10-06_2.8/Tests/
> 	XQuery  0.8.*: pathfinder/tests/BugDay_2005-10-06_0.8/Tests/
> 
> Current head:
> 	MonetDB 4.9.3:    MonetDB/tests/BugDay_2005-10-06_4.9.3/Tests/
> 	SQL     2.9.3:        sql/src/test/BugDay_2005-10-06_2.9.3/Tests/
> 	XQuery  0.9.3: pathfinder/tests/BugDay_2005-10-06_0.9.3/Tests/
> 
> The reason for having different direcories is mainly convenience with
> check-ins and later propagations.
> 
> 
> The Task/Procedure
> ==================
> 
> Claiming a bug
> --------------
> 
> As mentioned above, you simply claim a bug by adding a comment like
> 
> 	"BugDay_2005-10-06: Claimed [by <name>]."
> 
> to the respective bug report on SF.
> 
> Checking for existence
> ----------------------
> 
> Once claimed, you first check, whether there is already a test script for
> this bug available in one of
> 
> 	MonetDB/tests/BugsViaSourgeforce/Tests/
> 	sql/src/test/bugs/Tests/
> 	pathfinder/tests/BugsViaSourgeforce/Tests/
> 
> If so, you should indicate this be adding a comment like
> 
> 	BugDay_2005-10-06: Done.
> 	test script available as
> 	MonetDB/tests/BugsViaSourgeforce/Tests/ID.602271.*
> 
> to the respective bug report on SF, and you're done with this bug, and can
> proceed to the next one.
> 
> If there is no test script, yet, you need to create one:
> 
> Naming conventions
> ------------------
> 
> Names of test scripts must include a descriptive part (of reasonable length)
> and the SF bug ID, like
> 
> 	"return_types_of_head_and_tail.SF-701641.*",
> 
> i.e., the pattern is 
> 
> 	<descriptive-name>.SF-<BugID>[.<extension>]
> 
> Creating test scripts
> ---------------------
> 
> As mentioned in my first email, most bug reports come with some MIL-, SQL-,
> XQuery-code, or some other description/instructions how to reproduce the
> bug. The basic task is to put this into a proper "Mtest-compliant" test script.
> In most cases, this is just a .milS, .sql, or .xq file.
> See http://monetdb.cwi.nl/monet/src/testing/README for details about Mtest.py .
> In case of doubt, please feel free to contact any of your colleagues that might 
> be more familiar with Mtest --- or me of course ;-)
> Of course, you're welcome to extend the test if appropriate.
> 
> In case it is obvious that the bug does not require a test script (e.g.,
> compilation, installation, or web site bugs), please indicate this by adding
> a comment like
> 
> 	BugDay_2005-10-06: Done.
> 	
> 	No test script added, because <good reason>.
> 
> to the respective bug report on SF, and proceed with the next bug.
> 
> If a test script is required, but you don't know how to make it (for this
> particular bug), please feel free to ask for help (or "release" the bug
> report, again).
> 
> Running Mtest
> -------------
> 
> Once you made the test script, you need to run the test. In case you're
> using `source conf/conf.bash` to setup your MonetDB environment, you can
> simply call one of
> 
> 	Mtest_MonetDB <descriptive-name>.SF-<BugID>
> 	Mtest_sql <descriptive-name>.SF-<BugID>
> 	Mtest_pathfinder <descriptive-name>.SF-<BugID>
> 
> in the respective directory as listed above.
> See `Mtest.py --help` for command line options.
> 
> Note: Do NOT change or check-in the "All" file!
> 
> This is not required to run tests that are explicitly given on Mtest's
> command line. I'll will fill the "All" files at the end of the BugDay.
> 
> Creating / fixing stable output
> -------------------------------
> 
> After running Mtest, you need to check the output by pointing your browser
> to the file given at the end of Mtest's console output.
> 
> In case the output is correct and as expected, you can approve it by running
> 
> 	Mapprove_MonetDB <descriptive-name>.SF-<BugID>
> 	Mapprove_sql <descriptive-name>.SF-<BugID>
> 	Mapprove_pathfinder <descriptive-name>.SF-<BugID>
> 
> in the respective directory as listed above.
> 	
> In case running the test failed, please fix the test and start again with
> running Mtest.
> 
> In case the output is not correct or no as expected, you first approve the
> output as described above, but then edit the files
> 
> 	<descriptive-name>.SF-<BugID>.stable.out
> 	<descriptive-name>.SF-<BugID>.stable.err
> 
> by hand to create the correct/expected output.
> 
> In any case, you then re-run Mtest, and check the output by pointing your
> browser to the file given at the end of Mtest's console output.
> 
> In case everything is "green" (i.e., OK), you can check-in all files related
> to the test (including the stable output!), indicate the success by adding a
> comment like
> 
> 	BugDay_2005-10-06: Done:
> 	test script added as
> 	tests/BugDay_2005-10-06_4.9.3/Tests/return_types_of_head_and_tail.SF-701641.*
> 
> to the respective bug report on SF, and proceed with the next bug.
> 
> In case there is still a red 'X' (because MonetDB does not produce the
> correct/expected output), you also check-in all files, but indicate the
> problem by a comment like
> 
> 	BugDay_2005-10-06: Done:
> 	test script added as
> 	tests/BugDay_2005-10-06_4.9.3/Tests/return_types_of_head_and_tail.SF-701641.*
> 
> 	Note: Test fails!
> 	<possibly a description of the failure>
> 
> to the respective bug report on SF, and re-open this bug report,
> before proceeding with the next bug.
> 
> 
> Anonymous CVS check outs and source distributions
> =================================================
> 
> All the above holds, except from the CVS check-in of the new test scripts.
> Instead, you send me (Stefan.Manegold at cwi.nl) a .tar[.gz] or .zip file
> including all your new test scripts by the end of your BugDay.
> 
> In case you intend to participate without CVS access, please let me know so
> as soon as you start.
> 
> 
> Pooh, I guess that's it for now.
> I hope, I didn't forget anything --- if so, feel free to ask!
> (via email, or personally tomorrow at CWI).
> 
> cu tomorrow,
> 
> Stefan
> 
> --
> | Dr. Stefan Manegold | mailto:Stefan.Manegold at cwi.nl |
> | CWI,  P.O.Box 94079 | http://www.cwi.nl/~manegold/  |
> | 1090 GB Amsterdam   | Tel.: +31 (20) 592-4212       |
> | The Netherlands     | Fax : +31 (20) 592-4312       |

-- 

Niels Nes, Centre for Mathematics and Computer Science (CWI)
Kruislaan 413, 1098 SJ Amsterdam, The Netherlands
room C0.02,  phone ++31 20 592-4098, fax ++31 20 592-4312
url: http://www.cwi.nl/~niels   e-mail: Niels.Nes at cwi.nl




More information about the developers-list mailing list