[Monetdb-developers] Re: TTT goes MonetDB-Bug-Day

Stefan Manegold Stefan.Manegold at cwi.nl
Thu Oct 6 10:38:27 CEST 2005


On Thu, Oct 06, 2005 at 10:05:58AM +0200, Sjoerd Mullender wrote:
> Niels Nes wrote:
> > On Wed, Oct 05, 2005 at 06:43:17PM +0200, Stefan Manegold wrote:
> > 
> >>Dear all,
> >>
> >>as promised, here are some more detailed informations for tomorrow.
> >>
> >>General issues
> >>==============
> >>
> >>To "decentralize" processing and prevent me from becoming the
> >>"bottleneck"/"hot spot", I not to assign tests to you; rather, each of you
> >>claims the bug that he/she is going to prepare a test script for
> >>by adding a comment like
> >>
> >>	"BugDay_2005-10-06: Claimed [by <name>]."
> >>
> >>to the respective bug report; adding a name is optional if you claim a bug
> >>for yourself; if you claim a bug for someone who does not have an account
> >>with SourceForge (and is not willing to request one) and hence cannot add
> >>comments to the bug tracker, adding his/her name is mandatory.
> >>See http://sourceforge.net/tracker/?group_id=56967&atid=482468 for a
> >>complete list of bug report, and don't for get to set your selection as
> >>follows:
> >>
> >>	Assignee:	Status:		Category:	Group:
> >>	Any		Any		Any		Any
> >>
> >>	Sort By:	ID	Ascending
> >>
> >>Of course, you're welcome to first claim multiple bugs, before actually
> >>starting to prepare and check-in the tests.
> >>
> >>To avoid that everyone starts (trying) to claim the same bugs, here are some
> >>bug IDs as entry points. Each one chooses a different entry point, and from
> >>there on processes the bugs in in ascending order of bug ID.
> >>
> >> 602271	http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=100&_category=100&_group=100&order=artifact_id&sort=ASC&offset=0
> > 
> > Okay I simply claim the first 50..
> > 
> > Niels
> > 
> 
> I'll claim the following 50:
> >> 899586	http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=100&_category=100&_group=100&order=artifact_id&sort=ASC&offset=50
> 

Then, I'll start with then next package:
> >> 920585	http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=100&_category=100&_group=100&order=artifact_id&sort=ASC&offset=100

Stefan

> >> 929158	http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=100&_category=100&_group=100&order=artifact_id&sort=ASC&offset=150
> >> 941693	http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=100&_category=100&_group=100&order=artifact_id&sort=ASC&offset=200
> >> 972833	http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=100&_category=100&_group=100&order=artifact_id&sort=ASC&offset=250
> >>1010604	http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=100&_category=100&_group=100&order=artifact_id&sort=ASC&offset=300
> >>1029099	http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=100&_category=100&_group=100&order=artifact_id&sort=ASC&offset=350
> >>1076604	http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=100&_category=100&_group=100&order=artifact_id&sort=ASC&offset=400
> >>1119199	http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=100&_category=100&_group=100&order=artifact_id&sort=ASC&offset=450
> >>1166456	http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=100&_category=100&_group=100&order=artifact_id&sort=ASC&offset=500
> >>1205565	http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=100&_category=100&_group=100&order=artifact_id&sort=ASC&offset=550
> >>1214335	http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=100&_category=100&_group=100&order=artifact_id&sort=ASC&offset=600
> >>1257463	http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=100&_category=100&_group=100&order=artifact_id&sort=ASC&offset=650
> >>1300366	http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=100&_category=100&_group=100&order=artifact_id&sort=ASC&offset=700
> >>
> >>Please indicate you choice by (group) replying to this mail. I will resolve
> >>clashes where necessary.  In case there are more than 15 people
> >>participating (concurrently), we can easily sub-divide the ranges.
> >>
> >>
> >>Requirements
> >>============
> >>
> >>Obviously, you need to have MonetDB (plus SQL or XQuery for those bugs)
> >>installed and running; preferably an up-to-date ssh-based (non anonymous)
> >>CVS check out of either the release branches (MonetDB_4-8, SQL_2-8,
> >>XQuery_0-8) or the respective CVS heads. Anonymous (pserver-based CVS check
> >>outs or source distributions should also work, but then you cannot check-in
> >>your tests directly (see below on how to participate and submit test script
> >>in this case).
> >>
> >>Stable vs. Current
> >>------------------
> >>
> >>In principle, it does not matter, whether you're using the Stable release
> >>branch(es) or the Current development head(s). Use what ever you like best
> >>(or have already running).
> >>
> >>Test directories
> >>----------------
> >>
> >>Your tests must be located (and hence check in) in different directories
> >>depending on the MonetDB version you're using:
> >>
> >>Stable branch:
> >>	MonetDB 4.8.*:    MonetDB/tests/BugDay_2005-10-06_4.8/Tests/
> >>	SQL     2.8.*:        sql/src/test/BugDay_2005-10-06_2.8/Tests/
> >>	XQuery  0.8.*: pathfinder/tests/BugDay_2005-10-06_0.8/Tests/
> >>
> >>Current head:
> >>	MonetDB 4.9.3:    MonetDB/tests/BugDay_2005-10-06_4.9.3/Tests/
> >>	SQL     2.9.3:        sql/src/test/BugDay_2005-10-06_2.9.3/Tests/
> >>	XQuery  0.9.3: pathfinder/tests/BugDay_2005-10-06_0.9.3/Tests/
> >>
> >>The reason for having different direcories is mainly convenience with
> >>check-ins and later propagations.
> >>
> >>
> >>The Task/Procedure
> >>==================
> >>
> >>Claiming a bug
> >>--------------
> >>
> >>As mentioned above, you simply claim a bug by adding a comment like
> >>
> >>	"BugDay_2005-10-06: Claimed [by <name>]."
> >>
> >>to the respective bug report on SF.
> >>
> >>Checking for existence
> >>----------------------
> >>
> >>Once claimed, you first check, whether there is already a test script for
> >>this bug available in one of
> >>
> >>	MonetDB/tests/BugsViaSourgeforce/Tests/
> >>	sql/src/test/bugs/Tests/
> >>	pathfinder/tests/BugsViaSourgeforce/Tests/
> >>
> >>If so, you should indicate this be adding a comment like
> >>
> >>	BugDay_2005-10-06: Done.
> >>	test script available as
> >>	MonetDB/tests/BugsViaSourgeforce/Tests/ID.602271.*
> >>
> >>to the respective bug report on SF, and you're done with this bug, and can
> >>proceed to the next one.
> >>
> >>If there is no test script, yet, you need to create one:
> >>
> >>Naming conventions
> >>------------------
> >>
> >>Names of test scripts must include a descriptive part (of reasonable length)
> >>and the SF bug ID, like
> >>
> >>	"return_types_of_head_and_tail.SF-701641.*",
> >>
> >>i.e., the pattern is 
> >>
> >>	<descriptive-name>.SF-<BugID>[.<extension>]
> >>
> >>Creating test scripts
> >>---------------------
> >>
> >>As mentioned in my first email, most bug reports come with some MIL-, SQL-,
> >>XQuery-code, or some other description/instructions how to reproduce the
> >>bug. The basic task is to put this into a proper "Mtest-compliant" test script.
> >>In most cases, this is just a .milS, .sql, or .xq file.
> >>See http://monetdb.cwi.nl/monet/src/testing/README for details about Mtest.py .
> >>In case of doubt, please feel free to contact any of your colleagues that might 
> >>be more familiar with Mtest --- or me of course ;-)
> >>Of course, you're welcome to extend the test if appropriate.
> >>
> >>In case it is obvious that the bug does not require a test script (e.g.,
> >>compilation, installation, or web site bugs), please indicate this by adding
> >>a comment like
> >>
> >>	BugDay_2005-10-06: Done.
> >>	
> >>	No test script added, because <good reason>.
> >>
> >>to the respective bug report on SF, and proceed with the next bug.
> >>
> >>If a test script is required, but you don't know how to make it (for this
> >>particular bug), please feel free to ask for help (or "release" the bug
> >>report, again).
> >>
> >>Running Mtest
> >>-------------
> >>
> >>Once you made the test script, you need to run the test. In case you're
> >>using `source conf/conf.bash` to setup your MonetDB environment, you can
> >>simply call one of
> >>
> >>	Mtest_MonetDB <descriptive-name>.SF-<BugID>
> >>	Mtest_sql <descriptive-name>.SF-<BugID>
> >>	Mtest_pathfinder <descriptive-name>.SF-<BugID>
> >>
> >>in the respective directory as listed above.
> >>See `Mtest.py --help` for command line options.
> >>
> >>Note: Do NOT change or check-in the "All" file!
> >>
> >>This is not required to run tests that are explicitly given on Mtest's
> >>command line. I'll will fill the "All" files at the end of the BugDay.
> >>
> >>Creating / fixing stable output
> >>-------------------------------
> >>
> >>After running Mtest, you need to check the output by pointing your browser
> >>to the file given at the end of Mtest's console output.
> >>
> >>In case the output is correct and as expected, you can approve it by running
> >>
> >>	Mapprove_MonetDB <descriptive-name>.SF-<BugID>
> >>	Mapprove_sql <descriptive-name>.SF-<BugID>
> >>	Mapprove_pathfinder <descriptive-name>.SF-<BugID>
> >>
> >>in the respective directory as listed above.
> >>	
> >>In case running the test failed, please fix the test and start again with
> >>running Mtest.
> >>
> >>In case the output is not correct or no as expected, you first approve the
> >>output as described above, but then edit the files
> >>
> >>	<descriptive-name>.SF-<BugID>.stable.out
> >>	<descriptive-name>.SF-<BugID>.stable.err
> >>
> >>by hand to create the correct/expected output.
> >>
> >>In any case, you then re-run Mtest, and check the output by pointing your
> >>browser to the file given at the end of Mtest's console output.
> >>
> >>In case everything is "green" (i.e., OK), you can check-in all files related
> >>to the test (including the stable output!), indicate the success by adding a
> >>comment like
> >>
> >>	BugDay_2005-10-06: Done:
> >>	test script added as
> >>	tests/BugDay_2005-10-06_4.9.3/Tests/return_types_of_head_and_tail.SF-701641.*
> >>
> >>to the respective bug report on SF, and proceed with the next bug.
> >>
> >>In case there is still a red 'X' (because MonetDB does not produce the
> >>correct/expected output), you also check-in all files, but indicate the
> >>problem by a comment like
> >>
> >>	BugDay_2005-10-06: Done:
> >>	test script added as
> >>	tests/BugDay_2005-10-06_4.9.3/Tests/return_types_of_head_and_tail.SF-701641.*
> >>
> >>	Note: Test fails!
> >>	<possibly a description of the failure>
> >>
> >>to the respective bug report on SF, and re-open this bug report,
> >>before proceeding with the next bug.
> >>
> >>
> >>Anonymous CVS check outs and source distributions
> >>=================================================
> >>
> >>All the above holds, except from the CVS check-in of the new test scripts.
> >>Instead, you send me (Stefan.Manegold at cwi.nl) a .tar[.gz] or .zip file
> >>including all your new test scripts by the end of your BugDay.
> >>
> >>In case you intend to participate without CVS access, please let me know so
> >>as soon as you start.
> >>
> >>
> >>Pooh, I guess that's it for now.
> >>I hope, I didn't forget anything --- if so, feel free to ask!
> >>(via email, or personally tomorrow at CWI).
> >>
> >>cu tomorrow,
> >>
> >>Stefan
> >>
> >>--
> >>| Dr. Stefan Manegold | mailto:Stefan.Manegold at cwi.nl |
> >>| CWI,  P.O.Box 94079 | http://www.cwi.nl/~manegold/  |
> >>| 1090 GB Amsterdam   | Tel.: +31 (20) 592-4212       |
> >>| The Netherlands     | Fax : +31 (20) 592-4312       |
> > 
> > 
> 
> 
> -- 
> Sjoerd Mullender



-- 
| Dr. Stefan Manegold | mailto:Stefan.Manegold at cwi.nl |
| CWI,  P.O.Box 94079 | http://www.cwi.nl/~manegold/  |
| 1090 GB Amsterdam   | Tel.: +31 (20) 592-4212       |
| The Netherlands     | Fax : +31 (20) 592-4312       |




More information about the developers-list mailing list