[Monetdb-developers] Re: 3rd MonetDB-Bug-Day: Mon, Dec 19 2005 (today)

Niels Nes Niels.Nes at cwi.nl
Mon Dec 19 09:34:52 CET 2005


On Mon, Dec 19, 2005 at 01:33:38AM +0100, Stefan Manegold wrote:
> 
> Dear all,
> 
> here are some more informations for the 3rd MonetDB-Bug-Day today.
> 
> Please note, though it's called Bug-*Day*, you don't have to spend the whole
> day writing test scripts (though no one would mind if you do ;-)).
> Below you find chunks of 10 bug reports each; simply claim one chunk at a
> time (by group-reply to this email) whenever you feel like you could spend
> some minutes on this "community task", check the 10 reports and provide the
> test script where necessary/applicable.
> If we don't manage to finish the 21 chunks tomorrow, everybody is welcome to
> pick more chunk(s) any time later...
> 
> Of course, next to (or instead of) adding test scripts, you're also welcome
> to fix open bugs (and add the test script for each bug you fix)!
> 
> 
> Chunks
> ======
> 
> No  Offsets      BugIDs        #  URL
> --------------------------------------
>  1  550-558  1206432-1206985   9  http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=0&_category=0&_group=0&order=artifact_id&sort=ASC&offset=550
done
> 
>  2  589-599  1210748-1214335  11  http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=0&_category=0&_group=0&order=artifact_id&sort=ASC&offset=589

Did first 4. ... Have to do something else no ;-)

Niels
>  3  600-609  1214372-1218281  10  http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=0&_category=0&_group=0&order=artifact_id&sort=ASC&offset=600
>  4  610-619  1218291-1227493  10  http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=0&_category=0&_group=0&order=artifact_id&sort=ASC&offset=610
>  5  620-629  1229018-1238228  10  http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=0&_category=0&_group=0&order=artifact_id&sort=ASC&offset=620
>  6  630-639  1238352-1242164  10  http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=0&_category=0&_group=0&order=artifact_id&sort=ASC&offset=630
>  7  640-649  1244326-1257463  10  http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=0&_category=0&_group=0&order=artifact_id&sort=ASC&offset=640
>  8  650-659  1259668-1263967  10  http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=0&_category=0&_group=0&order=artifact_id&sort=ASC&offset=650
>  9  660-669  1265413-1274819  10  http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=0&_category=0&_group=0&order=artifact_id&sort=ASC&offset=660
> 10  670-679  1274903-1281996  10  http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=0&_category=0&_group=0&order=artifact_id&sort=ASC&offset=670
> 11  680-689  1283842-1293196  10  http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=0&_category=0&_group=0&order=artifact_id&sort=ASC&offset=680
> 12  690-698  1294712-1298707   9  http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=0&_category=0&_group=0&order=artifact_id&sort=ASC&offset=690
> 
> 13  710-719  1313271-1315339  10  http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=0&_category=0&_group=0&order=artifact_id&sort=ASC&offset=710
> 14  720-729  1315653-1324575  10  http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=0&_category=0&_group=0&order=artifact_id&sort=ASC&offset=720
> 15  730-739  1327854-1334536  10  http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=0&_category=0&_group=0&order=artifact_id&sort=ASC&offset=730
> 16  740-749  1337630-1347331  10  http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=0&_category=0&_group=0&order=artifact_id&sort=ASC&offset=740
> 17  750-759  1347758-1353883  10  http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=0&_category=0&_group=0&order=artifact_id&sort=ASC&offset=750
> 18  760-769  1354321-1358222  10  http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=0&_category=0&_group=0&order=artifact_id&sort=ASC&offset=750
> 19  770-779  1359071-1363729  10  http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=0&_category=0&_group=0&order=artifact_id&sort=ASC&offset=750
> 20  780-789  1363733-1371524  10  http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=0&_category=0&_group=0&order=artifact_id&sort=ASC&offset=750
> 21  790-799  1373391-1380287  10  http://sourceforge.net/tracker/index.php?func=browse&group_id=56967&atid=482468&set=custom&_assigned_to=0&_status=0&_category=0&_group=0&order=artifact_id&sort=ASC&offset=750
>                              ---
>                              209 
> 
> 
> General issues
> ==============
> 
> Please pick one chunk at a time and indicate you choice by (group) replying
> to this mail. I will resolve clashes where necessary.  I don't expect more
> than 21 people participating (concurrently), hence sub-dividing the chunks
> should not be necessary ;-)
> In case you cannot finish your last chunk by tonight (for whatever reason),
> please let me know (via email) which of the reports you finished, and which
> you didn't.
> (Please see below for details what to do in case your hit a report that
> requires a test script, but for which you cannot create a test script.)
> 
> 
> Requirements
> ============
> 
> Obviously, you need to have MonetDB (plus SQL or XQuery for those bugs)
> installed and running; preferably an up-to-date ssh-based (non anonymous)
> CVS check out of either the release branches (MonetDB_4-8, SQL_2-8,
> XQuery_0-8) or the respective CVS heads. Anonymous (pserver-based) CVS check
> outs or source distributions should also work, but then you cannot check-in
> your tests directly (see below on how to participate and submit test script
> in this case).
> 
> Stable vs. Current
> ------------------
> 
> In principle, it does not matter, whether you're using the Stable release
> branch(es) or the Current development head(s). Use what ever you like best
> (or have already running).
> 
> Test directories
> ----------------
> 
> Your tests must be located (and hence checked in) in the following
> directories:
> 
> 	MonetDB 4.9.3:    MonetDB/tests/BugDay_2005-12-19_4.9.3/Tests/
> 	SQL     2.9.3:     sql/src/test/BugDay_2005-12-19_2.9.3/Tests/
> 	XQuery  0.9.3: pathfinder/tests/BugDay_2005-12-19_0.9.3/Tests/
> 
> 
> Codes
> =====
>  
> To indicate the progress (see below for details), please use the following
> codes when adding comments to the bug report
> 
> code                      explanation
> ------------------------+---------------------------------------------
> RELEASED / REASSIGN     | the "claimer" of the bug wasn't able to create a
>                         | script from the bug due to insufficient
>                         | knowledge.  Append reason, e.g. "XQuery".
>                         | Also set status of closed bugs to "pending".
> ALREADY IN TESTWEB      | the script was already in CVS and TestWeb 
>                         | (added location of existing test script).
> TEST ADDED / SUCCESS    | a script was added (append location & name) and
>                         | works correctly.
> TEST ADDED / FAILURE    | a script was added, but it fails (bad output).
>                         | Re-open closed bug!
> TEST SUBMITTED          | a script was made, but sent to Stefan (for
>                         | non-CVS participants)
> NO TEST / COMPILATION   | no test was added because the bug deals with a
>                         | compilation problem. 
> NO TEST / DOCUMENTATION | no test was added because the bug deals with a
>                         | documentation issue, or misleading help text,
>                         | etc.
> NO TEST / REDUNDANT     | no test was added, because either the bug itself
>                         | is a duplicate, or the test that follows out of
>                         | the bug would clearly be duplicating another
>                         | existing bug.  When extending another bug to
>                         | remove redundancy use TEST ADDED / SUCCESS and
>                         | give the name of the already existing updated
>                         | bug.
> 
> Note: the last action is only meant for experts in their field ;)
> 
> An example message on the SF bug could be:
> 
>   	BugDay_2005-12-19, <your name>: TEST ADDED / SUCCESS
>   	sql/src/tests/bugday/like_join_union.SF-0123345.SQL
> 
> 
> The Task/Procedure
> ==================
> 
> Claim a chunk of bugs as described above, and then process each bug as
> follows:
> 
> Checking for existence
> ----------------------
> 
> Once claimed, you first check, whether there is already a test script for
> this bug (i.e., a test script that contains the respective ID in its
> filename) available in one of
> 
> 	MonetDB/tests/BugsViaSourgeforce/Tests/
> 	sql/src/test/bugs/Tests/
> 	pathfinder/tests/BugsViaSourgeforce/Tests/
> 
> If so, you should indicate this be adding a comment like
> 
> 	BugDay_2005-12-19, <your name>: ALREADY IN TESTWEB
> 	MonetDB/tests/BugsViaSourgeforce/Tests/ID.602271.*
> 
> to the respective bug report on SF, and you're done with this bug, and can
> proceed to the next one.
> 
> If there is no test script, yet, you need to create one:
> 
> Naming conventions
> ------------------
> 
> Names of test scripts must include a descriptive part (of reasonable length)
> and the SF bug ID, like
> 
> 	"return_types_of_head_and_tail.SF-701641.*",
> 
> i.e., the pattern is 
> 
> 	<descriptive-name>.SF-<BugID>[.<extension>]
> 
> Creating test scripts
> ---------------------
> 
> Most bug reports come with some MIL-, SQL-, XQuery-code, or some other
> description/instructions how to reproduce the bug. The basic task is to put
> this into a proper "Mtest-compliant" test script. In most cases, this is
> just a .milS, .sql, or .xq file.
> See http://monetdb.cwi.nl/monet/src/testing/README for details about Mtest.py .
> In case of doubt, please feel free to contact any of your colleagues that might 
> be more familiar with Mtest --- or me of course ;-)
> Of course, you're welcome to extend the test if appropriate.
> 
> In case it is obvious that the bug does not require a test script (e.g.,
> compilation, installation, documentation, or web site bugs), please indicate
> this by adding a comment like
> 
> 	BugDay_2005-12-19, <your name>: NO TEST / <good reason>
> 
> to the respective bug report on SF, and proceed with the next bug.
> 
> If a test script is required, but you don't know how to make it (for this
> particular bug), please feel free to ask for help (or "release" the bug
> report, again).
> 
> Running Mtest
> -------------
> 
> Once you made the test script, you need to run the test. In case you're
> using `source conf/conf.bash` to setup your MonetDB environment, you can
> simply call one of
> 
> 	Mtest_MonetDB <descriptive-name>.SF-<BugID>
> 	Mtest_sql <descriptive-name>.SF-<BugID>
> 	Mtest_pathfinder <descriptive-name>.SF-<BugID>
> 
> in the respective directory as listed above.
> See `Mtest.py --help` for command line options.
> 
> Note: Do NOT change or check-in the "All" file!
> 
> This is not required to run tests that are explicitly given on Mtest's
> command line. I'll will fill the "All" files at the end of the BugDay.
> 
> Creating / fixing stable output
> -------------------------------
> 
> After running Mtest, you need to check the output by pointing your browser
> to the file given at the end of Mtest's console output.
> 
> In case the output is correct and as expected, you can approve it by running
> 
> 	Mapprove_MonetDB <descriptive-name>.SF-<BugID>
> 	Mapprove_sql <descriptive-name>.SF-<BugID>
> 	Mapprove_pathfinder <descriptive-name>.SF-<BugID>
> 
> in the respective directory as listed above.
> 	
> In case running the test failed, please fix the test and start again with
> running Mtest.
> 
> In case the output is not correct or not as expected, you first approve the
> output as described above, but then edit the files
> 
> 	<descriptive-name>.SF-<BugID>.stable.out
> 	<descriptive-name>.SF-<BugID>.stable.err
> 
> by hand to create the correct/expected output.
> 
> In any case, you then re-run Mtest, and check the output by pointing your
> browser to the file given at the end of Mtest's console output.
> 
> In case everything is "green" (i.e., OK), you can check-in all files related
> to the test (including the stable output!), indicate the success by adding a
> comment like
> 
> 	BugDay_2005-12-19, <your name>: TEST ADDED / SUCCESS
> 	tests/BugDay_2005-12-19_4.9.3/Tests/return_types_of_head_and_tail.SF-701641.*
> 
> to the respective bug report on SF, and proceed with the next bug.
> 
> In case there is still a red 'X' (because MonetDB does not produce the
> correct/expected output), you also check-in all files, but indicate the
> problem by a comment like
> 
> 	BugDay_2005-12-19, <your name>: TEST ADDED / FAILURE
> 	tests/BugDay_2005-12-19_4.9.3/Tests/return_types_of_head_and_tail.SF-701641.*
> 
> 	<possibly a description of the failure>
> 
> to the respective bug report on SF, and *re-open* this bug report,
> before proceeding with the next bug.
> 
> 
> Anonymous CVS check outs and source distributions
> =================================================
> 
> All the above holds, except from the CVS check-in of the new test scripts.
> Instead, you send me (Stefan.Manegold at cwi.nl) a .tar[.gz] or .zip file
> including all your new test scripts by the end of your BugDay.
> 
> In case you intend to participate without CVS access, please let me know so
> as soon as you start.
> 
> 
> Pooh, I guess that's it for now.
> I hope, I didn't forget anything --- if so, feel free to ask!
> (via email, skype, or personally later at CWI).
> 
> cu later,
> 
> Stefan
> 
> 
> 
> On Fri, Dec 16, 2005 at 01:52:30PM +0100, Stefan Manegold wrote:
> > Dear all,
> > 
> > given the success of the first two MonetDB-Bug-Days on Oct 06 2005 and Nov
> > 09 2005,
> > 
> > --- we managed to check 460 bug reports, added missing test scripts,
> > re-opened bugs that still/again fail and filed new bug reports ---
> > 
> > there will by a 3rd MonetDB-Bug-Day to (hopefully) check the remaining ~225
> > bug reports, and produce proper test scripts for our automatic Nightly
> > Multi-Platform Regression Test System (aka "TestWeb"; cf.
> > http://monetdb.cwi.nl/Development/TestWeb/):
> > 
> > 
> > When:   Monday, Dec 19, 2005, "all day" (or as much time as you can spent)
> > 
> > Where:  Mainly at CWI, but you're also welcome to join "virtually" working
> >         from home or any other place in the world;
> >         for those who (again) want to turn the Bug-Day into a Bug-"Party", I
> >         reserved our meeting room (C0,01) from 9:00 - 17:00 --- you need to
> >         bring your own "equipment" though ...
> > 
> > Who:    Primarily everyone who is in touch with MonetDB, and/or one of its
> >         companions/front ends (SQL, XQuery, etc.), has rights to check-in to
> >         the MonetDB CVS repository on SourceForge; experiences with
> >         "Mtest.py" (cf.  http://monetdb.cwi.nl/monet/src/testing/README) are
> >         helpful, but can also be aquired during the MonetDB-Bug-Day.
> > 
> > What:   I will prepare a list of all bugs that have no test script, yet, and
> > 	chop this in chunks, and assign each chunk to one participant so
> > 	that [s]he can check them, and prepare the respective test scripts.
> > 
> >         More detailed instructions will follow before Monday.
> > 
> > Note:	Though any help is appreciated, no one is "obliged" to help.
> > 	Especially, there is no need to spent the whole day (though nine
> > 	will complain if you do ;-)), anyone can join-in and leave whenever
> > 	[s]he wants.
> > 
> > 	For those of you participating at CWI, there will be free snacks and
> > 	soft-drinks during the day (for the time you're "active"; as long as
> > 	supplies lasts).
> > 	Furthermore, there will be "real" drinks at the end of the Bug-Day
> > 	(for participants, only; as long as supplies lasts; (maximum) amount
> > 	 per person will be proportional to the number of test scripts added).
> > 
> > 
> > I hope to see especially (though not only) participants that I haven't seen
> > on the previous MonetDB-Bug-Days ;-)
> > 
> > Once again thank you very much for your cooperation!
> > 
> > 
> > CU next Monday,
> > 
> > Stefan
> > 
> > 
> > ps: Testing & the TestWeb are not (only?) my "toys"; it's "serious
> >     business", and helps all of you to (1) get a stable and reliable
> >     MonetDB, (2) monitor when your own bug reports get fixed, and (3)
> >     prevent the bugs you once thankfully detected, reported and/or fixed
> >     from occurring again!
> 
> -- 
> | Dr. Stefan Manegold | mailto:Stefan.Manegold at cwi.nl |
> | CWI,  P.O.Box 94079 | http://www.cwi.nl/~manegold/  |
> | 1090 GB Amsterdam   | Tel.: +31 (20) 592-4212       |
> | The Netherlands     | Fax : +31 (20) 592-4312       |

-- 

Niels Nes, Centre for Mathematics and Computer Science (CWI)
Kruislaan 413, 1098 SJ Amsterdam, The Netherlands
room C0.02,  phone ++31 20 592-4098, fax ++31 20 592-4312
url: http://www.cwi.nl/~niels   e-mail: Niels.Nes at cwi.nl




More information about the developers-list mailing list