[Monetdb-developers] Re: [MonetDB-users] MonetDB Bug-Etiquette

Stefan Manegold Stefan.Manegold at cwi.nl
Mon Nov 21 12:45:48 CET 2005


On Mon, Nov 21, 2005 at 11:45:35AM +0100, Sjoerd Mullender wrote:
> 
> I guess it depends on the submitter.  For some it is (almost) trivial to
> create a test script, and in any case, it helps the developper
> enormously if a small test can be made that shows the problem.  In fact
> I would go so far as to say that for all real bugs (i.e. not compilation
> issues and typos and such) a small test is essential.  I do think it is
> the developper's responsibility that the test gets added to the test web
> (that doesn't mean the developper has to do it, it means the developper
> must make sure it is done).

That exactly what I intended to say --- apparently, I need to work on the
"efficiency" of my words ... ;-)

Users and developers should not be "eminies" blaming each other, rather they
should work together, and support each other as good as possible to solve
bug with minimal overhead. The user know how to trigger the bug, hence (s)he
should provide the developer with as much information as possible to
resproduce the bug and thus check whether her/his fixed indeed solve the
problem --- and a test query/script is the most efficient way to describe
and reproduce a bug. Likewise, the developer is responsible for ensuring
that his/her fix solves the problem *permanently*; again, running the users
test script/query interatively via Mtest AND automatically every night in
our test system is the most efficient/convenient/effective way to do so.
Since we cannot assume that every user is familiar with our testing facilities,
it is the developer's responsibility that test scripts indeed get added to
CVS (and hence the testing system and test web) before a bug is closed.

Both user and developer share the reposibility of verifying that the
provided solution indeed solved the problem permanently.

> >>- Please provide a detailed description how to repeat the bug you report, or
> >>  even a concise test script to do so. Preferably, the test script should
> >>  also be added to the standard test-suite (in the CVS repository; see
> >>  "Adding test scripts" below for more details!).
> > 
> > 
> > I strongly suggest to drop the CVS thing.  Many users don't have it, and
> > it's confusing and more work for devs, see below.
> 
> I'm sort of with Fabian here.  Maybe the "preferably" should be changed
> in "if possible".

fine with me.

> >>- Please feel free to close your bug report, in case you notice it does not
> >>  occur any more (your test script in the standard test-suite is your
> >>  free-of-charge monitor; simply check our "TestWeb" at
> >>  http://monetdb.cwi.nl/Development/TestWeb/ ;-)). 
> >>  See "Closing bug reports" below for more details!
> > 
> > 
> > This is weird.  So a bug can suddenly vanish?  Yes it can, but then the
> > exact cause of the bug was not found, so it can be gone just by
> > coincidence, which is *not* good.  Submitters should only close their
> > bug if they think they made a mistake and that their bug is invalid.
> 
> Bugs do sometimes disappear because of fixes that were made for some
> other reason.  It is appropriate that the report gets closed in that
> case.  The resolution should then be "Out of Date".

Exactly. ;-)

> >>- Once a developer closes your bug report, please double-check whether the
> >>  solution indeed works for you (simply check the status of your test script
> >>  in our "TestWeb" at http://monetdb.cwi.nl/Development/TestWeb/ ;-)).
> >>  If is does not work for you, feel free to re-open the bug report,
> >>  or file a new one. In either case, please give a detailed description
> >>  of the remaining or new problem!
> > 
> > 
> > Why not let the developer *always* add the test script before closing
> > the bug.  Checking that a bug is already there (hoping it complies to
> > the naming conventions as pointed out below) is just some extra hassle.
> > Besides that, IMHO it's just more well organised if the dev adds the
> > script once (s)he fixes the problem and as such the correct output can
> > be determined as well as the test possibly made somewhat more extensive
> > to cover more possible problems.  The dev that fixes the problem should
> > ideally have the best knowledge on why something went wrong.
> 
> As I mentioned above, it is the developer's responsibility.  That
> doesn't mean the developer must do it, and if the submitter can create
> and check in a test script, I would encourage that.

This (c|sh)ould have been my words ;-)


Anyway, thank you all very much for your comments so far. I'll try to
include them (as well as the comments that are still to come) in the
MonetDB-Bug-Etiquette.


Stefan

-- 
| Dr. Stefan Manegold | mailto:Stefan.Manegold at cwi.nl |
| CWI,  P.O.Box 94079 | http://www.cwi.nl/~manegold/  |
| 1090 GB Amsterdam   | Tel.: +31 (20) 592-4212       |
| The Netherlands     | Fax : +31 (20) 592-4312       |




More information about the developers-list mailing list