Nightly testing policy

From MonetDB
Revision as of 08:02, 25 May 2014 by Martin Kersten (talk | contribs) (Created page with "For your convenience, please check our nightly testing system [1] --- aka. "TestWeb" --- for the latest status of "your" branch as well as the branch that you forked your bran...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigationJump to search

For your convenience, please check our nightly testing system [1] --- aka. "TestWeb" --- for the latest status of "your" branch as well as the branch that you forked your branch from and/or you sync your branch with.

Every night, starting from just after 7pm, the system compiles (and tests) each branch that has been modified since the last run on some 20+ platform. Since testing takes quite some time (up to 5 hours on our slowest machines), only the most "important" branches are tested; the remaining ones are only compiled. The current list of branches is a follows

compiled & tested:

http://monetdb.cwi.nl/testweb/web/status.php?branch=Jan2014 http://monetdb.cwi.nl/testweb/web/status.php?branch=default http://monetdb.cwi.nl/testweb/web/status.php?branch=SciQL-2 http://monetdb.cwi.nl/testweb/web/status.php?branch=int128

only compiled:

http://monetdb.cwi.nl/testweb/web/status.php?branch=SciQL-2-NetCDF http://monetdb.cwi.nl/testweb/web/status.php?branch=DVframework http://monetdb.cwi.nl/testweb/web/status.php?branch=DVframework_bam http://monetdb.cwi.nl/testweb/web/status.php?branch=bamloader http://monetdb.cwi.nl/testweb/web/status.php?branch=holindex http://monetdb.cwi.nl/testweb/web/status.php?branch=mutation http://monetdb.cwi.nl/testweb/web/status.php?branch=rdf http://monetdb.cwi.nl/testweb/web/status.php?branch=ssdb http://monetdb.cwi.nl/testweb/web/status.php?branch=xid http://monetdb.cwi.nl/testweb/web/status.php?branch=newstorage http://monetdb.cwi.nl/testweb/web/status.php?branch=RIntegration http://monetdb.cwi.nl/testweb/web/status.php?branch=bwd http://monetdb.cwi.nl/testweb/web/status.php?branch=geo http://monetdb.cwi.nl/testweb/web/status.php?branch=multifarm http://monetdb.cwi.nl/testweb/web/status.php?branch=transaction-replication http://monetdb.cwi.nl/testweb/web/status.php?branch=bloomfilters http://monetdb.cwi.nl/testweb/web/status.php?branch=generator

In case your branch is missing, or your branch is only compiled while you also want it tested, please ask Sjoerd or myself.

Modified branches will be tested in the order given above. Testing is stopped before 9am the next morning. In case not all modified branches could be tested by then, the left over ones will be postponed til the next night.

Please check the TestWeb (of "your" branch) whenever you checked in (i.e., a day later), and fix any compilation or testing errors at your earliest convenience. Our testing system also send you emails to remind you, in case there where any compilation failures after your checkin. (There are no emails about failing tests; you need to check that yourself.)

Having said that, automatic nightly testing is mainly there to check compilation and testing on other that your primary development platform. We still expect that you do verify the compilation (preferably both in debugging mode and in optimized mode) and run Mtest.py and verify its result on your primary development platform prior to pushing your changes.

In case you have any questions, suggestions, complaints, etc. about our automatic nightly testing systems or Mtest.py, please do not hesitate to contact Sjoerd or myself.

In case this is the first time you hear about Mtest.py or our automatic nightly testing system, you should definitely contact Sjoerd or Stefan!

ps: The "Nightly testing" link in the "Ongoing development" section

   of the left-hand "Overview" panel under http://www.monetdb.org/Home
   gets you to:

[1] http://monetdb.cwi.nl/testweb/web/status.php