Difference between revisions of "Nightly testing policy"

From MonetDB
Jump to navigationJump to search
(Created page with "For your convenience, please check our nightly testing system [1] --- aka. "TestWeb" --- for the latest status of "your" branch as well as the branch that you forked your bran...")
 
 
(12 intermediate revisions by 3 users not shown)
Line 1: Line 1:
For your convenience, please check our nightly testing system [1] --- aka. "TestWeb" ---
+
For your convenience, please check our '''nightly testing system''' [1] --- aka. "''[http://monetdb.cwi.nl/testweb/web/status.php TestWeb]''" --- for the latest status of "your" branch as well as the branch that you forked your branch from and/or you sync your branch with.
for the latest status of "your" branch as well as the branch that you forked your
+
 
branch from and/or you sync your branch with.
+
Every night, starting from just after 7pm, the system compiles (and tests) each branch that has been modified since the last run on some 20+ platforms. Since testing takes quite some time (up to 5 hours on our slowest machines), only the most "important" branches, i.e., release (-candidate) branches, default branch, and branches that are supposed to be merged into the default development branch soon, are tested; the remaining ones are only compiled. The current list of branches is a follows
  
Every night, starting from just after 7pm, the system compiles (and tests) each branch
 
that has been modified since the last run on some 20+ platform. Since testing takes
 
quite some time (up to 5 hours on our slowest machines), only the most "important"
 
branches are tested; the remaining ones are only compiled. The current list of branches
 
is a follows
 
  
 
compiled & tested:
 
compiled & tested:
  
http://monetdb.cwi.nl/testweb/web/status.php?branch=Jan2014
+
http://monetdb.cwi.nl/testweb/web/status.php?branch=^default
http://monetdb.cwi.nl/testweb/web/status.php?branch=default
+
 
http://monetdb.cwi.nl/testweb/web/status.php?branch=SciQL-2
+
http://monetdb.cwi.nl/testweb/web/status.php?branch=Jul2015
http://monetdb.cwi.nl/testweb/web/status.php?branch=int128
+
 
 +
http://monetdb.cwi.nl/testweb/web/status.php?branch=profiler
 +
 
 +
http://monetdb.cwi.nl/testweb/web/status.php?branch=HTM
 +
 
 +
http://monetdb.cwi.nl/testweb/web/status.php?branch=geo
 +
 
 +
http://monetdb.cwi.nl/testweb/web/status.php?branch=notepad
 +
 
 +
http://monetdb.cwi.nl/testweb/web/status.php?branch=bamloader
 +
 
 +
http://monetdb.cwi.nl/testweb/web/status.php?branch=SciQL-2$
 +
 
  
 
only compiled:
 
only compiled:
  
 
http://monetdb.cwi.nl/testweb/web/status.php?branch=SciQL-2-NetCDF
 
http://monetdb.cwi.nl/testweb/web/status.php?branch=SciQL-2-NetCDF
http://monetdb.cwi.nl/testweb/web/status.php?branch=DVframework
+
 
 +
http://monetdb.cwi.nl/testweb/web/status.php?branch=DVframework$
 +
 
 
http://monetdb.cwi.nl/testweb/web/status.php?branch=DVframework_bam
 
http://monetdb.cwi.nl/testweb/web/status.php?branch=DVframework_bam
http://monetdb.cwi.nl/testweb/web/status.php?branch=bamloader
+
 
 
http://monetdb.cwi.nl/testweb/web/status.php?branch=holindex
 
http://monetdb.cwi.nl/testweb/web/status.php?branch=holindex
 +
 
http://monetdb.cwi.nl/testweb/web/status.php?branch=mutation
 
http://monetdb.cwi.nl/testweb/web/status.php?branch=mutation
 +
 
http://monetdb.cwi.nl/testweb/web/status.php?branch=rdf
 
http://monetdb.cwi.nl/testweb/web/status.php?branch=rdf
 +
 
http://monetdb.cwi.nl/testweb/web/status.php?branch=ssdb
 
http://monetdb.cwi.nl/testweb/web/status.php?branch=ssdb
 +
 
http://monetdb.cwi.nl/testweb/web/status.php?branch=xid
 
http://monetdb.cwi.nl/testweb/web/status.php?branch=xid
 +
 
http://monetdb.cwi.nl/testweb/web/status.php?branch=newstorage
 
http://monetdb.cwi.nl/testweb/web/status.php?branch=newstorage
http://monetdb.cwi.nl/testweb/web/status.php?branch=RIntegration
+
 
 
http://monetdb.cwi.nl/testweb/web/status.php?branch=bwd
 
http://monetdb.cwi.nl/testweb/web/status.php?branch=bwd
http://monetdb.cwi.nl/testweb/web/status.php?branch=geo
+
 
http://monetdb.cwi.nl/testweb/web/status.php?branch=multifarm
 
http://monetdb.cwi.nl/testweb/web/status.php?branch=transaction-replication
 
 
http://monetdb.cwi.nl/testweb/web/status.php?branch=bloomfilters
 
http://monetdb.cwi.nl/testweb/web/status.php?branch=bloomfilters
http://monetdb.cwi.nl/testweb/web/status.php?branch=generator
 
  
In case your branch is missing, or your branch is only compiled
+
http://monetdb.cwi.nl/testweb/web/status.php?branch=mosaic
while you also want it tested, please ask Sjoerd or myself.
+
 
 +
http://monetdb.cwi.nl/testweb/web/status.php?branch=BDCC
 +
 
 +
http://monetdb.cwi.nl/testweb/web/status.php?branch=Protobuf
 +
 
 +
http://monetdb.cwi.nl/testweb/web/status.php?branch=pyapi
 +
 
 +
http://monetdb.cwi.nl/testweb/web/status.php?branch=strip
 +
 
 +
http://monetdb.cwi.nl/testweb/web/status.php?branch=leftmart
 +
 
 +
http://monetdb.cwi.nl/testweb/web/status.php?branch=partitioned-hash
 +
 
 +
http://monetdb.cwi.nl/testweb/web/status.php?branch=embedded
 +
 
 +
http://monetdb.cwi.nl/testweb/web/status.php?branch=2PC
 +
 
 +
http://monetdb.cwi.nl/testweb/web/status.php?branch=DataVaults
 +
 
 +
http://monetdb.cwi.nl/testweb/web/status.php?branch=maven
 +
 
 +
 
 +
In case your branch is missing, or your branch is only compiled while you also want it tested, please ask Sjoerd or Stefan.
 +
 
 +
Modified branches will be tested in the order given above. Testing might be stopped before 9am the next morning, if machines need to be freed for interactive use. In case not all modified branches could be tested by then (or by 7pm the next day), the left over ones will be postponed til the next night.
  
Modified branches will be tested in the order given above.
+
''Please check the TestWeb (of "your" branch) whenever you checked in (i.e., a day later), and fix any compilation or testing errors at your earliest convenience. Our testing system also sends you emails to remind you, in case there where any compilation failures after your checkin. (There are no emails about failing tests; you need to check that yourself.)''
Testing is stopped before 9am the next morning.
 
In case not all modified branches could be tested by then,
 
the left over ones will be postponed til the next night.
 
  
Please check the TestWeb (of "your" branch) whenever you checked in
+
'''Having said that, automatic nightly testing is mainly there to check compilation and testing on other than your primary development platform.'''
(i.e., a day later), and fix any compilation or testing errors at
+
'''''We still expect that you do verify the compilation (preferably both in debugging mode and in optimized mode) and run [http://dev.monetdb.org/hg/MonetDB/file/a9696d037a93/testing/README Mtest.py] and verify its result on your primary development platform prior to pushing your changes.'''''
your earliest convenience. Our testing system also send you emails
 
to remind you, in case there where any compilation failures after
 
your checkin. (There are no emails about failing tests; you need to
 
check that yourself.)
 
  
Having said that, automatic nightly testing is mainly there to check
+
In case you have any questions, suggestions, complaints, etc. about our automatic nightly testing systems or [http://dev.monetdb.org/hg/MonetDB/file/a9696d037a93/testing/README Mtest.py], please do not hesitate to contact Sjoerd or Stefan.
compilation and testing on other that your primary development platform.
 
We still expect that you do verify the compilation (preferably both
 
in debugging mode and in optimized mode) and run Mtest.py and verify
 
its result on your primary development platform prior to pushing your
 
changes.
 
  
In case you have any questions, suggestions, complaints, etc.
+
In case this is the first time you hear about [http://dev.monetdb.org/hg/MonetDB/file/a9696d037a93/testing/README Mtest.py] or our automatic '''nightly testing system''', you should definitely contact Sjoerd or Stefan!
about our automatic nightly testing systems or Mtest.py,
 
please do not hesitate to contact Sjoerd or myself.
 
  
In case this is the first time you hear about Mtest.py
+
ps:
or our automatic nightly testing system,
 
you should definitely contact Sjoerd or Stefan!
 
  
ps: The "Nightly testing" link in the "Ongoing development" section
+
The "''Nightly testing''" link in the "''Ongoing development''" section of the left-hand "''Overview''" panel under http://www.monetdb.org/Home gets you to:
    of the left-hand "Overview" panel under http://www.monetdb.org/Home
 
    gets you to:
 
  
 
[1] http://monetdb.cwi.nl/testweb/web/status.php
 
[1] http://monetdb.cwi.nl/testweb/web/status.php

Latest revision as of 00:05, 7 October 2015

For your convenience, please check our nightly testing system [1] --- aka. "TestWeb" --- for the latest status of "your" branch as well as the branch that you forked your branch from and/or you sync your branch with.

Every night, starting from just after 7pm, the system compiles (and tests) each branch that has been modified since the last run on some 20+ platforms. Since testing takes quite some time (up to 5 hours on our slowest machines), only the most "important" branches, i.e., release (-candidate) branches, default branch, and branches that are supposed to be merged into the default development branch soon, are tested; the remaining ones are only compiled. The current list of branches is a follows


compiled & tested:

http://monetdb.cwi.nl/testweb/web/status.php?branch=^default

http://monetdb.cwi.nl/testweb/web/status.php?branch=Jul2015

http://monetdb.cwi.nl/testweb/web/status.php?branch=profiler

http://monetdb.cwi.nl/testweb/web/status.php?branch=HTM

http://monetdb.cwi.nl/testweb/web/status.php?branch=geo

http://monetdb.cwi.nl/testweb/web/status.php?branch=notepad

http://monetdb.cwi.nl/testweb/web/status.php?branch=bamloader

http://monetdb.cwi.nl/testweb/web/status.php?branch=SciQL-2$


only compiled:

http://monetdb.cwi.nl/testweb/web/status.php?branch=SciQL-2-NetCDF

http://monetdb.cwi.nl/testweb/web/status.php?branch=DVframework$

http://monetdb.cwi.nl/testweb/web/status.php?branch=DVframework_bam

http://monetdb.cwi.nl/testweb/web/status.php?branch=holindex

http://monetdb.cwi.nl/testweb/web/status.php?branch=mutation

http://monetdb.cwi.nl/testweb/web/status.php?branch=rdf

http://monetdb.cwi.nl/testweb/web/status.php?branch=ssdb

http://monetdb.cwi.nl/testweb/web/status.php?branch=xid

http://monetdb.cwi.nl/testweb/web/status.php?branch=newstorage

http://monetdb.cwi.nl/testweb/web/status.php?branch=bwd

http://monetdb.cwi.nl/testweb/web/status.php?branch=bloomfilters

http://monetdb.cwi.nl/testweb/web/status.php?branch=mosaic

http://monetdb.cwi.nl/testweb/web/status.php?branch=BDCC

http://monetdb.cwi.nl/testweb/web/status.php?branch=Protobuf

http://monetdb.cwi.nl/testweb/web/status.php?branch=pyapi

http://monetdb.cwi.nl/testweb/web/status.php?branch=strip

http://monetdb.cwi.nl/testweb/web/status.php?branch=leftmart

http://monetdb.cwi.nl/testweb/web/status.php?branch=partitioned-hash

http://monetdb.cwi.nl/testweb/web/status.php?branch=embedded

http://monetdb.cwi.nl/testweb/web/status.php?branch=2PC

http://monetdb.cwi.nl/testweb/web/status.php?branch=DataVaults

http://monetdb.cwi.nl/testweb/web/status.php?branch=maven


In case your branch is missing, or your branch is only compiled while you also want it tested, please ask Sjoerd or Stefan.

Modified branches will be tested in the order given above. Testing might be stopped before 9am the next morning, if machines need to be freed for interactive use. In case not all modified branches could be tested by then (or by 7pm the next day), the left over ones will be postponed til the next night.

Please check the TestWeb (of "your" branch) whenever you checked in (i.e., a day later), and fix any compilation or testing errors at your earliest convenience. Our testing system also sends you emails to remind you, in case there where any compilation failures after your checkin. (There are no emails about failing tests; you need to check that yourself.)

Having said that, automatic nightly testing is mainly there to check compilation and testing on other than your primary development platform. We still expect that you do verify the compilation (preferably both in debugging mode and in optimized mode) and run Mtest.py and verify its result on your primary development platform prior to pushing your changes.

In case you have any questions, suggestions, complaints, etc. about our automatic nightly testing systems or Mtest.py, please do not hesitate to contact Sjoerd or Stefan.

In case this is the first time you hear about Mtest.py or our automatic nightly testing system, you should definitely contact Sjoerd or Stefan!

ps:

The "Nightly testing" link in the "Ongoing development" section of the left-hand "Overview" panel under http://www.monetdb.org/Home gets you to:

[1] http://monetdb.cwi.nl/testweb/web/status.php