On Wed, Aug 15, 2007 at 02:54:29PM +0200, Fabian Groffen wrote:
On 15-08-2007 14:48:12 +0200, Stefan Manegold wrote: ...
exit_on_error should not be necessary, if you assume the following:
- the language being used has control for errors itself, e.g. SQL's auto-commit, or
- an error should not happen in batches, thus
- the client continues executing when reading from stdin, but dies on the first error when reading from file
Don't know how much this would affect testweb, but somewhere I feel it is only in an interactive session sensible to continue after an error.
According to Stefan this does not affect the test web.
That's NOT what I said.
I said:
- changing the default SQL-rendering of MapiClient in case input is received via stdin (as opposed to from a file given on the commandline) did (apparenly) not affect testing, since Mtest.py runs SQL tests via `MapiClient -lsql TST.sql` and not via `MapiClient -lsql < TST.sql` (I derived this from running Mtest.py after Martin's respective checkin)
Martin didn't implement anything, he only threw away the exit_on_error thing, which is fine with me too.
I was refering to =================================================================== 2007/08/13 - mlkersten: clients/src/mapiclient/MapiClient.mx,1.24 With a better SQL rendering, we might as well use it as the default for stdin. All other use remains the same, so it does not affect the SQL testing output. =================================================================== which sounds like Martin implemented something.
I considered Martin's "According to Stefan this does not affect the test web." as refering to the lunch discussion we had about above checkin.
In fact, this is NOT true: Also for SQL, like for MIL client and server tests, we give the tests file content via stdin to MapiClient and/or Mserver instead of providing the file ion the commandline, i.e., `MapiClient -lLANG < TST.EXT` `Mserver < TST.milS` instead of `MapiClient -lLANG TST.EXT < /dev/null` `Mserver TST.milS < /dev/null`
We do this explicitly to keep tests running after the first error, as we want to be able to have tests that are supposed to trigger multiple errors, or to simply proceed after an error, instead of stopping immediately after the first error.
Only for M5/MAL and XQuery, we give the test file on the commandline, i.e., `MapiClient -lxquery TST.xq < /dev/null` `mserver5 TST.mal < /dev/null` maily bacause the former (XQuery) did not support stdin initially, and the latter (M5) show(ed|s) significantly(?) different behavious between `mserver5 TST.mal < /dev/null` and `mserver5 < TST.mal`
Hence,
(1) I'm actually not sure, why changing the default SQL-rendering of MapiClient in case input is receivedvia stdin did/does not affect SQL testing --- possibly, because receiving input redirected from a file is (still) different from an interactive session, and only the latter has been modified?
Rendering has nothing to do with this, IMO.
Neither had my statement about effects of rendering changes (triggered by a checkin message that was talking about rendering changes, only; see above) anything to do with changing MapiClient commandline options and/or changing the behaviour of MapiClient in case of errors. Apparently not being phrased clearly/precisely enough, my orginal statement got mis-interpreted as "According to Stefan this(?) does not affect the test web.". I just tried to clearify this.
It's about quiting after the first error encountered or not. Anyway, I think testweb should just pass --raw or something to MapiClient when using it, to obtain the raw protocol view that was normal before Martin started working on this.
Fine with me. So far, I haven't seen any request to do any changes to Mtest.py's use of MapiClient.
(2) Changes in the currect behaviour reagarding the treatment of errors can well affect testing!
That was what I was wondering, with regard to my suggested scheme.
(3) The actual results of running testing (Mtest.py) is the *only* indication whether and how any code changes affect testing. Hence, simply run Mtest.py, and you'll know. That's *excatly* what testing and Mtest.py have been designed and implemented for, and what the are supposed to be used for.
Sometimes asking someone is much faster than implementing the code, and running Mtest.
Right. And I answered also though I was not asked ;-)
Stefan