tests: Support tests that are expected to fail.

Authored by justus on May 4 2017, 3:12 PM.

Description

tests: Support tests that are expected to fail.

* tests/gpgscm/tests.scm (test-pool): Rework reporting.  Filter using
the computed test status instead of the return value.  Also print the
new categories 'failed expectedly' and 'passed unexpectedly'.
(test): If a test ends with a bang (!), it is expected to fail.  Adapt
status, status-string, and xml accordingly.

Allow tests to be marked as being expected to fail by appending a bang
(!) to the tests name. If such a test fails, it will not be counted
as failure, but will still be prominently displayed in the report. If
it succeeds unexpectedly, this is counted as a failure.

Fixes T3134.

  • GnuPG-bug-id: T3134
  • Signed-off-by: Justus Winter <justus@g10code.com>

Details

Committed
justusMay 4 2017, 3:22 PM
Parents
rE272b0ad65ec4: gpgscm: Create and re-use frame objects.
Branches
Unknown
Tags
Unknown
Tasks
T3134: Handle tests that are expected to fail