A TETStream formats results as a TET journal.
Provides special handling for DejaGNUTest results.
TET: http://tetworks.opengroup.org/
TET journal format: see appendix C and D of
http://tetworks.opengroup.org/documents/3.7/uguide.pdf
For the meaning of TET result codes, we use as guidelines the LSB
test faq, question Q1.11:
PASS - a test result belonging to this group is considered to
be a pass for compliance testing purposes:
o Pass - the test has been executed correctly and to
completion without any kind of problem
o Warning - the functionality is acceptable, but you
should be aware that later revisions of the relevant
standards or specification may change the requirements
in this area.
o FIP - additional information is provided which needs to
be checked manually.
o Unsupported - an optional feature is not available or
not supported in the implementation under test.
o Not in Use - some tests may not be required in certain
test modes or when an interface can be implemented by a
macro or function and there are two versions of the test
only one is used.
o Untested - no test written to check a particular feature
or an optional facility needed to perform a test is not
available on the system.
[There are also "notimp" and "unapproved" cases mentioned in
the LSB-FHS README, but they are otherwise undocumented, and
don't correspond to any DejaGNU or QMTest outcomes anyway.]
FAIL - a test result belonging to this group is considered to
be a fail for compliance testing purposes (unless the failure
has been waived by an agreed Problem Report in the
Certification Problem Reporting database):
o Fail - the interface did not behave as expected.
o Uninitiated - the particular test in question did not
start to execute.
o Unresolved - the test started but did not reach the
point where the test was able to report success or
failure.
o Unreported - a major error occurred during the testset
execution. (The TET manual calls this NORESULT.)
(From http://www.linuxbase.org/test/lsb-runtime-test-faq.html )
DejaGNU test results are described as:
PASS - A test has succeeded.
FAIL - A test has produced the bug it was intended to
capture.
WARNING - Declares detection of a minor error in the test case
itself. Use WARNING rather than ERROR for cases (such as
communication failure to be followed by a retry) where the
test case can recover from the error. Note that sufficient
warnings will cause a test to go from PASS/FAIL to
UNRESOLVED.
ERROR - Declares a severe error in the testing framework
itself. An ERROR also causes a test to go from PASS/FAIL to
UNRESOLVED.
* UNRESOLVED - A test produced indeterminate results. Usually,
this means the test executed in an unexpected fashion; this
outcome requires that a human being go over results, to
determine if the test should have passed or failed. This
message is also used for any test that requires human
intervention because it is beyond the abilities of the testing
framework. Any unresolved test should be resolved to PASS or
FAIL before a test run can be considered finished. Examples:
- a test's execution is interrupted
- a test does not produce a clear result (because of
WARNING or ERROR messages)
- a test depends on a previous test case which failed
UNTESTED - a test case that isn't run for some technical
reason. (E.g., a dummy test created as a placeholder for a
test that is not yet written.)
UNSUPPORTED - Declares that a test case depends on some
facility that does not exist in the testing environment; the
test is simply meaningless.
(From a combination of DejaGNU manual sections "Core Internal
Procedures", "C Unit Testing API", and "A POSIX conforming test
framework".)
Methods
|
|
|
|
Summarize
|
Summarize ( self )
|
|
WriteAnnotation
|
WriteAnnotation (
self,
key,
value,
)
|
|
WriteResult
|
WriteResult ( self, result )
|
|
_ExtractTime
|
_ExtractTime (
self,
result,
key,
)
Extracts the start time from a result.
|
|
_IsDejaGNUResult
|
_IsDejaGNUResult ( self, result )
Returns True if result has DejaGNU subtests.
|
|
_TETFormatTime
|
_TETFormatTime ( self, time_string )
Converts an ISO-format date-time to a TET-format date-time.
- returns
- A 2-tuple whose first element is the time as a string,
and whose second is the date as a string.
|
|
_WriteDejaGNUResult
|
_WriteDejaGNUResult ( self, result )
Write out a result that has DejaGNU subtest information.
|
|
_WriteInitialStuff
|
_WriteInitialStuff ( self )
Print TET header information, but only on first call.
Second and later calls are no-ops.
|
|
_WriteLine
|
_WriteLine (
self,
code,
data,
comment,
)
Write a line in TET journal format.
|
|
_WriteResourceResult
|
_WriteResourceResult ( self, result )
Write out information on a resource result.
TET has no concept of resources, so we ignore successful
resources, and print out "test case controller messages" for
ERRORs and FAILUREs.
|
|
_WriteResultAnnotations
|
_WriteResultAnnotations (
self,
result,
purpose,
num_restrict=None,
seq_start=1,
)
Writes out annotations for a result in TET format.
Annotations are represented as (sequences of) "test case
information" lines.
-
result
- The
Result whose annotations should be written.
-
num_restrict
- Only write out annotations that end with this
number. If the number is
1 , also writes out all results that
don't end in any number, with "INFO: " prefixed. If None ,
writes out all annotations.
-
seq_start
- The TET test case information sequence number to
start with.
|
|
_WriteTCStart
|
_WriteTCStart ( self, result )
Write a TET test case start line.
|
|
_WriteTestResult
|
_WriteTestResult ( self, result )
Write out a result that does not have DejaGNU annotations.
|
|
__init__
|
__init__ ( self, arguments )
|
|