A Result describes the outcome of a test.
A Result contains two pieces of data: an outcome and a set
of annotations. The outcome indicates whether the test passed
or failed. More specifically, the outcome may be one of the
following constants:
-
Result.PASS
- The test passed.
-
Result.FAIL
- The test failed.
-
Result.ERROR
- Something went wrong in the process of trying to
execute the test. For example, if the Python code implementing
the
Run method in the test class raised an exception, the
outcome would be Result.ERROR .
-
Result.UNTESTED
- QMTest did not even try to run the test.
For example, if a prerequiste was not satisfied, then this outcome
will be used.'
The annotations are a dictionary, mapping strings to strings.
The indices should be of the form class.name where class is
the name of the test class that created the annotation. Any
annotations created by QMTest, as opposed to the test class, will
have indices of the form qmtest.name .
The annotation values are HTML. When displayed in the GUI, the
HTML is inserted directly into the result page; when the
command-line interface is used the HTML is converted to plain
text.
Currently, QMTest recognizes the following built-in annotations:
-
Result.CAUSE
- For results whose outcome is not
FAIL , this
annotation gives a brief description of why the test failed. The
preferred form of this message is a phrase like "Incorrect
output." or "Exception thrown." The message should begin with a
capital letter and end with a period. Most results formatters
will display this information prominently.
-
Result.EXCEPTION
- If an exeption was thrown during the
test execution, a brief description of the exception.
-
Result.TARGET
- This annotation indicates on which target the
test was executed.
-
Result.TRACEBACK
- If an exeption was thrown during the test
execution, a representation of the traceback indicating where
the exception was thrown.
A Result object has methods that allow it to act as a dictionary
from annotation names to annotation values. You can directly add
an annotation to a Result by writing code of the form
'result[CAUSE] = "Exception thrown."'.
A Result object is also used to describe the outcome of
executing either setup or cleanup phase of a Resource .
Methods
|
|
|
|
Annotate
|
Annotate ( self, annotations )
Add annotations to the current set of annotations.
|
|
Fail
|
Fail (
self,
cause=None,
annotations={},
)
Mark the test as failing.
-
cause
- If not
None , this value becomes the value of the
Result.CAUSE annotation.
-
annotations
- The annotations are added to the current set
of annotations.
|
|
GetCause
|
GetCause ( self )
Return the cause of failure, if the test failed.
- returns
- If the test failed, return the cause of the
failure, if available.
|
|
GetId
|
GetId ( self )
Return the label for the test or resource.
- returns
- A label indicating indicating to which test or
resource this result corresponds.
|
|
GetKind
|
GetKind ( self )
Return the kind of result this is.
- returns
- The kind of entity (one of the
kinds ) to which
this result corresponds.
|
|
GetOutcome
|
GetOutcome ( self )
Return the outcome associated with the test.
- returns
- The outcome associated with the test. This value
will be one of the
Result.outcomes .
|
|
MakeDomNode
|
MakeDomNode ( self, document )
Generate a DOM element node for this result.
Note that the context is not represented in the DOM node.
-
document
- The containing DOM document.
- returns
- The element created.
|
|
NoteException
|
NoteException (
self,
exc_info=None,
cause=None,
outcome=ERROR,
)
Note that an exception occurred during execution.
-
exc_info
- A triple, in the same form as that returned
from
sys.exc_info . If None , the value of sys.exc_info()
is used instead.
-
cause
- The value of the
Result.CAUSE annotation. If
None , a default message is used.
-
outcome
- The outcome of the test, now that the exception
has occurred.
A test class can call this method if an exception occurs while
the test is being run.
|
|
Quote
|
Quote ( self, string )
Return a version of string suitable for an annotation value.
Performs appropriate quoting for a string that should be taken
verbatim; this includes HTML entity escaping, and addition of
<pre> tags.
-
string
- The verbatim string to be quoted.
- returns
- The quoted string.
|
|
SetOutcome
|
SetOutcome (
self,
outcome,
cause=None,
annotations={},
)
Set the outcome associated with the test.
-
outcome
- One of the
Result.outcomes .
-
cause
- If not
None , this value becomes the value of the
Result.CAUSE annotation.
-
annotations
- The annotations are added to the current set
of annotations.
|
|
__delitem__
|
__delitem__ ( self, key )
|
|
__getitem__
|
__getitem__ ( self, key )
|
|
__getstate__
|
__getstate__ ( self )
Return a representation of this result for pickling.
By using an explicit tuple representation of 'Result's when
storing them in a pickle file, we decouple our storage format
from internal implementation details (e.g., the names of private
variables).
|
|
__init__
|
__init__ (
self,
kind,
id,
outcome=PASS,
annotations={},
)
Construct a new Result .
-
kind
- The kind of result. The value must be one of the
Result.kinds .
-
id
- The label for the test or resource to which this
result corresponds.
-
outcome
- The outcome associated with the test. The value
must be one of the
Result.outcomes .
-
annotations
- The annotations associated with the test.
|
|
__setitem__
|
__setitem__ (
self,
key,
value,
)
|
|
__setstate__
|
__setstate__ ( self, pickled_state )
Construct a Result from its pickled form.
|
|
get
|
get (
self,
key,
default=None,
)
|
|
has_key
|
has_key ( self, key )
|
|
items
|
items ( self )
|
|
keys
|
keys ( self )
|
|