Test assessment in TPT.
As a good tester, you make sure that the tests are performed correctly. You also have expectations of the test result when you create the test cases. TPT supports you in formulating these expectations in such a way that an automatic assessment can be performed by TPT after test execution.
The automatic test assessment by TPT can be repeated without errors, which considerably reduces your test effort.
With TPT you can:
- define expected values directly in the test cases
- define expectations across test cases
- easily specify conditional assessments in test phases
- compare different test executions with each other (regression test and back-to-back test)
You can of course also assess tests manually with TPT. The Signal Viewer and the freely configurable Dashboard are available for this purpose.
Assess tests manually.
Two graphical tools are available in TPT for manual assessment: the Signal Viewer and the Dashboard.
With the Signal Viewer you can graphically analyze test data in detail. Functions such as zoom, view scaling, multi-windowing, management of display configurations as well as the use of multiple cursors and difference displays are possible.
With PikeTec’s Dashboard, you can easily create graphical user interfaces that allow you to interact with the system under test and receive visual feedback in real-time.
TPT Signal Viewer
The Signal Viewer in TPT can handle test and measurement data in different file formats like *.mat, *.mdf, *.dat, or *.csv. Browsing the different data and its dependencies is easy and allows debugging and test evaluation.
TPT Dashboard
Define expected values in test cases.
The simplest form of assessment is the specification of expected values in the test cases. Here, directly next to the test stimulation, the expected values are specified as system output. In TPT, it is very easy to include tolerance specifications.
Regression tests and back-to-back tests.
With TPT, you can easily compare test results from multiple test executions.
This means that if you make changes to the software under test after the test run, you can run the same tests again and TPT then compares the results of both test runs for you (regression testing).
If you are developing different software components that should behave identically, simply use the same test setting for all components and TPT will automatically compare the test results for you (back-to-back test).
The same applies if you test a software component as a model, then convert it into program code, use it as compiled code on a processor or use it in a hardware environment. Just use the same test setting for the MiL test, the SiL test, the PiL test and the HiL test, and TPT compares the test results of the different test executions for you (back-to-back test). For robustness it is very easy to include time and value tolerances.
Assess tests rule-based with assesslets.
TPT uses so-called Assesslets for test assessment. Assesslets are pre-defined graphical forms that allow you to specify your expectations on the behavior of signals in no time.
To make an assessment if, for example, the signal exceeds a certain value, you can express this as a condition. You can also specify that the assessment only takes place if a particular state variant is used or a particular test phase is executed.
Assesslets can be specified independently of the test cases or linked to tests and test phases. If all assesslets test positive, the test case turns green.
Check value ranges
The Min/Max Comparison assesslet is used to check if a signal lies in a predefined value range or if values always occur between two other signals.
Compare signals
The Signal Comparison assesslet makes it possible to compare one or more signals in consideration of possible tolerances. The tolerances can be absolute, relative, or temporal. Tolerances can also depend on signal scaling.
Check equivalence classes
The Equivalence Classes assesslet is used to inspect if the executed tests have been sufficient to cover all defined equivalence classes. The coverage of the equivalence classes is document in a statistical overview
Check causal rules
Using the Trigger Rule assesslet, if-then rules with temporal and logical marginal conditions can be checked. The check itself is activated or deactivate by specific activation /deactivation conditions. Restrictions in time and stop criteria can be set.
Check sequences
The Sequence Check assesslet can inspect consecutive phases based on signal characteristics.
Condition Tree
The Condition Tree assesslet is used to check a number of rules and conditions inside a time context, based on a
tree structure.
You can use this assesslet for:
- checking for signal boundaries (min and max)
- comparing signals
- defining signal data by calculation
- defining intervals and conditions
Assess tests using a script.
For lovers of scripting languages, we have extended Python to allow any test assessment.
This so-called TPT assessment language enables you to
- define additional variables that can be used for intermediate calculations
- assess at specific time intervals
- analyze signals and their dependencies
- control the report
You create the script in the so-called Script Assesslet, which you then link to test cases and requirements.
If desired, the results of the script-based analyses can be displayed in the test report. If you want to reuse script-based analyses in different TPT projects, you can create Assessment Libraries.
The TPT assessment language is based on Python, so that all Python functionalities are available in addition to the TPT-specific functions.
Signal preprocessing
To extract interesting information from signals, often a preprocessing is necessary. In TPT, calculations can be performed with single or more signals. To run these calculations, mathematic operators and signal processing functions like filter functions are available.
Extracting interesting phases
Test phases can be found in signals by setting up temporal example descriptions. In the identified test phases, further restrictions of phases can be calculated or characteristics can be checked.
Assessment functions
To assess tests, the TPT assessment provides a multitude of functions. Among other things, runtime, gradients, signal leaps, differences, and chronological orders can be inspected.
Assess requirements.
Test cases are used to check whether requirements are fulfilled or not. Accordingly, each test case should be linked to at least one requirement. With TPT you keep track of which test cases are linked to which requirements.
In TPT, you can also link requirements to assesslets. If the test case is then positively assessed, the linked requirement is also marked as passed. If the test case fails according to the assessment rules, the linked requirement is also marked as failed.
The advantage is obvious: you can see at a glance which requirements have not been fulfilled, which have failed and which have not even been tested.
Assess across test cases.
You want to know without much effort whether, for example, the average value of a signal for all test cases of a test execution is between 20 and 30 or whether a certain error occurred? Then the “global assessment” in TPT is just the right thing for you.
With the “global assessment” in TPT, you can quickly and easily perform calculations and assessments across test cases.
Form fields are available for specifying the most frequent calculations, for example the caluclation of the mean value or the sum of all values.
You can perform further calculations across test cases with your own global assessment script. The script also allows you to adjust the display of the assessments in the test report.
The results of the “global assessments” are listed separately in the report.
