Assess test results
With TPT it’s possible to manually or automatically assess tests with a multitude of different, problem-specific techniques. The spectrum extends from simple checks of signal values at a given characteristic moment, to signal comparison by using reference curves, up to freely configurable assessment rules based on a script.
The test results and occuring deviations from the expected behavior are
recorded by TPT and listed in a clearly arranged test report.
Assess tests manually
Manual assessment of test results is done during the test execution in the Dashboard or after the test execution based on the recorded signal data in the Test Data Viewer.
Assess tests online
It's possible to define assessment rules that are executed during the test run to check predefined results. More complex inspections can be done by setting several branches with different conditions graphically in an automaton, or by using for example If-steps or While-steps in a test step list.
Assess tests offline
After the test execution, the test can be assessed based on the recorded test data. Graphical user interfaces for standardized checks like back-to-back analysis are available. You can also use an elaborated specific script language for more complex analysis.
The simplest form of test assessment is the manual assessment. TPT provides the Test Data Viewer for this kind of assessment. The Test Data Viewer enables the user to assess individual test runs in detail. It’s the signal data and the parameters that are displayed. Functions like zooming, scaling of the view, several parallel windows, managing of display configurations, as well as several cursors at the same time, and the display of differences are part of the Test Data Viewer.
As an alternative, the TPT–Dashboard can be used for creating customized interfaces that enable the test assessment
and the interaction with the system to be tested during the test run.
TPT Test Data Viewer
The Test Data Viewer in TPT can handle test and measurement data in different file formats like *.mat, *.mdf, *.dat, or *.csv. Browsing the different data and dependencies is easy and allows debugging and test evaluation.
- Import and export of measurement files
- Depiction of curves, arrays, and maps
- Synchronize irregular sampled signals
with the sampling time of the test signals
- Linear or hold last value interpolations
- Scalar data types can be converted from
decimal to binary or to hexadecimal
- Display of strings, structured, and integer signals
- Gaussian sample dragging
- Open several test cases at the same time
- Two cursor lines to measure the value between two points
- State sequence view
- Filter options
- Multiple views
- Several signal operations:
selecting, moving, zooming or editing
Assessing tests online
Within the test definition, input signals of the system to be tested are determined section by section. Just the same, expected values can be defined in the test model a priori. If a value has been reached inside a specific interval always or only once can be checked in TPT. The performance or non-performance of logical or time conditions is also observable.
Rule-based test assessment
The test assessment in TPT is based on the collected test data from the executed test run.
For typical and standardized assessments, a number of predefined graphical forms, called in TPT “assesslets”, are available.
The Min/Max comparison assesslet is used to check if a signal lies in a pre-defined value range or if values always occur between two other signals.
The signal comparison assesslet makes it possible to compare one or more signals in consideration of possible tolerances. The tolerances can be absolute, relative, or temporal. Tolerances can also depend on signal scaling.
Equivalence classes check
The equivalence classes check assesslet is used to inspect if the executed tests have been sufficient to cover all defined equivalence classes. The coverage of the equivalence classes is document in a statistical overview
Check causal rules
Using the trigger rule assesslet, if-then rules with temporal and logical marginal conditions can be checked. The check itself is activated or deactivate by specific activation /deactivation conditions. Restrictions in time and stop criteria can be set.
The sequence check assesslet can inspect consecutive phases based on signal characteristics.
If test results from former tests are available, a regression or back-to-back test is an option. That is, the new test results are compared with the old ones. This way, it can be automatically verified if the system still behaves the same. If differences are detected, TPT will document these deviations as well as their location of occurrence during the test run.
TPT can handle regression tests with fault-tolerant signal comparison. A tolerant range for the reference data can be specified. Temporal, absolute, or relative tolerance is definable in TPT as well as tolerances depending on the scaling of the signals. This scaling-dependent tolerance is especially useful for the comparison of physical tests as Model-in-the-Loop (MiL) with data from Software-in-the-Loop (SiL) tests.
Script based test assessment
In TPT, a very powerful and flexible script language can be used for the test assessment. With this so-called TPT assessment script, it’s possible to calculate signals deduced from recorded test data, to identify test phases, to assess signals and their dependencies, to document the analysis in a report, and to reuse repetitive assessment rules as libraries.
The TPT assessment script is based on Python, so that besides the TPT-specific functions,
all other Python functions can be used.
Test assessments can be test case specific or they can represent universal rules that are run in all test cases
to observe the fulfillment or non-fulfillment of the rules.
To extract interesting information from signals, often a preprocessing is necessary. In TPT, calculations can be performed with single or more signals. To run these calculations, mathematic operators and signal processing functions like filter functions are available.
Test phases can be found in signals by setting up temporal example descriptions. In the identified test phases, further restrictions of phases can be calculated or characteristics can be checked.
The TPT report summarizes all test results in a clear manner.
It is an extensive and individually adjustable summary of the test and
assessment details wherein charts and interactive signal diagrams are available.
The report can be generated in HTML or PDF.
Besides configurable default elements, the report can contain additional information from the dynamic test assessments.