Assess test results

With TPT it’s possible to manually or automatically assess tests with a multitude of different, problem-specific techniques. The spectrum extends from simple checks of signal values at a given characteristic moment, to signal comparison by using reference curves, up to freely configurable assessment rules based on a script.

The test results and occuring deviations from the expected behavior are
recorded by TPT and listed in a clearly arranged test report.


Assess tests manually

Manual assessment of test results is done during the test execution in the Dashboard or after the test execution based on the recorded signal data in the Test Data Viewer.

Assess tests online

It's possible to define assessment rules that are executed during the test run to check predefined results. More complex inspections can be done by setting several branches with different conditions graphically in an automaton, or by using for example If-steps or While-steps in a test step list.

Assess tests offline

After the test execution, the test can be assessed based on the recorded test data. Graphical user interfaces for standardized checks like back-to-back analysis are available. You can also use an elaborated specific script language for more complex analysis.

Manual assessment

The simplest form of test assessment is the manual assessment. TPT provides the Test Data Viewer for this kind of assessment. The Test Data Viewer enables the user to assess individual test runs in detail. It’s the signal data and the parameters that are displayed. Functions like zooming, scaling of the view, several parallel windows, managing of display configurations, as well as several cursors at the same time, and the display of differences are part of the Test Data Viewer.

As an alternative, the TPT–Dashboard can be used for creating customized interfaces that enable the test assessment
and the interaction with the system to be tested during the test run.


TPT Test Data Viewer

The Test Data Viewer in TPT can handle test and measurement data in different file formats like *.mat, *.mdf, *.dat, or *.csv. Browsing the different data and dependencies is easy and allows debugging and test evaluation.

  • Import and export of measurement files
  • Depiction of curves, arrays, and maps
  • Synchronize irregular sampled signals
    with the sampling time of the test signals
  • Linear or hold last value interpolations
  • Scalar data types can be converted from
    decimal to binary or to hexadecimal
  • Display of strings, structured, and integer signals
  • Gaussian sample dragging
  • Open several test cases at the same time
  • Two cursor lines to measure the value between two points
  • State sequence view
  • Filter options
  • Multiple views
  • Several signal operations:
    selecting, moving, zooming or editing
You can use several filter options, several windows, signal display modes, and two cursors in the Test Data Viewer for the manual evaluation.
5 images  

Dashboard

The TPT-Dashboard enables the user to create customized interfaces, observe test executions, and interact with the system to be tested. This way, the test assessment is easy
and can be problem-oriented.

With the Dashboard, you can stimulate a system by using grafical elements and also observe the behavior of the system during runtime.

Assessing tests online

Within the test definition, input signals of the system to be tested are determined section by section. Just the same, expected values can be defined in the test model a priori. If a value has been reached inside a specific interval always or only once can be checked in TPT. The performance or non-performance of logical or time conditions is also observable.


Compare steps belong to the assessments, and are inserted in the test step where the comparison has to be done.
2 images  

Rule-based test assessment

The test assessment in TPT is based on the collected test data from the executed test run.
For typical and standardized assessments, a number of predefined graphical forms, called in TPT “assesslets”, are available.


Min/Max comparison

The Min/Max comparison assesslet is used to check if a signal lies in a pre-defined value range or if values always occur between two other signals.

Signal comparison

The signal comparison assesslet makes it possible to compare one or more signals in consideration of possible tolerances. The tolerances can be absolute, relative, or temporal. Tolerances can also depend on signal scaling.

Equivalence classes check

The equivalence classes check assesslet is used to inspect if the executed tests have been sufficient to cover all defined equivalence classes. The coverage of the equivalence classes is document in a statistical overview

Check causal rules

Using the trigger rule assesslet, if-then rules with temporal and logical marginal conditions can be checked. The check itself is activated or deactivate by specific activation /deactivation conditions. Restrictions in time and stop criteria can be set.

Sequence check

The sequence check assesslet can inspect consecutive phases based on signal characteristics.

Regression tests

If test results from former tests are available, a regression or back-to-back test is an option. That is, the new test results are compared with the old ones. This way, it can be automatically verified if the system still behaves the same. If differences are detected, TPT will document these deviations as well as their location of occurrence during the test run.

TPT can handle regression tests with fault-tolerant signal comparison. A tolerant range for the reference data can be specified. Temporal, absolute, or relative tolerance is definable in TPT as well as tolerances depending on the scaling of the signals. This scaling-dependent tolerance is especially useful for the comparison of physical tests as Model-in-the-Loop (MiL) with data from Software-in-the-Loop (SiL) tests.


TPT is able to run regression tests.

Script based test assessment

In TPT, a very powerful and flexible script language can be used for the test assessment. With this so-called TPT assessment script, it’s possible to calculate signals deduced from recorded test data, to identify test phases, to assess signals and their dependencies, to document the analysis in a report, and to reuse repetitive assessment rules as libraries.

The TPT assessment script is based on Python, so that besides the TPT-specific functions,
all other Python functions can be used.

Test assessments can be test case specific or they can represent universal rules that are run in all test cases
to observe the fulfillment or non-fulfillment of the rules.


Signal preprocessing

To extract interesting information from signals, often a preprocessing is necessary. In TPT, calculations can be performed with single or more signals. To run these calculations, mathematic operators and signal processing functions like filter functions are available.

In the script assesslet, you can use special TPT filter functions. The context menu helps you to find out how and for what to use them.

Extracting phases

Test phases can be found in signals by setting up temporal example descriptions. In the identified test phases, further restrictions of phases can be calculated or characteristics can be checked.

Script assesslet with TPT assessment functions.

Assessment

To assess tests, the TPT assessment provides a multitude of options. Among other things, runtime, gradients, signal leaps, differences, and chronological orders can be inspected.

In the script assesslet, you can use a bunch of special TPT functions for the assessment. The context help will show you how and when to use them.

Reporting

The TPT report summarizes all test results in a clear manner.
It is an extensive and individually adjustable summary of the test and
assessment details wherein charts and interactive signal diagrams are available.
The report can be generated in HTML or PDF.

Besides configurable default elements, the report can contain additional information from the dynamic test assessments.


Copyright © www.piketec.com    All rights reserved

Follow us