Test reporting in TPT.
It is useful to document tests. Standards such as IEEE829 provide guidance on what should be documented. The following recommended documentation is automatically generated by TPT:
- Level Test Design (LTD): Describes the test cases of the test level and the expected behaviour and test criteria.
- Level Test Case (LTC): Describes the test implementation and test data from the LTD.
- Level Test Procedure (LTPr): Describes how each test case runs including initialization and preconditions.
- Level Test Log (LTL): The test logs document who performed what when, where and in which order and what the test result is.
- Anomaly Report (AR): The report of deviations from the desired results documents abnormalities during the test, including unexpected results and test aborts. This report should help to identify the reason for the deviation.
- Level Interim Test Status Report (LITSR): The interim report optionally summarizes interim test results in order not to wait for the final report.
- Level Test Report (LTR): The LTR finally summarizes the test results of the test level and contains assessments.
You only need to generate the test strategy documents and cross-test level documentation such as the Master Test Plan (MTP), Level Test Plan (LTP) and Master Test Report (MTR) outside of test automation in so-called test management tools. TPT provides interfaces for this purpose.
Test results reports can be generated with TPT as HTML, PDF or XML.
Overview report.
The TPT Overview Report contains an overview of all test results of a test execution. For each test case it is briefly documented whether the test was successful, failed, had test execution problems or the result is unclear.
Additionally, important meta-information is given, e.g. about the tester, the test environment, the time as well as statistics about test results, requirement coverage, test assessments and code coverage.
The overview report can be adapted to your own needs.
Test case report.
An individual report is created for each test case. In addition to the meta-information, this report contains the test procedure including parameters, constants and signal characteristics, the test specification, as well as detailed information on the test evaluations of the TPT assessments. The information is displayed graphically. The report can be configured freely. You can decide for yourself what information you want to display and how it is displayed. With a click on the signal data in the report they can be opened in TPT’s Signal Viewer for manual evaluation.
Besides information about the execution, such as test case name, execution environment or execution date, there are lists of the linked requirements and the respective requirement-specific test result.
Requirements report.
In the requirements report the coverage matrix of the requirements can be easily and clearly understood. In addition to statistics on coverage, there is a detailed test result for each individual requirement. If requirements are linked to assessment rules created in TPT (so-called assesslets), there is information about how often and how accurately the respective requirement was assessed.
Variables report.
The statistics on the assessment rules (assesslets) and the assessment variables created in TPT, which finally contributed to the test assessment, can be found in the variable report. Here you can see exactly which test has been assessed how often and with which result.
Global Assessment report.
The Global Assessment Report presents the assessment rules and the results of the global assessment. The global assessment is used to determine irregularities or deviations across all test cases of a test run.
Test documentation in TPT.
Test cases in TPT can of course be documented. For this purpose, test case attributes can be created for each test case, in which test case specification, preconditions, acceptance criteria, but also variant assignments, links to other documents, etc. are stored.
TPT test documentation can be easily exchanged with test management tools or Excel.
Each individual test step and assessment can be commented on. The documentation can be displayed in the report depending on the test result.