Definitions
ISTQB defines regression testing as the retesting of an already tested program or a partial functionality after its modification. The aim is to prove that no error states have been introduced or (previously masked error states) exposed as a result of the changes made.
ISTQB defines back-to-back testing as comparing two or more variants of a System under Test or a simulation model of the same System under Test by executing the same test cases for all variants and comparing the results.
The main difference between these two dynamic test methods is the type of test oracle. A test oracle is the assessment basis for the success of a test case.
In regression testing, the evaluation is based on expected results derived from requirements. In back-to-back testing, the expected result is the same behavior to another software version.
From our point of view, a regression test is always suitable if you want to make or determine a statement regarding your functional behavior. If tests are linked to requirements, bugs in features and requirements can be clearly and unambiguously assigned and assessable.
A Continuous Integration & Continuous Testing (CI/CT) environment represents an automation of regression tests. In most configurations, tests are performed daily based on changes. Through the CI/CT construction plan, only tests that have an impact on product are performed.
The back-to-back test is always very suitable when the reference, i.e. the original version of your software, is highly trusted. This is the case, among other things, when code is generated from a model. High confidence is given if the model has already been extensively tested.
In practice, especially for conversions in floating point arithmetic, the settings back-to-back tests can be challenging, e.g. when converting models to C code.
At first glance, setting tolerances seems straightforward.
With cascades of signals, errors can propagate and settings such as Least Significant Bit (LSB) as a tolerance value must be reconsidered. If tolerances are chosen too small, tests will fail incorrectly. If the tolerance is chosen too large, tests always pass and the significance of the back-to-back test is lost. The correct setting thus becomes a technically difficult engineering task.
Summary
In TPT, both regression testing and back-to-back testing can be performed and both testing methods can be combined in one execution.
We recommend that test managers check the operational suitability in terms of effort and benefit by conducting their own investigations before selecting the test methodology and make a conscious decision.
Related topics
