PhD Proposal: Model Based Testing of Off-Nominal Behaviors

Talk
Christoph Schulze
Time: 
07.02.2015 10:00 to 11:30
Location: 

AVW 3450

Off-nominal behaviors (ONB) are unexpected or unintended behaviors that may be exhibited by the implementation of a system. They can be triggered by unanticipated external stimuli, such as unforeseen sequences of events, out of range data values, or environmental issues. System requirements, which typically focus on nominal behavior, often do not refer to ONB, their causes, and how the system should react to occurrences of them. Untested occurrences of ONB can compromise the safety and reliability of a system. This can be very dangerous in safety critical systems, like spacecraft, where software issues can lead to expensive mission failures, injuries, or even loss of life. In order to ensure the safety of the system, potential causes for ONB need to be identified and their handling in the implementation has to be verified and documented.
In present practice Model Based Testing (MBT) techniques have been used in other areas of software testing to provide automated support for thorough evaluation of software behavior. In MBT, models are used to describe the system under test (SUT) and to derive test cases for that SUT. The MBT approach requires the formalization of requirements into models. This formalization step helps with the identification of issues in the requirements. Furthermore, automated test generation algorithms allow the testers to create a large number of test cases from these models. However, MBT is still mostly used for the testing of nominal behaviors. The main challenge of testing ONB with MBT is that a system can have many more ONB then nominal ones. This leads to larger and more complex models, which increases the overall effort and skill that is necessary to conduct MBT.
This work proposes a model-based framework for the identification, verification and documentation of ONB. The proposed approach is divided into three parts: model generation, test case generation and requirement extraction. The model generation is used to identify OBN by automatically creating off-nominal models from existing nominal models, thereby, reducing the complexity and possible errors of this process. In addition, test case generation contributes to the identification and verification of OBN by creating specific test cases that aim to uncover ONB. This process is based on observations of the system runtime behavior, on modeling patterns, and on information within nominal and off-nominal models. In order to document ONB and to complete the requirements of the system with respect to the ONB, we propose to employ requirement extraction from existing development artifacts by using invariant mining and temporal logic query checking. The framework and a tool chain that will support the tester throughout the process will be evaluated on different safety critical systems in terms of its effectiveness and efficiency by comparing the results to our previous case studies.
Examining Committee:
Committee Chair: - Dr. Rance Cleaveland
Dept's Representative - Dr. Donald Perlis
Committee Member(s): - Dr. Adam Porter
- Dr. Michael Lindvall