SITAR: GUI Test Script Repair
Zebao Gao, Zhenyu Chen, Yunxiao Zou, and Atif M. Memon
Supplementary Data -- TSE Revision
Notice

This page contains a supplementary data for our TSE paper. While it is under review, please do not link to, or share this page. We will make this page public upon publication of this work.

Abstract

System testing of a GUI-based application requires that test cases, consisting of sequences of user actions/events, be executed and the software’s output be verified. To enable automated re-testing, such test cases are increasingly being coded as low-level test scripts, to be replayed automatically using test harnesses. Whenever the GUI changes—widgets get moved around, windows get merged—some scripts become unusable because they no longer encode valid input sequences. Moreover, because the software’s output may have changed, their test oracles—assertions and checkpoints—encoded in the scripts may no longer correctly check the intended GUI objects. We present ScrIpT repAireR (SITAR), a technique to automatically repair unusable low-level test scripts. SITAR uses reverse engineering techniques to create an abstract test for each script, maps it to an annotated event-flow graph (EFG), uses repairing transformations and human input to repair the test, and synthesizes a new “repaired” test script. During this process, SITAR also repairs the reference to the GUI objects used in the checkpoints yielding a final test script that can be executed automatically to validate the revised software. SITAR ammortizes the cost of human intervention across multiple scripts by accumulating the human knowledge as annotations on the EFG. An experiment using QTP test scripts suggest that SITAR is effective in that 41–89% unusable test scripts were repaired. Annotations significantly reduced human cost after 20% test scripts had been repaired.

Framework

Framework

A logical perspective of the components of SITAR is shown in the figure on the left.

SITAR deals with three distinct spaces (levels of abstraction): (1) execution, (2) script, and (3) model.

Ripping raises the level from execution to model.

And mapping first raises the level of abstraction from script (for version n) to model and then, once the scripts have been repaired, from the model back to script (version n+1).


Experiment Settings
Supplemental Data

The QTP scripts used in our experiments are available for download from the following links. The scripts are named as StudentId_FUID_TCID where FUID is the id of the functional unit and TCID is the id of the test case. To view the contents of a script, please go to "Action1/Scripts.mts" under the directory of the test case.
Instructions for Creating Test Scripts