Towards model-based regression test selection
Date
2019
Authors
Al-Refai, Mohammed, author
Ghosh, Sudipto, advisor
Cazzola, Walter, advisor
Bieman, James M., committee member
Ray, Indrakshi, committee member
Vijayasarathy, Leo, committee member
Journal Title
Journal ISSN
Volume Title
Abstract
Modern software development processes often use UML models to plan and manage the evolution of software systems. Regression testing is important to ensure that the evolution or adaptation did not break existing functionality. Regression testing can be expensive and is performed with limited resources and under time constraints. Regression test selection (RTS) approaches are used to reduce the cost. RTS is performed by analyzing the changes made to a system at the code or model level. Existing model-based RTS approaches that use UML models have some limitations. They do not take into account the impact of changes to the inheritance hierarchy of the classes on test case selection. They use behavioral models to perform impact analysis and obtain traceability links between model elements and test cases. However, in practice, structural models such as class diagrams are most commonly used for designing and maintaining applications. Behavioral models are rarely used and even when they are used, they tend to be incomplete and lack fine-grained details needed to obtain the traceability links, which limits the applicability of the existing UML-based RTS approaches. The goal of this dissertation is to address these limitations and improve the applicability of model-based RTS in practice. To achieve this goal, we proposed a new model-based RTS approach called FLiRTS 2. The development of FLiRTS 2 was driven by our experience accrued from two model-based RTS approaches. The first approach is called MaRTS, which we proposed to incorporate the information related to inheritance hierarchy changes for test case selection. MaRTS is based on UML class and activity diagrams that represent the fine-grained behaviors of a software system and its test cases. The second approach is called FLiRTS, which we proposed to investigate the use of fuzzy logic to enable RTS based on UML sequence and activity diagrams. The activity diagrams lack fine-grained details needed to obtain the traceability links between models and test cases. MaRTS exploits reverse engineering tools to generate complete, fine-grained diagrams from source code. FLiRTS is based on refining a provided set of abstract activity diagrams to generate fine-grained activity diagrams. We learned from our experience with MaRTS that performing static analysis on class diagrams enables the identification of test cases that are impacted by changes made to the inheritance hierarchy. Our experience with FLiRTS showed that fuzzy logic can be used to address the uncertainty introduced in the traceability links because of the use of refinements of abstract models. However, it became evident that the applicability of MaRTS and FLiRTS is limited because the process that generates complete behavioral diagrams is expensive, does not scale up to real world projects, and may not always be feasible due to the heterogeneity, complexity, and size of software applications. Therefore, we proposed FLiRTS 2, which extends FLiRTS by dropping the need for using behavioral diagrams and instead relying only on the presence of UML class diagrams. In the absence of behavioral diagrams, fuzzy logic addresses the uncertainty in determining which classes and relationships in the class diagram are actually exercised by the test cases. The generalization and realization relationships in the class diagram are used to identify test cases that are impacted by the changes made to the inheritance hierarchy. We conducted a large evaluation of FLiRTS 2 and compared its safety, precision, reduction in test suite size, and the fault detection ability of the reduced test suites with that of two code-based RTS approaches that represent the state-of-art for dynamic and static RTS. The results of our empirical studies showed that FLiRTS 2 achieved high safety and reduction in test suite size. The fault detection ability of the reduced test suites was comparable to that achieved by the full test suites. FLiRTS 2 is applicable to a wide range of systems of varying domains and sizes.