MUKEA-TCP: A mutant kill-based local search augmented evolutionary algorithm approach for test case prioritization

Abstract

The test case prioritization (TCP) problem is defined as determining an execution order of test cases so that important tests are executed early. Different metrics have been proposed to measure importance of test cases. While coverage and fault-detection based measures have benefits and have been used in a lot of studies, mutation kill-based measures have emerged in TCP recently, since they have benefits addressing issues with other approaches. Moreover, in the TCP problem, finding the optimal solution has a complexity of the factorial of the number of test cases, making meta-heuristic algorithms a highly suitable approach. In this study, we propose an end-to-end pipeline for TCP, Mutation Kill-based Evolutionary Algorithm (MuKEA-TCP), which allows users to have fast and efficient TCP results from existing source code, or directly from the mutant kill report of a system, without the need for any coverage information or real faults. An evolutionary algorithm utilizing Average Percentage Mutant Killed (APMK) as the objective function augmented with a local search procedure enhancing is used in MuKEA-TCP. We performed our case study on five open-source Java projects, in which we compared the APMK values of the final TCP results of some well-known greedy algorithms, and MuKEA-TCP using different initialization methods. Our results have shown that providing additional method as an initial input to the proposed augmented evolutionary algorithm has improved the results and outperformed other methods for our case study. Findings of this study have shown that using an evolutionary algorithm augmented with local search with mutation kill-based APMK as the objective function enhances the commonly used greedy prioritization methods, with a minor execution time trade-off.