An Evaluation of Internal Program Metrics as Predictors of Mutation Operator Score

Context: Mutation testing is effective in producing high quality test sets. On the downside, it is expensive due to factors like the large number of mutants and the need for manual analysis tasks. Over time, researchers devised several ways of reducing its costs and achieved noticeable results. However, results are little generalizable due to characteristics of the programs and of the applied mutation operators. Objective: This paper reports on an exploratory study that aimed to evaluate internal program characteristics as predictors of mutation testing scores. Method: By applying a clustering algorithm, the core idea consists in identifying a group R of tested programs that can be used as a baseline for testing a new program u using mutation at reduced cost. The same cost reduction results obtained for R is expected to produce relevant results for u. Results: We experimented our approach with 38 programs used in previous experiments. The programs have mutation-adequate test sets. For each program, we analyzed if the results achieved for the cluster that contained the program were superior to results achieved for the remaining clusters, as well as to results achieved for the whole set of programs. Results indicate that the best mutation operators for the cluster to which the program is associated, in the majority of the cases, are also the best mutation operators for the program itself. Some variations in results occurred with reduced and increased sets of programs. Conclusion: Overall, results are promising and motivate further experiments with varying settings. We hypothesize that program similarity can be useful predictors of consistent results of mutation testing applied at low cost.