Hybrid Algorithm Based Whole Test Suite Generation

Abstract

Not all bugs lead to program crashes, and not always is there a formal specification to check t\he correctness of a software test’s outcome. A common scenario in software testing is therefore that test data are generated, and a tester manually adds test oracles. As this is a difficult task, it is important to produce small yet representative test sets, and this representativeness is typically measured using code coverage. There is, however, a fundamental problem with the common approach of targeting one coverage goal at a time. Coverage goals are not independent, not equally difficult, and sometimes infeasible the result of test generation is therefore dependent on the order of coverage goals and how many of them are feasible. To overcome this problem, a novel paradigm is proposed in which whole test suites are evolved with the aim of covering all coverage goals at the same time while keeping the total size as small as possible. Genetic Algorithms have been successfully applied to the generation of unit tests for classes, and are well suited to create complex objects through sequences of method calls. However, because the neighborhood in the search space for method sequences is huge, even supposedly simple optimizations on primitive variables (e.g., numbers and strings) can be ineffective or unsuccessful. To overcome this problem, we extend the global search applied in the EvoSuite test generation tool with local search on the individual statements of method sequences. In contrast to previous work on local search, we also consider complex data types including strings and arrays.