Not all bugs lead to program crashes, and not always is there a formal specification to check the correctness of a software test�s outcome. A common scenario in software testing is therefore that test data are generated, and a tester manually adds test oracles. As this is a difficult task, it is important to produce small yet representative test sets, and this representativeness is typically measured using code coverage. There is, however, a fundamental problem with the common approach of targeting one coverage goal at a time. Coverage goals are not independent, not equally difficult, and sometimes infeasible the result of test generation is therefore dependent on the order of coverage goals and how many of them are feasible. To overcome this problem, a novel paradigm is proposed in which whole test suites are evolved with the aim of covering all coverage goals at the same time while keeping the total size as small as possible. Genetic Algorithms have been successfully applied to the generation of unit tests for classes, and are well suited to create complex objects through sequences of method calls. However, because the neighborhood in the search space for method sequences is huge, even supposedly simple optimizations on primitive variables (e.g., numbers and strings) can be ineffective or unsuccessful. To overcome this problem, we extend the global search applied in the EvoSuite test generation tool with local search on the individual statements of method sequences. In contrast to previous work on local search, we also consider complex data types including strings and arrays.

EvoSuite, Search-based Software Engineering, Object-oriented, Evolutionary Testing, length, branch coverage, infeasible goal, Collateral coverage

[1] G.Fraser, A.Arcuri �whole test suite generation� IEEE Transactions on software engineering VOL 39 NO.2, FEBRUARY 2013
[2] S. Ali, L. Briand, H. Hemmati, and R. Panesar-Walawege, �A Systematic Review of the Application and Empirical Investigation of Search-Based Test-Case Generation,�IEEE Trans. Software Eng., vol. 36, no. 6, pp. 742-762, Nov./Dec. 2010.
[3] M. Alshraideh and L. Bottaci, �Search-Based Software Test Data Generation for String Data Using Program-Specific Search Operators: Research Articles,�Software Testing, Verification, and Reliability,vol. 16, no. 3, pp. 175-203, 2006.
[4] L. Araujo and J. Merelo, �Diversity through Multiculturality: Assessing Migrant Choice Policies in an Island Model,�IEEE Trans. Evolutionary Computation,vol. 15, no. 4, pp. 1-14, Aug. 2011.
[5] A. Arcuri, �It Really Does Matter How You Normalize the Branch Distance in Search-Based Software Testing,�Software Testing, Verification and Reliability, http://dx.doi.org/10.1002/stvr.457,2011.
[6] A. Arcuri, �A Theoretical and Empirical Analysis of the Role of Test Sequence Length in Software Testing for Structural Coverage,�IEEE Trans. Software Eng., vol. 38, no. 3, pp. 497-519, May/June 2011
[7] L. Baresi, P.L. Lanzi, and M. Miraz, �Testful: An Evolutionary Test Approach for Java,� Proc. IEEE Int�l Conf. Software Testing, Verification and Validation, pp. 185-194, 2010.
[8] B. Baudry, F. Fleurey, J.-M. Je�ze�quel, and Y. Le Traon, �Automatic Test Cases Optimization: A Bacteriologic Algorithm,� IEEE Software, vol. 22, no. 2, pp. 76-82, Mar./Apr. 2005.
[9] G. Fraser and A. Arcuri, �Evolutionary Generation of Whole Test Suites,� Proc. 11th Int�l Conf. Quality Software, pp. 31-40, 2011.
[10] G. Fraser and A. Arcuri, �Evosuite: Automatic Test Suite Generation for Object-Oriented Software,� Proc. 19th ACM SIGSOFT Symp. and the 13th European Conf. Foundations of Software Eng., 2011.