Mutation Testing is a fault based software testing technique, was proposed in the 1970’s, it has been considered as an effective technique of software testing process for evaluating the quality of the test data. In other words, Mutation Testing is used to evaluate the fault detection capability of the test data by inserting errors into the original program to generate mutations, and after then check whether tests are good enough to detect them. A lot of solutions have been proposed to solve that problem. A new form of Mutation Testing is Higher Order Mutation Testing, was first proposed by Harman and Jia in 2009 and is one of the most promising solutions. In this paper, we consider the main limitations of Mutation Testing and previous proposed solutions to solve that problems. This paper also refers to the development of Higher Order Mutation Testing and reviews the methods for finding the good Higher Order Mutant.

FOM, HOM, SHOM, GI.

[1]. M. Harman and Y. Jia (2009). Higher Order Mutation Testing. King’s college London, CREST centre.
[2]. M. Harman et al. (2010). A Manifesto for Higher Order Mutation Testing. King’s College London, CREST centre, Strand, London, WC2R 2LS, UK.
[3]. S. Kapoor (2011). Test Case Effectiveness of Higher Order Mutation Testing. International Journal of Computer Technology Application. Volume 2 (5), 1206-1211.
[4]. Aderonke Olusola Akinde (2012). Using Higher Order Mutation For Reducing

Equivalent Mutants In Mutation Testing, Asian Journal Of Computer Science And Information Technology 2: 3 (2012) 13 –18. www.innovativejournal.in
[5]. Lisherness, P., Lesperance, N., Cheng, K.T (2013). Mutation Analysis with Coverage Discounting. Design, Automation and Test in Europe Conference and Exhibition.
[6]. Nguyen, Q. V., and Madeyski, L (2014). Problems of mutation testing and higher order mutation testing. In Advanced Computational Methods for Knowledge Engineering, T. Do, H. A. L. Thi, and N. T. Nguyen, Eds., vol. 282 of Advances in Intelligent Systems and Computing. Springer International Publishing, 2014, pp. 157–172.
[7]. Ahmed S. Ghiduk, Moheb R. Girgis, Marwa H. Shehata (2017). Higher order mutation testing: A Systematic Literature Review, Received 1 July 2016 Received in revised form 8 June 2017, Accepted 15 June 2017 Available online 4 August 2017, www.elsevier.com/locate/cosrev, http://dx.doi.org/10.1016/j.cosrev.2017.06.001
[8]. E. Omar, S. Ghosh, D. Whitley (2017). Subtle higher order mutants, Inf. Softw. Technol. 81 (2017) 3–18.
[9]. Y. Jia, F. Wu, M. Harman, J. Krinke (2015) Genetic Improvement using Higher Order Mutation, in: GECCO Companion’15: Proceedings of the Companion Publication of the 2015 Annual Conference on Genetic and Evolutionary Computation, pp. 803–804.
[10]. Q.V. Nguyen, L. Madeyski, (2016). Empirical evaluation of multi-objective optimization algorithms searching for higher order mutants, in: Cybernetics an Systems — Smart Experience and Knowledge Engineering for Optimization earning, and Classification/Recommendation Problems, vol. 47, 2016, pp. 48–68.
[11]. Q. Vu Nguyen, L. Madeyski, (2016). On the relationship between the order of mutation testing and the properties of generated higher order mutants, in: Ngoc Thanh Nguyen, Bogdan Trawiński, Hamido Fujita, Tzung-Pei Hong (Eds.), Intelligent Information and Database Systems, ACIIDS 2016, in: Lecture Notes in Artificial Intelligence, vol. 9621, Springer-Verlag, Berlin Heidelberg, 2016.
[12]. A.S. Ghiduk, (2016). Reducing the number of higher-order mutants with the aid of data flow, e-Inform. Softw. Eng. J. 10 (2016) 31–49.
[13]. M. Kintis, M. Papadakis, N. Malevris (2010), Evaluating mutation testing alternatives: A collateral experiment, in: Proc. 17th Asia Pacific Soft. Eng. Conf., APSEC.
[14]. M. Papadakis, N. Malevris,(2010). An empirical evaluation of the first and second order mutation testing strategies, in: Proceedings of the 2010 Third
[15]. M. Polo, M. Piattini, I. Garcia-Rodriguez (2008). Decreasing the cost of mutation testing with second-order mutants, Softw. Test. Verif. Reliab. 19 (2) (2008) 111–131.
[16]. M. Kintis, M. Papadakis, N. Malevris (2012). Isolating First Order Equivalent Mutants via Second Order Mutation, in: IEEE Fifth International Conference on Software Testing, Verification and Validation, 2012, pp. 701–710.
[17]. L. Madeyski, W. Orzeszyna, R. Torkar, M. Józala, (2014). Overcoming the equivalent mutant problem: A systematic literature review and a comparative experiment of second order mutation, IEEE Trans. Softw. Eng.. 40 (1) (2014) 23–44.
[18]. Y. Jia, M. Harman (2009). Higher order mutation testing, J. Inf. Softw. Technol. 51 (10) (2009) 1379–1393.
[19]. M. Harman, Y. Jia, W.B. Langdon, (2011). Strong higher order mutation-based test data generation, in: Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering, ESEC/FSE’11, 2011, pp. 212–222.
[20]. A.O. Akinde, (2012). Using higher order mutation for reducing equivalent mutants in mutation testing, Asian J. Comput. Sci. Inf. Technol. 2 (3) (2012) 13–18.
[21]. A. Derezińska, K. Hałas, (2014). Experimental evaluation of mutation testing approaches to python programs, in: Proc. of 7th IEEE Inter. Conf. on Software Testing Verification and Validation Workshops, ICSTW, IEEE Comp. Soc, 2014, pp. 156–164.
[22]. Anna Lumelsky (2018). Genetic Testing and Government Regulation: The Growing Significance of Pharmacogenomics Accessed August 19, 2018 11:52:12 AM EDT http://nrs.harvard.edu/urn-3:HUL.InstRepos:8852131
[23]. B Bapat, H Noorani, Z Cohen, T Berk, A Mitri, B Gallie, K Pritzker, S Gallinger, A S Detsky (2018). Cost comparison of predictive genetic testing versus conventional clinical screening for familial adenomatous polyposis, Download on http://gut.bmj.com/ on 19 August 2018 by guest
[24]. E.J Weyuker and T.J Ostrand, (1980). Theories of Program Testing and the Application of Revealing Subdomains, IEEE Transaction Software Engineering., vol. SE.
[25]. J. Good enough and S. L. Gerhart, (1977), Towards a theory of Test Data Selection,”IEEE Transaction Software Engineering., vol. SE-3.
[26]. R.G Hamlet, (1977). Testing programs with the AID of a Compiler, IEEE Transactions on Software engineering.
[27]. R. DeMillo, R. Lipton and F Sayward,(1978), Hints on Test Data Selection: Help for the Practicing Programmer, Computer, 11(4): 34-41: April, 1978.
[28]. Antonia Estero-Botaro Palomo-Lozano and Inmaculada Medina Bulo, (2015). Quantitative Evaluation of Mutation Operators for WS-BPEL Compositions, Department of Computer Languages and Systems, University of C? adiz, Spain.
[29]. M.Woodward (1993). Errors in Algebaric Specification and an Experimental Mutaion Testing Tool” Software Engineering Journal, pages 211-224, July 1993.
[30]. Y. Jia and M. Harman, (2009). An Analysis and Survey of the Development of Mutation Testing”, CREST Center, King’s College, London, Tech. Rep. TR-09-06, 2009.
[31]. A.J. Offut. (1992). Investigations of the Software Testing Coupling Effect, ACM Transactions on Software engineering Methodology 1(1):3-18 January 1992.
[32]. A . Derzinska, (2006). Quality Assessment of Mutation Operators Dedicated for C# Programs” in QSIC 2006: sixth International Conference on Quality Software, Beijing, China: IEEE, Computer society , 2006, pp 227-234.
[33]. Howden W. E. (1982), Weak Mutation Testing and Completenes of Test Sets, IEEE transaction on Software Engineering, 8(4): page 371-379.
[34]. W. E Howden, (1987). Functional Programming Testing and Analysis, McGraw-hill Book company New York NY 1987.