In recent years, by increasing the volume of information available in data warehouse most of the system may be affected by the replicas. Record deduplication is the important key operation in data integration from multiple data sources on server. To achieve high quality information, remove replica data and more simplified data representation, data preprocessing is required. Data clean-up is one among the data preprocessing steps. Data clean-up includes the process of parsing, Data tree analysis, data transformation, duplicate elimination and arithmetic methods. If two data sets represent the same real world entity then it is called duplicated data’s. The problem of detecting and eliminating duplicate data’s is called record deduplication. This survey presents an analysis of record BAT algorithm, Modified BAT algorithm and Hidden Face algorithms that identify and remove the duplicate records. Duplicate data removal to possible savings in computational time and resources to process this data.

Data deduplication, Data Integration, Data preprocessing, BAT algorithm, Modified BAT algorithm, Hidden Face algorithm

[1] Moises G. de Carvalho, Alberto H.F. Laender, Marcos AndreGoncalves, and Altigran S. da silva, “A Genetic Programming Approach to Record Deduplication”, IEEE Trans. Knowledge and Data Eng., vol. 24,no. 3, pp. 399-412, Mar. 2012.
[2] A.K. Elmagarmid, P.G. Ipeirotis, and V.S. Verykios, “Duplicate Record Detection: A Survey”, IEEE Trans. Knowledge and Data Eng., vol. 19, no. 1, pp. 1-16, Jan. 2007.
[3] V. Subramaniyaswamy, S. Chenthur Pandian, “A Complete Survey of Duplicate Record Detection Using Data Mining Techniques”, Information Technology Journal 11(8)., ISSN 1812-5638, pp.941- 945, 2012.
[4]. M. Carvalho, A. Laender, M. Goncalves, and A. da Silva. Replica identification using genetic programming. In Proceedings of the 2008 ACM symposium on Applied computing, pages 1801-1806. ACM, 2008.
[5] Faritha Banu, A, Chandrasekar C, “An Optimized Approach of Modified BAT Algorithm to Record Deduplication”, International Journal of Computer Applications (0975 – 8887) Volume 62– No.1, January 2013.
[6] Baoping Zhang, Yuxin Chen, Weiguo Fan, Edward A. Fox , Marcos Gonc¸alves, Marco Cristo, P´avel Calado, “Intelligent GP Fusion from Multiple Sources for Text Classification”
[7] An´ısio Lacerda1 Marco Cristo1 Marcos Andr´e Gonc¸ alves, “Learning to Advertise”, “SIGIR’06, August 6– 11, 2006, Seattle, Washington, USA. Copyright 2006 ACM 1595933697/06/0008.
[8] N. Koudas, S. Sarawagi, and D. Srivastava, “Record linkage: similarity measures and algorithms,” ACM SIGMOD International Conference on Management of Data, pp. 802–803, 2006