In large organizations same data is stored on the different places by different users. This will occupy the storage space. In the duplicate removal process one can remove the file duplicate with the original file and make space empty for the further storage. It works by eliminating redundant data and ensuring that only one unique instance of the data is actually retained on storage. The data deduplication technique works by tracking each data file and eliminate each file that it found more than one copy of it in the storage. There are many techniques for deduplication. Our proposed algorithm depends on reducing the data before it`s stored in the storage or backup. Basically the procedure is the system analyses the data before storing it by one of mechanism for checking like hash value. If the system found the same data is stored already, ignore the data or document else store the data and save its analysis for future processing. There are many advantages by using this technique. No need for extra storage space.

data deuplication, classification, storage, hashing

[1] Jyoti Malhotra , Jagdish Bakal “A Survey and Comparative Study of Data Deduplication Techniques” International Conference on Pervasive Computing (ICPC)
[2] Junbeom Hur, Dongyoung Koo, Youngjoo Shin,And Kyungtae Kang “Secure Data Deduplication With Dynamic Ownership Management in Cloud Storage” IEEE Transactions on Knowledge and Data Engineering
[3] Jin Li, Xiaofeng Chen, Xinyi Huang, Shaohua Tang and Yang Xiang Senior Member, IEEE And Mohammad Mehedi Hassan Member,IEEE and Abdulhameed Alelaiwi Member,IEEE “Secure Distributed Deduplication Systems with Improved Reliability” IEEE Transactions on Computers
[4] Ms. Priyanka S. Savaji and Dr. K. H. Walse, “Survey on Data Deduplication system,” In Proceedings International Conference- EECCMC 2018, pp. 1-7, 2018.
[5] Shai Halevi, Danny Harnik, Benny Pinkas, Alexandra Shulman-Peleg “Proofs of Ownership in Remote Storage Systems” October 17–21, 2011, Chicago, Illinois, USA2011ACM.
[6] Roberto Di Pietro , Alessandro Sorniotti “Proof of ownership for deduplication systems: A secure, scalable, And efficient solution” 2016 Elsevier
[7] Pawar P.R, Aarti Waghmare “Data Deduplication in Cloud Storage”, National Conference on Advances in Computing 2015, pp 1-5, 2015.
[8] Nagapramod Mandagere, Pin Zhou, Mark A Smith, Sandeep Uttamchandani, “Demystifying Data Deduplication”, 2008 ACM, pp 12-17, 2008.
[9] Yujuan Tan, Hong Jiang, Dan Feng, Lei Tian, Zhichao Yan, Guohui Zhou “SAM: A Semantic-AwareMulti-Tiered Source De-duplication Framework for Cloud Backup” 2010 39th International Conference on Parallel Processing.
[10] K. H. Walse , Dr. R. V. Dharaskar, Manisha V. Kharat ,” Survey on Soft Computing Approaches for Human Activity Recognition”, International Journal of Science and Research (IJSR), Vol. 6, 2017, pp 1328-1334, 2017.
[11] Shai Halevi, Danny Harnik, Benny Pinkas, Alexandra Shulman-Peleg,” Proofs of Ownership in Remote Storage Systems”, 2011 ACM, pp 491-500, 2011.
[12] Yinjin F, Hong Jian, Nong Xiao, Lei Tian, Fang Liu “AA-Dedupe: An Application-Aware Source Deduplication Approach for Cloud Backup Services in the Personal Computing Environment” In IEEE International Conference on Cluster Computing 2011.pp 112-120, 2011.
[13] Iuon-Chang Lin and Po-Ching Chien,” Data Deduplication Scheme for Cloud Storage”, International Journal of Computer, Consumer and Control (IJ3C), Vol. 1, 2012, pp 26-31, 2012.
[14] R. V. Dharaskar, Ph.D, Pravin R Futane, V. M. Thakare, Ph.D,” Summarization of Own Contributory Efforts in the Field of Indian Sign Language Recognition System”, Proceedings published by International Journal of Computer Applications NCIPET-2013, pp 26-30, 2013.
[15] Amrita Upadhyay, Pratibha R Balihalli, Shashibhushan Ivaturi and Shrisha Rao “Deduplication and Compression Techniques in Cloud Design” 2012 IEEE,pp 1-6, 2012.