This paper is concerned with the study and analysis of Data Emplacement and Redistribution (DER) in large set of databases called Big Data and proposes a model for improving the efficiency of data processing and storage utilization for dynamic load imbalance among nodes in a heterogeneous cloud environment. With the era of explosive information and data receiving, more and more fields need to deal with massive, large scale of data. A method has been proposed with an improved Data Placement algorithm called Effective Data Emplacement and Redistribution approach (EDER) with computing capacity of each node as a predominant factor that promotes and improves the efficiency in data processing in a short duration time from large set of data. The proposed solution improves the performance of the heterogeneous cluster environment by effectively distributing data based on the performance oriented sampling as the experimental results made with word count applications.

Cloud Computing, Big Data, HDFS, MapReduce, Data emplacement, MapReduce applications

[1] Jiong Xie, Shu Yin, Xiaojun Ruan, Zhiyang Ding, Yun Tian, “ Improving MapReduce Performance through Data Placement in Heterogeneous Hadoop Clusters”, 19th International Heterogeneity in Computing Workshop, Atlanta, Georgia, April 2010.
[2] Yuanquan Fan, Weiguo Wu, Haijun Cao, Huo Zhu, Xu Zhao, Wei Wei, “A heterogeneity-aware data distribution and rebalance method in Hadoop cluster”, Seventh ChinaGrid Annual Conference, 2012.
[3] Mahesh Maurya, Sunita Mahajan “Performance analysis of MapReduce Programs on Hadoop Cluster” IEEE World Congress on Information and Communication technologies,2012.
[4] Wentao Zhao, Lingjun Meng, Jiangfeng Sun, Yang Ding, “An Improved Data Placement Strategy in a Heterogeneous Hadoop Cluster”, The Open Cybernetics & Systemics Journal, 2014.
[5] Chia-Wei Lee, Kuang-Yu Hsieh, Sun-Yuan Hsieh, Hung-Chang Hsiao , “A Dynamic Data Placement Strategy for Hadoop in Heterogeneous Environments”, Big Data Research, 2014.
[6] Dipayan Dev, Ripon Patgiri “Performance Evaluation of HDFS in Big Data Management”, International Comference on High Performance Computing and Applications (ICHPCA), 2014.
[7] Suhas V. Ambade, Priya R. Deshpande, “Heterogenity-based files placement in Big Data Cluster”, International Conference on Computational Intelligence and Communication Networks, 2015.
[8] Vrushali Ubarhande, “Novel Data-Distribution Technique for Hadoop in Heterogeneous Cloud Environments”, IEEE Transactions 2015.
[9] Ch. Bhaskar VishnuVardhan and Pallav Kumar Baruah, “Improving the Performance of Heterogeneous Hadoop Cluster”, Fourth International Conference on Parallel, Distributed and Grid Computing (PDGC), 2016.
[10] Anton Spivak and Denis Nasonov “Data Preloading and Data Placement for MapReduce Performance Inproving” Procedia Computer Science 101, 2016.
[11] Ramchandani Hema Megharajbhai, Viral Parmar, “Heterogeneity based Fairly Data Distribution in Cluster Environment”, International Journal of Advance Engineering and Research Development, 2018.
[12] S. Annapoorani, Dr. B. Srinivasan, “Initial Dynamic Data Allocation for Heterogeneous hadoop clusters” International Journal of Scientific Research in Computer Science Applications and Management Studies, Volume 7, Issue 3, 2018.
[13] S. Annapoorani, Dr. B. Srinivasan, “Improving performance of data in Hadoop clusters using dynamic data replication” International Journal of Engineering sciences & Research Technology, Feb, 2018.