Open Access   Article Go Back

Challenges and Opportunities of Resource-Aware Allocation Frameworks for Big data tools in Cloud Computing

R. Rengasamy1 , M.Chidambaram 2

Section:Research Paper, Product Type: Journal Paper
Volume-6 , Issue-12 , Page no. 99-105, Dec-2018

CrossRef-DOI:   https://doi.org/10.26438/ijcse/v6i12.99105

Online published on Dec 31, 2018

Copyright © R. Rengasamy, M.Chidambaram . This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

View this paper at   Google Scholar | DPI Digital Library

XML View
PDF Download

How to Cite this Paper

  • IEEE Citation
  • MLA Citation
  • APA Citation
  • BibTex Citation
  • RIS Citation

IEEE Style Citation: R. Rengasamy, M.Chidambaram, “Challenges and Opportunities of Resource-Aware Allocation Frameworks for Big data tools in Cloud Computing,” International Journal of Computer Sciences and Engineering, Vol.6, Issue.12, pp.99-105, 2018.

MLA Style Citation: R. Rengasamy, M.Chidambaram "Challenges and Opportunities of Resource-Aware Allocation Frameworks for Big data tools in Cloud Computing." International Journal of Computer Sciences and Engineering 6.12 (2018): 99-105.

APA Style Citation: R. Rengasamy, M.Chidambaram, (2018). Challenges and Opportunities of Resource-Aware Allocation Frameworks for Big data tools in Cloud Computing. International Journal of Computer Sciences and Engineering, 6(12), 99-105.

BibTex Style Citation:
@article{Rengasamy_2018,
author = {R. Rengasamy, M.Chidambaram},
title = {Challenges and Opportunities of Resource-Aware Allocation Frameworks for Big data tools in Cloud Computing},
journal = {International Journal of Computer Sciences and Engineering},
issue_date = {12 2018},
volume = {6},
Issue = {12},
month = {12},
year = {2018},
issn = {2347-2693},
pages = {99-105},
url = {https://www.ijcseonline.org/full_paper_view.php?paper_id=3300},
doi = {https://doi.org/10.26438/ijcse/v6i12.99105}
publisher = {IJCSE, Indore, INDIA},
}

RIS Style Citation:
TY - JOUR
DO = {https://doi.org/10.26438/ijcse/v6i12.99105}
UR - https://www.ijcseonline.org/full_paper_view.php?paper_id=3300
TI - Challenges and Opportunities of Resource-Aware Allocation Frameworks for Big data tools in Cloud Computing
T2 - International Journal of Computer Sciences and Engineering
AU - R. Rengasamy, M.Chidambaram
PY - 2018
DA - 2018/12/31
PB - IJCSE, Indore, INDIA
SP - 99-105
IS - 12
VL - 6
SN - 2347-2693
ER -

VIEWS PDF XML
625 378 downloads 322 downloads
  
  
           

Abstract

System virtualization is the backbone of Cloud computing has been liberalizing its services to distributed data-intensive platforms such as MapReduce and Hadoop. Cloud computing empowers consumers to access online resources using the internet, from anywhere at any time without considering the underlying hardware, technical management and maintenance problems of the original resources. Cloud services are obtained from data centres which are distributed throughout the world. Big Data Applications with resource aware allocation has become an active research area in last three years. The Hadoop framework has been adopted to work efficiently in cloud computing. System virtualization is the backbone of Cloud computing, has been liberalizing its services to distributed data-intensive platforms such as MapReduce and Hadoop. Cloud computing empowers consumers to access online resources using the internet, from anywhere at any time without considering the underlying hardware, technical management and maintenance problems of the original resources.We present a detail study of various resource allocation and other scheduling challenges as well as frameworks for Hadoop Jobs in Cloud Computing.

Key-Words / Index Term

Hadoop++, Cloud computing, MapReduce, YARN, Resource-allocation

References

[1] Z. Abbasi, M. Pore, and S. K. S. Gupta. “Online server and workload management for joint optimization of electricity cost and carbon footprint across data centers”. In Proc. IEEE IPDPS, 2014.
[2] R.Udendhran, “A Hybrid Approach to Enhance Data Security in Cloud Storage ”, ICC `17 Proceedings of the Second International Conference on Internet of things and Cloud Computing at Cambridge University, United Kingdom — March 22 - 23, 2017, ACM ISBN: 978-1-4503-4774-7 doi>10.1145/3018.
[3] R Udendhran, K Muth Uramlingam, “A dynamic data-aware scheduling for map reduce in cloud ”, Advanced Computing and Communication Systems (ICACCS), 2017 4th IEEE International Conference, DOI: 10.1109/ICACCS.2017.8014617.
[4] G. Ananthanarayanan, C. Douglas, R. Ramakrishnan, S. Rao, and I. Stoica. “True elasticity in multi-tenant data-intensive compute clusters”. In Proc. of the ACM Symposium on Cloud Computing (SOCC), 2012.
[5] R. Appuswamy, C. Gkantsidis, D. Narayanan, O. Hodson, and A. Rowstron. “Scale-up vs scale-out for hadoop: Time to rethink” In Proc. ACM Symposium on Cloud Computing (SoCC), 2013.
[6] D. Borthakur, J. Gray, J. S. Sarma, K. Muthukkaruppan, N. Spiegelberg, H. Kuang, K. Ranganathan, D. Molkov, A. Menon, S. Rash, R. Schmidt, and A. Aiyer. “Apache hadoop goes realtime at facebook”. In Proc. of the ACM SIGMOD, 2011.
[7] D. Carrera, M. Steinder, I. Whalley, J. Torres, and E. Ayguad,”e. Enabling resource sharing between transactional and batch workloads using dynamic application placement”. In Proc. ACM/IFIP/USENIX Int’l Conf. on Middleware (Middleware), 2008.
[8] D. Carrera, M. Steinder, I. Whalley, J. Torres, and E. Ayguade. “Autonomic placement of mixed batch and transactional workloads”. IEEE Trans. on Parallel and Distributed Systems (TPDS), 2012.
[9] R. Chaiken, B. Jenkins, P.-A. Larson, B. Ramsey, D. Shakib, S. Weaver, and J. Zhou. “SCOPE: Easy and efficient parallel processing of massive data sets”. Proc. VLDB Endowment, 1(2):1265–1276, Aug. 2008.
[10] H. Chen, M. K. Cheng, and Y. Kuo.” Assigning real-time tasks to heterogeneous processors by applying ant colony optimization”. Journal of Parallel and Distributed Computing, 71, 2011.
[11] Y. Chen, S. Alspaugh, D. Borthakur, and R. Katz. “Energy efficiency for large-scale mapreduce workloads with significant interactive analysis”. In Proc. of the EuroSys Conference (EuroSys), 2012.
[12] S. Rao, R. Ramakrishnan, A. Silberstein, M. Ovsiannikov, and D. Reeves. Sailfish: “A framework for large scale data processing”. In Proc. of ACM Symposium on Cloud Computing (SoCC), 2012.
[13] A. Jinda, J. Quian-Ruiz, and J. Dittrich. “Trojan data layouts: Right shoes for a running elephant”. In Proc. of ACM Symposium on Cloud Computing (SoCC), 2011.
[14] Y. Guo, J. Rao, and X. Zhou. “shuffle: Improving hadoop performance with shuffle-on-write”. In Proc. Int’l Conference on Autonomic Computing (ICAC), 2013.
[15] J. Dittrich, J.-A. Quian´e-Ruiz, A. Jindal, Y. Kargin, V. Setty, and J. Schad.” Hadoop++: making a yellow elephant run like a cheetah (without it even noticing)”. In Proc. Int’l Conf. on Very Large Data Bases (VLDB), 2010.
[16] V. K. Vavilapalli, A. C. Murthy, C. Douglas, S. Agarwal, M. Konar, R. Evans, T. Graves, J. Lowe, H. Shah, S. Seth, B. Saha, C. Curino, O. O’Malley, S. Radia, B. Reed, and E. Baldeschwieler. “Apache hadoop yarn: Yet another resource negotiator”. In Proc. ACM Symposium on Cloud Computing (SoCC), 2013.
[17] H. Herodotou and S. Babu. “Profiling, what-if analysis, and cost-based optimization of mapreduce programs”. In Proc. Int’ Conf. on Very Large Data Bases (VLDB), 2011.
[18] K. Kambatla, A. Pathak, and H. Pucha. “Towards optimizing hadoop provisioning in the cloud”. In Proc. USENIX, HotCloud Workshop, 2009.
[19] P. Lama and X. Zhou.” Aroma: Automated resource allocation and configuration of mapreduce environment in the cloud”. In Proc. Int’l Conf. on Autonomic computing (ICAC), 2012.
[20] H. Herodotou, H. Lim, G. Luo, N. Borisov, L. Dong, F. B. Cetin, and S. Babu. “Starfish: A self-tuning system for big data analytics”. In Proc. Conference on Innovative Data Systems Research (CIDR), 2011.
[21] A. Verma, L. Cherkasova, and R. H. Campbell. “ARIA: automatic resource inference and allocation for mapreduce environments”. In Proc. of the ACM Int’l Conference on Autonomic Computing (ICAC), 2011.
[22] A. Verma, L. Cherkasova, and R. H. Campbell. “Resource provisioning framework for mapreduce jobs with performance goals”. In Proc. ACM/IFIP/USENIX Int’l Middleware Conference (Middleware), 2011.
[23] V. K. Vavilapalli, A. C. Murthy, C. Douglas, S. Agarwal, M. Konar, R. Evans, T. Graves, J. Lowe, H. Shah, S. Seth, B. Saha, C. Curino, O. O’Malley, S. Radia, B. Reed, and E. Baldeschwieler. “Apache hadoop yarn: Yet another resource negotiator”. In Proc. ACM Symposium on Cloud Computing (SoCC), 2013.
[24] Facebook. Hadoop corona: the next version of mapreduce. https://github.com/facebookarchive/hadoop-20/tree/master/src/contrib/corona.
[25] J. Leverich and C. Kozyrakis. “On the energy (in)efficiency of hadoop clusters”. In Proc. USENIX HotPower, 2009.