The web is vast, diverse and dynamic and increases scalability, temporal data and multimedia issues respectively. The expansion of the Internet has given rise to a wealth of data as big data that is now available for user access. Different types of data must be managed and organized so that can be accessed by different users effectively and efficiently. The log analysis is an important issue for the web application. Log file is not to be over emphasized as a source of information in systems and network management. Whereas conduct efficient investigation and gathering of use full information need to correlate different log file. Task of analyzing event log files with the ever-increasing size and complexity of today’s event logs has become cumbrous to carry out manually. Nowadays latest spotlighted is automatic analysis of these logs files. This paper is a review of the basics of log analysis as big data in the web environment.

Web Application, Log File, Data Mining, Big Data

[1]. Xiuqin Lin; Peng Wang; Bin Wu, "Log analysis in cloud computing environment with Hadoop and Spark," in Broadband Network & Multimedia Technology (IC-BNMT), 2013 5th IEEE International Conference on , vol., no., pp.273-276, 17-19 Nov. 2013
[2]. Chaofei Wang; Jing Chen; Xiaopeng Liu; Jinwei Zhao, "An improved deep log analysis method based on data reconstruction," in Cloud Computing and Intelligence Systems (CCIS), 2014 IEEE 3rd International Conference on , vol., no., pp.86-90, 27-29 Nov. 2014
[3]. da Silva Machado, Roger; Borges Almeida, Ricardo; Correa Yamin, Adenauer; Marilza Pernas, Ana, "LogA-DM: An Approach of Dynamic Log Analysis," in Latin America Transactions, IEEE (Revista IEEE America Latina) , vol.13, no.9, pp.3096-3102, Sept. 2015
[4]. Xiaokui Shu; Smiy, J.; Danfeng Yao; Heshan Lin, "Massive distributed and parallel log analysis for organizational security," in Globecom Workshops (GC Wkshps), 2013 IEEE , vol., no., pp.194-199, 9-13 Dec. 2013
[5]. Hingave, H.; Ingle, R., "An approach for MapReduce based log analysis using Hadoop," in Electronics and Communication Systems (ICECS), 2015 2nd International Conference on , vol., no., pp.1264-1268, 26-27 Feb. 2015
[6]. K Savitha and MS Vijaya , "Mining of Web Server Logs in a Distributed Cluster using Big Data Technologies" , International Journal of Advanced Computer Science and Applications (IJACSA), vol. 5 , 2014
[7]. T. K. Das , "BIG Data Analytics: A Framework for Unstructured Data Analysis" , International Journal of Engineering and Technology (IJET) , vol. 5 , no. 1 , 2013
[8]. Jiang Dawei, K.H. Antony and Gang Chen , "MAP-JOINREDUCE: Toward Scalable and Efficient Data Analysis on Large Clusters" , IEEE Transactions on Knowledge and Data Engineering, pp.1299 -1311 , 2011
[9]. Pavlo Andrew, Paulson Erik, Rasin Alexander, J. Daniel, J. David, De Witt, Samuel Madden and Michael Stonebraker , "A Comparison of Approaches to Large-Scale Data Analysis" , ACM SIGMOD International Conference on Management of data , pp.165 -178 , 2009
[10]. W. Xu, L. Huang, A. Fox, D. Patterson, M. Jordan. "Online System Problem Detection by Mining Patterns of Console Logs". In the Proceeding of ICDM '09 Proceedings of the 2009 Ninth IEEE International Conference on Data Mining.
[11]. AiLing Duan et. al. “Research and Practice of Distributed Parallel Search Algorithm on Hadoop_MapReduce”, 2012 International Conference on Control Engineering and Communication Technology, 2012 IEEE DOI 10.1109/ICCECT.2012.131, pp 105108
[12]. D. Agrawal, A. El Abbadi, S. Antony, and S. Das. Data Management Challenges in Cloud Computing Infrastructures. In DNIS, pages 1–10, 2010.
[13]. White paper on “Solution Brief Big Data in the Cloud: Converging Technologies , How to Create Competitive Advantage Using Cloud- Based Big Data Analytics.
[14]. Tharam Dillon et. al. “Cloud Computing: Issues and Challenges”, 2010 24th IEEE International Conference on Advanced Information Networking and Applications,
[15]. D. Agrawal, S. Das, and A. E. Abbadi. Big data and cloud computing: New wine or just new bottles? PVLDB, 3(2):1647– 1648.
[16]. Nacim Fateh Chikhi, Bernard Rothenburger, Nathalie Aussenac-Gilles “A Comparison of Dimensionality Reduction Techniques for Web Structure Mining”, Proceedings of the IEEE/WIC/ACM International Conference on Web Intelligence 2007, pp 116-119.
[17]. Lefteris Moussiades, Athena Vakali, "Mining the Community Structure of a Web Site," bci Fourth Balkan Conference in Informatics 2009, pp.239-244.
[18]. Toufiq Hossain Kazi, Wenying Feng and Gongzhu Hu, “Web Object Prefetching: Approaches and a New Algorithm”, IEEE 2010, pp 115-120.
[19]. Brijendra Singh and Hemant Kumar Singh, “Web Data Mining Research: A Survey”, IEEE 2010.
[20]. Kavita Sharma, Gulshan Shrivastava and Vikas Kumar, “Web Mining: Today and Tomorrow”, IEEE 2011, pp 399-403.
[21]. WANG Yong-gui and JIA Zhen, “Research on Semantic Web Mining” IEEE 2010, pp 67-70.
[22]. P. Sampath, C. Ramesh, T. Kalaiyarasi, S. Sumaiya Banu and G. Arul Selvan, “An Efficient Weighted Rule Mining for Web Logs Using Systolic Tree”, IEEE 2012, pp 432-436.
[23]. Nizar R. Mabroukeh and C. I. Ezeife, “Semantic-rich Markov Models for Web Prefetching”, IEEE 2009, pp 465-470.
[24]. A.B.M.Rezbaul Islam and Tae-Sun Chung, “An Improved Frequent Pattern Tree Based Association Rule Mining Technique”, IEEE 2011.
[25]. R.Agrawal, and R.Srikant, “Fast algorithms for mining association rules”, In VLDB’94, pp. 487-499, 1994 Borges and M. Levene,”A dynamic clustering-based markov model for web usage Mining”, cs.IR/0406032, 2004.
[26]. Zhu, J., Hong, J. and Hughes, J. G. (2002a) Using Markov Chains for Link Prediction in Adaptive Web Sites. In Proc. of Soft-Ware 2002: the First International Conference on Computing in an Imperfect World, pp. 60-73, Lecture Notes in Computer Science, Springer, Belfast, April.
[27]. K.Ramu Dr.R.Sugumar and B.Shanmugasundaram “A Study on Web Prefetching Techniques” Journal of Advances in Computational Research: An International Journal Vol. 1 No. 1-2 (January-December, 2012)