Big data is referred as a term that describes volume of data (terabytes to Exabyte’s), unstructured (include text and multimedia content), and complex in processing (from Medical data, Business transactions, Data capture by sensors, Social media/networks, Banking, Marketing, Government data, etc.). The traditional technologies are not sufficient to store, process and analyze the data. The unique technologies should be needed to analyze, manage the huge amount and unprocessed data. There are number of sources producing huge volume and variety of data. The number of sources produce amount of various descriptions for same object. This leads to data conflict and source conflict, when various sources generate various descriptions for same objects. Here it is the challenging one to identify which source produces quality information. The source could be identified by discovering the weight of the sources by using optimization method. Here optimization playing an important role to find highest achievable performance under the given constraints, by maximizing desired factors and minimizing undesired ones. In comparison, maximization means trying to attain the highest or maximum result or outcome without regard to cost or expense.

Big data, optimization, Reliability, Accuracy, Consistency and Integrity

[1]. Amir Gandomi and Murtaza Haider “Beyond the hype: Big data Concepts, Methods and analytics”, International Journal of Information Management (IJIM) ELSEVIER, 2015, pp: 137-144.
[2]. Provost, F., & Fawcett, T. (2013).Data science and its relationship to big data and data driven decision making.Big Data, 1(1), 51–59.
[3]. Jerry Gao, Chunli Xie, Chuanqi Tao, “Big Data Validation and Quality Assurance –Issuses, Challenges, and Needs”, 2016 IEEE Symposium on Service-Oriented System Engineering, 978-1-5090-2253-3/16 $31.00 © 2016 IEEE , DOI 10.1109/SOSE.2016.63, 433-41.
[4]. Kushal Patel ,“Big Data, its Issues and Challenges”, 2017 IJEDR, Volume 5, Issue 3, ISSN: 2321-9939,123-27
[5].http://www.iso.org/iso/catalogue_detail?csnumber=45481)
[6]. Q. Li, Y. Li, J. Gao, B. Zhao, W. Fan, and J. Han, “Resolving conflicts in heterogeneous data by truth discovery and source reliability estimation,” in Proceedings of the 2014 ACM SIGMOD international conference on Management of data. ACM, 2014, pp. 1187–1198.
[7]. Cai, L and Zhu, Y 2015 The Challenges of Data Quality and Data Quality Assessment in the Big Data Era. Data Science Journal, 14: 2, pp. 1-10, DOI: http://dx.doi.org/10.5334/dsj-2015-002.
[8]. S. Boyd and L. Vandenberghe. Convex optimization. Cambridge University Press, 2004
[9]. Fan Zhang, Li Yu, Xiangrui Cai, Ying Zhang, Haiwei Zhang,“Truth Finding from Multiple Data Sources by Source Confidence Estimation”, 978-1-4673-9372-0/15 $31.00 © 2015 IEEE DOI 10.1109/WISA.2015.45, 153-56.
[10]. A. Banerjee, S. Merugu, I. S. Dhillon, and J. Ghosh. Clustering with bregman divergences. JMLR, 6:1705–1749, 2005.
[11]. Arunima Kumari, Dr. Dinesh Singh, Reviewing Truth Discovery Approaches And Methods For Big Data Integration. International Journal of Science, Engineering and Technology Research (IJSETR), Volume 5, Issue 8, Pages: 2766-2774, August 2016.
[12]. J. Bleiholder and F. Naumann. Conflict handling strategies in an integrated information system. In Proc. of IIWeb, 2006.
[13]. Steven Ji-fan Ren, Samuel Fosso Wamba, Shahriar Akter, Rameshwar Dubey
& Stephen J. Childe (2016): Modelling quality dynamics, business value and firm performance
in a big data analytics environment, International Journal of Production Research, DOI:10.1080 /00207543.2016.1154209