Multilingual Information Retrieval System (MLIR) allows users to provide queries in one language and extract the relevant content in multiple languages. Appraising the quality of these systems is a promising task. A wide variety of metrics are available for estimating the performance of IR systems, Precision and Recall are considered as the basic measures among them. However, less number of metrics is available in the literature to analyze the performance of MLIR Systems. This paper demonstrates the significance of MLIR systems when the retrieved documents are in various languages and the weights assigned by the user based on his preference languages. This is achieved by comparing the performances of IR and MLIR using the proposed weight based Precision oriented metrics. In addition, four essential parameters of the retrieval systems are considered to compare the significance of the proposed metrics with traditional metrics. The analyses of these metrics demonstrate positive and promising results. Statistical Analyses are also performed to show the importance of the proposed metrics. Thus we can conclude that weight based precision oriented metrics plays a vital role in MLIR domain area.

Average Precision, Information Retrieval, MLIR, Precision, Normalized Precision, P@k

[1] C. J. van Rijsbergen, et al., “Information Retrieval,” Butterworths, London, 2nd edition, 1979.
[2] Chetana Sidige, et al., “Normalized Distance Measure: A Measure for Evaluating MLIR Merging Mechanisms,” IJCSI International Journal of Computer Science Issues, Vol. 8, no. 1, pp. 209-214, 2011.
[3] Christian Fluhr, E. Robert, et al., “Chapter 2 Multilingual (or Cross-lingual) Information Retrieval,” http://www.cs.cmu.edu/~ref/mlim/chapter2.html
[4] C.W. Cleverdon, et al., “Aslib Cranfield research project - Factors determining the performance of indexing systems,” Vol. 2, 1966.
[5] Dan Wu, et al., “Cross-Language Query Expansion Using Pseudo Relevance Feedback,” Journal of the Chinese Society for Scientific and Technical Information, Vol. 29, no.2, pp. 232-239, 2013.
[6] Davis, et al., “A TREC evaluation of query translation methods for multi-lingual text retrieval,” Proceedings of the 5th Text Retrieval Conference (TREC), 1996.
[7] Ellen M. Voorhees, et al., “Overview of fifth Text Retrieval Conference,” Proceedings of the 5th Text Retrieval Conference (TREC), 1995.
[8] EmineYilmaz, et al., “Estimating average precision when judgments are incomplete,” Knowledge and Information Systems, Vol. 16, pp. 173-211, 2008.
[9] Hull D, et al. “Querying across Languages a dictionary-based approach to Multilingual Information Retrieval,” Proceedings of the 19th Annual ACM Conference on Information Retrieval (SIGIR), pp. 49-57, 1996.
[10] M. Rogati, et al., “Multilingual Information Retrieval Using Open, Transparent Resources in CLEF 2003,” Comparative Evaluation of Multilingual Information Access Systems, Vol. 3237, pp. 133-139, 2004.
[11] Mangala Madankar, et al., “Information Retrieval System and Machine Translation: A Review,” International Conference on Information Security and Privacy (ICISP2015), Vol. 78, pp. 845-850, 2016.
[12] Olivier Chapelle, et al., “Gradient descent optimization of smoothed information retrieval metrics,” Information Retrieval Journal, Vol.13, pp. 261-235.
[13] Parmatma Yadav, et al., “Weight based precision oriented metrics for MLIR systems,” IEEE International Conference on Advanced Communication Control and Computing Technologies (ICACCCT), pp. 1114-1119, 2014.
[14] Pothula Sujatha, et al., “Performance Evaluation of Multilingual Information Retrieval (MLIR) System over Information Retrieval (IR) System,” IEEE-International Conference on Recent Trends in Information Technology, ICRTIT 2011, pp. 722-727, 2011.
[15] Pothula Sujatha, et al., “Precision at k in multilingual information retrieval,” International Journal of Computer Applications, Vol. 24, no. 9, pp. 40-43, 2011.
[16] R. R. Korfhage, “Information Storage and Retrieval,” John Wiley & Sons, 1997.
[17] René Hackl, et al., “Multilingual Retrieval Experiments with MIMOR at the University of Hildesheim,” Comparative Evaluation of Multilingual Information Access Systems, Vol. 3237, pp.166–173, 2004.
[18] Sanjay Misra, et al., “Weyuker’s Properties, Language Independency and Object Oriented Metrics,” Computational Science and Its Applications – ICCSA 2009, Vol. 5593, pp 70-81, 2009.
[19] Tetsuya Sakai, et al., “Ranking the NTCIR systems based on multigrade relevance,” Proceedings of Asia information retrieval symposium, Vol. 3411, pp. 251-262, 2004.
[20] Tetsuya Sakai, et al., “Give me just one highly relevant document: P-measure,” Proceedings of the 29th annual international ACM SIGIR conference on research and development in information retrieval (SIGIR 2006), pp. 695-696, 2006.
[21] Tetsuya Sakai, et al., “On information retrieval metrics designed for evaluation with incomplete relevance assessments,”Information Retrieval, Vol. 11, no.5, pp. 447–470, 2008.
[22] Toyin Enikuomehin and J.S. Sadiku, “Text Wrapping Approach to natural Language Information retrieval using significant Indicator”, IAES International Journal of Artificial Intelligence (IJ-AI,) Vol. 2, No. 3, pp. 136-142, 2013.
[23] V. Ravisankar, et al., “A statistical Methodology for Evaluating MLIR Systems: Web Service based Analysis,” International Journal of Applied Engineering Research, Vol. 10, no. 2, pp. 3337-3348, 2015.
[24] Vu, Gallinari, “On effectiveness measures and relevance functions in ranking INEX systems,” Proceedings of Second Asia Information Retrieval Symposium, AIRS 2005, Vol. 3689, pp. 312–327, 2005.
[25] W. Mendenhall, et al., “Mathematical Statistics with applications,” PWS-KENT Publishing Company, 1990.
[26] Yuemin Wang, “Design of Information Retrieval System Using Rough Fuzzy Set,” TELKOMNIKA Indonesian Journal of Electrical Engineering, Vol. 12, no. 1, pp. 844-851, 2014.