In this paper, first ever attempt to automatic detection and extraction of Gujarati text from image has been presented. This method is based on Haar discrete wavelet transform (HDWT), edge detection and connected component analysis. The distinct features of Gujarati script make it hard to directly apply the existing text detection methods designed for English, Chinese and other languages. In proposed work, first high-frequency wavelet coefficients are extracted using HDWT and then sobel filter is applied on to detect candidate text edges. Then connected component analysis is performed using area geometric feature to get rid of non-text area. The proposed method is tested on a variety of images such as images of complex background and images of different fonts, colour, and size of text. The experiment on over 878 images show that the proposed method can detect regions and isolation of text perfectly including modifier. The proposed framework has achieved a precision of 0.91, recall of 0.97 and f measure of 0.94.

Gujarati text detection, Text extraction, Text binarization, HDWT, Connected Componenet

[1] J. M. Patel, A. A. Desai, “Segmentation and Recognition of printed Gujarati numeral from Image”, International Journal of Engineering Research & Technology, Vol. 3, Issue. 2, pp. 1012-1020, February 2014
[2] SH Kim, KJ An, SW Jang, GY Kim, "Texture feature-based text region segmentation in social multimedia data", Multimedia Tools and Applications 75, No. 20, pp. 12815-12829, 2016
[3] Y. Zhong, H. J. Zhang, and A. K. Jain, “Automatic caption localization in compressed video”, IEEE transactions on pattern analysis and machine intelligence, Vol. 22, No. 4, pp. 385–392, Apr. 2000.
[4] L. Neumann , J. Matas, “Scene text localization and recognition with oriented stroke detection”, In Proceedings of the IEEE International Conference on Computer Vision, pp. 97–104, 2013
[5] I. Haritaoglu, “Scene text extraction and translation for handheld devices”, in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2, pp. 408–413, 2001
[6] J. Liang, D. Doermann, H. Li, “Camera-based analysis of text and documents: A survey”, International Journal of Document Analysis and Recognition, Vol. 7, Issue. 2-3, pp. 84–104, 2005.
[7] L. Lin , C. L. Tan, “Text extraction from name cards using neural network”, Proceedings. IEEE International Joint Conference on, Vol. 3, pp. 1818–1823, 2005.
[8] P Sermanet, S Chintala, Y LeCun, "Convolutional neural networks applied to house numbers digit classification", In the Proceedings of international conference on pattern recognition ICPR, pp 10–13, 2012
[9] Lee, Chung-Mong, A Kankanhalli. "Automatic extraction of characters in complex scene images", International Journal of Pattern Recognition and Artificial Intelligence, Vol. 9, No. 01, pp. 67-82, 1995
[10] X Liu, J Samarabandu, "An edge-based text region extraction algorithm for indoor mobile robot navigation", In Mechatronics and Automation, IEEE International Conference, Vol. 2, No. 7, pp. 2008–2015, 2005
[11] G. S. Hsu, J. C. Chen , Y. Z. Chung, "Application-Oriented License Plate Recognition", in IEEE Transactions on Vehicular Technology, Vol. 62, No. 2, pp. 552-561, Feb. 2013
[12] P. Prabhakar , P. Anupama, "A novel design for vehicle license plate detection and recognition", Second International Conference on Current Trends In Engineering and Technology - ICCTET, pp. 7-12, 2014
[13] Liu, Xiaoqian, Ke Lu, ,W. Wang, "Effectively localize text in natural scene images", In Pattern Recognition (ICPR), 21st International Conference on, pp. 1197-1200, 2012.
[14] N. Ezaki, M. Bulacu, L. Schomaker, "Text detection from natural scene images: towards a system for visually impaired persons", Proceedings of the 17th International Conference on Pattern Recognition, ICPR, Cambridge, Vol. 2, pp. 683-686, 2004
[15] X. Chen , A. L. Yuille, “Detecting and reading text in natural scenes”, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2, pp. 366–373, 2004
[16] B. T. Nassu, R. Minetto, L. E. S. d. Oliveira, “Text line detection in document images: Towards a support system for the blind”, 12th International Conference on Document Analysis and Recognition, pp. 638–642, 2013
[17] C. N. Patel, A.A. Desai, "Segmentation of text lines into words for Gujarati handwritten text", In Proceedings of international conference on signal and image processing (ICSIP’10), IEEEXplore, 15–17, 2010
[18] J Dholakia, A Negi, M. S. Rama, "Zone identification in the printed Gujarati text", In the Proceedings of the eight international conference on document analysis and recognition (ICDAR’05), 2005.
[19] L Neumann, J Matas, "Real-time scene text localization and recognition", In the Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. pp. 3538–3545, 2012.
[20] C. Yi, Y L. Tian , "Text string detection from natural scenes by structurebased partition and grouping", IEEE Transactions on Image Processing, Vol. 20, No. 9, pp. 2594–2605, 2011.
[21] X. Yin, X. Yin, K. Huang , H. Hao, "Robust Text Detection in Natural Scene Images", in IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 36, No. 5, pp. 970-983, May 2014.
[22] G. K. Myers, R. C. Bolles, Q. Luong, J. A. Herson, H. B. Aradhye, "Rectification and recognition of text in 3-d scenes", International Journal on Document Analysis and Recognition Vol. 7, No. 2, pp. 147-158, 2005.
[23] C. Yi , Y. Tian, ”Localizing text in scene images by boundary clustering, stroke segmentation and string fragment classification”, IEEE Transaction on Image Process., Vol. 21, No. 9, pp. 4256-4268, Sep. 2012.
[24] Huang, Weilin, Z. Lin, J. Yang, , J. Wang. "Text localization in natural images using stroke feature transform and text covariance descriptors", In Proceedings of the IEEE International Conference on Computer Vision, pp. 1241-1248, 2013.
[25] H. Li , D. Doermann, “Text enhancement in digital video using multiple frame integration”, In Proceedings of the seventh ACM international conference on Multimedia (Part 1), pp. 19–22, 1999.
[26] Ali, Suhad A., Ashwaq T. Hashim. "Wavelet transform based technique for text image localization”, Karbala International Journal of Modern Science Vol. 2, No. B, pp. 138-144., 2016.
[27] M. Kumar, G. Lee, "Automatic text location from complex natural scene images", International conference on computer and automation engineering (ICCAE), Vol. 3, pp. 594–597, 2010.
[28] S. Lu, T. Chen, S. Tian, Lim J-H, Tan C-L, "Scene text extraction based on edges and support vector regression", International Journal on Document Analysis and Recognition (IJDAR) Vol. 18, Issue. 2, pp 125–135, 2015.
[29] A. Zhu, G. Wang, Y. Dong, "Detecting natural scenes text via auto image partition, two-stage grouping and two-layer classification", Pattern Recognition Letters, Vol. 67, pp. 153–162 ,2015.
[30] C Yu, Y Song, Y Zhang, "Scene text localization using edge analysis and feature pool", Neurocomputing Vol. 175, pp. 652–661, 2016.
[31] W Huang, Y Qiao, X Tang, "Robust scene text detection with convolution neural network induced Mser trees", In Proceedings of European Conference on Computer Vision, Springer, pp. 497–511, 2014
[32] Y. F. Pan, X. Hou, C. L. Liu, “A hybrid approach to detect and localize texts in natural scene images”, IEEE Transaction on Image Processing, Vol. 20, No. 3, pp. 800–813, Mar. 2011.
[33] Y Liu, S Goto, T Ikenaga, "A contour-based robust algorithm for text detection in color", IEICE TRANSACTIONS on Information and Systems, Vol. 89, No. 3, pp. 1221–1230, 2006.
[34] Y. Pan, X. Hou , C. Liu, "Text Localization in Natural Scene Images Based on Conditional Random Field", 10th International Conference on Document Analysis and Recognition, Barcelona, pp. 6-10, 2009.
[35] M. Petter, V. Fragoso, M. Turk, C. Baur, "Automatic text detection for mobile augmented reality translation", IEEE international conference on computer vision workshops (ICCV), pp 48–55, 2011.
[36] Y. Pan, X. Hou , C. Liu, "A Hybrid Approach to Detect and Localize Texts in Natural Scene Images", in IEEE Transactions on Image Processing, Vol. 20, No. 3, pp. 800-813, March 2011.
[37] C. Shi, B. Xiao, C. Wang and Y. Zhang, "Graph-Based Background Suppression for Scene Text Detection," 10th IAPR International Workshop on Document Analysis Systems, pp. 210-214, 2012.
[38] P. Shivakumara, W. Huang, T. Phan, and C. Tan, “Accurate video text detection through classification of low and high contrast images”, Pattern Recognition, Vol. 43, No. 6, pp. 2165–2185, 2010.
[39] J. Lee, P. Lee, S. Lee, A. Yuille, and C. Koch, “AdaBoost for text detection in natural scene”, in Proceeding of IEEE International Conference on Document Analysis and Recognition, pp. 429–434, 2011.
[40] X. Chen , A. L. Yuille, “Detecting and reading text in natural scenes”, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 366–373., 2004.
[41] X. Yin, X.-C. Yin, H.-W. Hao, and K. Iqbal, “Effective text localization in natural scene images with MSER, geometry-based grouping and AdaBoost”, Proceedings of the 21st International Conference on Pattern Recognition, pp. 725–728, Nov. 2012.
[42] C. Liu, C. Yang, X. Q. Ding, and K. Wang, “An improved scene text extraction method using conditional random field and optical character recognition”, International Conference on Document Analysis and Recognition, pp. 708–712, 2011.
[43] Shi, Baoguang, Xiang Bai, and Cong Yao. "An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition”, IEEE transactions on pattern analysis and machine intelligence, Vol. 39, No. 11, pp. 2298-2304, 2017.
[44] C. Yao, X. Zhang, X. Bai, W. Liu, Y. Ma, and Z. Tu, ”Detecting texts of arbitrary orientations in natural images”, IEEE Conference on Computer Vision and Pattern Recognition, pp. 1083–1090, 2012.
[45] A. R. Chowdhury, U. Bhattacharya, and S. K. Parui, ”Scene text detection using sparse stroke information and MLP”, in Proceeding of IEEE International Conference on Pattern Recognition., pp. 294–297, 2012.
[46] S. Lee, M. Cho, K. Jung, J. Kim, “Scene text extraction with edge constraint and text collinearity”, International Conference on Pattern Recognition, IEEE, pp. 3983–3986, 2010.
[47] K. Esther Amulya, P. Sanoop Kumar, "Scene Text Extraction using Stroke Width Transform", International Journal of Computer Sciences and Engineering, Vol. 6, Issue. 6, pp.375-379, 2018.
[48] C. Yan, H. Xie, S. Liu, J. Yin, Y. Zhang and Q. Dai, "Effective Uyghur Language Text Detection in Complex Background Images for Traffic Prompt Identification", in IEEE Transactions on Intelligent Transportation Systems, Vol. 19, No. 1, pp. 220-229, Jan. 2018.
[49] Dey, Soma, Rajat Subhra Goswami. "A morphological segmentation and curve-let features extraction on text region classification using SVM”, In Computational Intelligence and Computing Research (ICCIC), IEEE International Conference on, pp. 1-6, 2015.
[50] C Shi, C Wang, B Xiao, Y Zhang, S Gao,"Scene text detection using graph model built upon maximally stable extremal regions", Pattern Recognition Letter Vol. 34, Issue. 2, pp.107–116, 2013.
[51] C. Shi, C. Wang, B. Xiao, Y. Zhang, S. Gao, and Z. Zhang, “Scene text recognition using part-based tree-structured character detections”, in Proceeding IEEE Conference on Computer Vision and Pattern Recognition, pp. 2961–2968, 2013.
[52] H. Koo, D. H. Kim, “Scene text detection via connected component clustering and non-text filtering”, IEEE Transaction Image Processing, Vol. 22, No. 6, pp. 2296–2305, Jun. 2013.
[53] L. Neumann , J. Matas,”Real-time scene text location and recognition”, IEEE Conference on Computer Vision and Pattern Recognition, pp. 3538–3545, 2012.
[54] L. Neumann, J. Matas," Text localization in real-world images using efficiently pruned exhaustive search", In Proceedings of the international conference on document analysis and recognition (ICDAR), pp 687–691, 2011.
[55] H. Chen, S. S. Tsai, G. Schroth, D. M. Chen, R. Grzeszczuk and B. Girod, "Robust text detection in natural images with edge-enhanced Maximally Stable Extremal Regions", 18th IEEE International Conference on Image Processing, pp. 2609–2612, 2011.
[56] D. Karatzas, S. R. Mestre, J. Mas, F. Nourbakhsh and P. P. Roy, "ICDAR 2011 Robust Reading Competition - Challenge 1: Reading Text in Born-Digital Images (Web and Email)", International Conference on Document Analysis and Recognition, Beijing, pp. 1485-1490. IEEE, 2011.
[57] A. Shahab, F. Shafait , A. Dengel, "ICDAR 2011 Robust Reading Competition Challenge 2: Reading Text in Scene Images", International Conference on Document Analysis and Recognition, pp 1491–1496, 2011.
[58] D. Karatzas et al., "ICDAR 2015 competition on Robust Reading", 13th International Conference on Document Analysis and Recognition (ICDAR), Tunis, pp. 1156-1160, 2015.
[59] D. Karatzas et. al., "ICDAR 2013 robust reading competition”, In Document Analysis and Recognition (ICDAR), 12th International Conference on, pp. 1484-1493. IEEE, 2013.
[60] Lucas et al., "ICDAR 2003 robust reading competitions”, In Document Analysis and Recognition, Proceedings. Seventh International Conference on, pp. 682-687. IEEE, 2003.
[61] Kumar, Deepak, M. N. Prasad, A. G. Ramakrishnan. "Multi-script robust reading competition in icdar 2013”, In Proceedings of the 4th International Workshop on Multilingual OCR, p. 14. ACM, 2013.
[62] X. Jiang, J. Lian, Z. Xia, X. Feng, A. Hadid, "Fast Chinese character detection from complex scenes", Sixth International Conference on Image Processing Theory, Tools and Applications (IPTA), Oulu, pp. 1-4, 2016.
[63] X. Ren, K. Chen, X. Yang, Y. Zhou, J. He and J. Sun, "A new unsupervised convolutional neural network model for Chinese scene text detection", IEEE China Summit and International Conference on Signal and Information Processing (ChinaSIP), Chengdu, pp. 428-432, 2015.
[64] Y Wang, L Peng, S Wang, "A Multi-stage Method for Chinese Text Detection in News Videos”, Procedia Computer Science, Vol. 96, pp. 1409-1417, 2016.
[65] O. Zayene, S. M. Touj, J. Hennebert, R. Ingold and N. E. Ben Amara, "Data, protocol and algorithms for performance evaluation of text detection in Arabic news video”, In Advanced Technologies for Signal and Image Processing (ATSIP), 2nd International Conference on, pp. 258-263. IEEE, 2016.
[66] X. Chen, J. Yang, J. Zhang, A. Waibel, “Automatic detection and recognition of signs from natural scenes”, IEEE Trans. Image Process., Vol. 13, No. 1, pp. 87–99, Jan. 2004.
[67] M. Iwamura, T. Kobayashi, and K. Kise, “Recognition of multiple characters in a scene image using arrangement of local features”, in Proc. IEEE International Conference on Document Analysis and Recognition, pp. 1409–1413, 2011.
[68] M. Moradi, S. Mozaffari and A. A. Orouji, "Farsi/Arabic text extraction from video images by corner detection", 6th Iranian Conference on Machine Vision and Image Processing, Isfahan, pp. 1–6, 2010.
[69] Ghoshal, Ranjit, A. Roy, S. Parui. "Recognition of Bangla text from scene images through perspective correction”, In Image Information Processing (ICIIP), 2011 International Conference on, pp. 1-6. IEEE, 2011.
[70] H Gaddour, S Kanoun, N Vincent, "A New Method for Arabic Text Detection in Natural Scene Image Based on the Color Homogeneity”, In International Conference on Image and Signal Processing, Springer International Publishing pp. 127-136., 2016.
[71] O. Zayene, S. M. Touj, J. Hennebert, R. Ingold and N. E. Ben Amara, "Data, protocol and algorithms for performance evaluation of text detection in Arabic news video", 2nd International Conference on Advanced Technologies for Signal and Image Processing (ATSIP), Monastir, pp. 258-263, 2016.
[72] L. Xu, H. Nagayoshi, H. Sako, “Kanji character detection from complex real scene images based on character properties”, In Document Analysis Systems, 2008. DAS`08. The Eighth IAPR International Workshop on, IEEE, pp. 278–285, 2008.
[73] J. Park, G. Lee, E. Kim, J. Lim, S. Kim, H. Yang, M. Lee, and S. Hwang, “Automatic detection and recognition of Korean text in outdoor signboard images”, Pattern Recognition Letter., Vol. 31, No. 12, pp. 1728–1739, 2010.
[74] SH Lee, MS Cho, K Jung, JH Kim, "Scene text extraction with edge constraint and text collinearity link", In Proceedings of International Conference on Pattern Recognition. , pp. 3983–3986, 2010.
[75] A. Jamil, I. Siddiqi, F. Arif, A. Raza, “Edge-based features for localization of artificial Urdu text in video images”, in Proceeding IEEE International Conference In Document Analysis and Recognition, pp. 1120–1124, 2011.
[76] Y. Song et. al., "Robust and parallel Uyghur text localization in complex background images”, Machine Vision and Applications, Vol. 28, Issue. 7, pp. 755-769, 2017.
[77] S Fang, H Xie, Z Chen, S Zhu, X Gu, X Gao, "Detecting Uyghur text in complex background images with convolutional neural network”, Multimedia Tools and Applications Vol.76, No. 13 ,pp. 15083-15103, 2017.
[78] C. Yao, X. Bai, W. Liu, "A Unified Framework for Multioriented Text Detection and Recognition", in IEEE Transactions on Image Processing, Vol. 23, No. 11, pp. 4737-4749, Nov. 2014.
[79] P. Shivakumara, T. Q. Phan, C. L. Tan, “A Laplacian approach to multi-oriented text detection in video”, IEEE transactions on pattern analysis and machine intelligence, Vol. 33, No. 2, pp. 412–419, Feb. 2011.
[80] L Li, S Yu, L Zhong, X Li, "Multilingual text detection with nonlinear neural network”, Mathematical Problems in Engineering 2015
[81] M. R. Lyu, J. Song, M. Cai, “A comprehensive method for multilingual video text detection, localization, and extraction”, IEEE transactions on circuits and systems for video technology, Vol. 15, No. 2, pp. 243– 255, Feb. 2005.
[82] G. Zhou, Y. Liu, Q. Meng, and Y. Zhang, ”Detecting multilingual text in natural scene”, 1st International Symposium on Access Spaces (ISAS), Yokohama,, pp. 116–120, 2011.
[83] V.N. Manjunath Aradhya, M.S. Pavithra, "A comprehensive of transforms, Gabor filter and k-means clustering for text detection in images and video", Applied Computing and Informatics, Vol. 12, Issue. 2, Pages 109-116, 2016.
[84] L. Gomez, D. Karatzas,”Multi-script text extraction from natural scenes”, in Proceeding IEEE International Conference in Document Analysis and Recognition, pp. 467–471, 2013.
[85] Gomez, L. & Karatzas, "A Fast Hierarchical Method for Multi-script and Arbitrary Oriented Scene Text Extraction", IJDAR, Vol. 19, Issue. 4, pp 335–349, December 2016.
[86] X. Liu, H. Fu, and Y. Jia, “Gaussian Mixture modeling and learning of neighboring characters for multilingual text extraction in images”, Pattern Recognition, Vol. 41, No. 2, pp. 484–493, 2008.
[87] A. Negi, N. Kasinadhuni, "Localization and extraction of text in Telugu document images", TENCON. Conference on Convergent Technologies for Asia-Pacific Region, Vol. 2, pp. 749-752, 2003.
[88] A Negi, KN Shanker, CK Chereddi, "Localization, extraction and recognition of text in Telugu document images”, In Document Analysis and Recognition, Proceedings. Seventh International Conference on, pp. 1193-1197. IEEE, 2003.
[89] U. Bhattacharya, S. K. Parui and S. Mondal, "Devanagari and Bangla Text Extraction from Natural Scene Images", 10th International Conference on Document Analysis and Recognition, Barcelona, pp. 171-175, 2009.
[90] "D.V. Sharma, S. Singh "Gurmukhi text detection and localization in natural scene images", International Journal of Advanced Research in Computer Science and Software Engineering Vol. 5, Issue 12, pp. 442–458, 426-428, 2015
[91] S. Kaur, G.S Josan, “Gurmukhi Text Extraction From Image Using Support Vector Machine (SVM)”, International Journal of Engineering Research and Applications (IJERA), Vol. 3, No. 4, pp. 9-15, 2011.
[92] R Ghoshal, A Roy, TK Bhowmik, SK Parui, "Decision tree based recognition of Bangla text from outdoor scene images”, In International Conference on Neural Information Processing, pp. 538-546. Springer, Berlin, Heidelberg, 2011.
[93] S. V. Seeri, S. Giraddi and B. M. Prashant, "A novel approach for Kannada text extraction", International Conference on Pattern Recognition, Informatics and Medical Engineering (PRIME-2012), Salem, Tamilnadu, pp. 444-448, 2012.
[94] S. A.Angadi, M. M. Kodabagi. "A robust segmentation technique for line, word and character extraction from Kannada text in low resolution display board images”, In Signal and Image Processing (ICSIP), Fifth International Conference on, pp. 42-49. IEEE, 2014.
[95] T. E. de Campos, B. R. Babu, M. Varma, “Character recognition in natural images”, VISAPP Vol. 2, Issue 7, 2009.
[96] A. A. Desai, "Support vector machine for identification of handwritten Gujarati alphabets using hybrid feature space”, CSI transactions on ICT, Vol. 2, No. 4, pp. 235-241, 2015.
[97] A. A. Desai, "Gujarati handwritten numeral optical character recognition through neural network", Pattern Recognition, Vol. 43, Issue. 7, pp. 2582–2589, 2010.
[98] A. A. Desai, "Handwritten Gujarati numeral optical character recognition using Hybrid feature extraction technique", In Proceeding of international conference on image processing, Computer vision and pattern recognition, (IPCV’10), Vol. 2, pp. 733–739, 2010.
[99] C. N. Patel, A. A. Desai, "Zone identification for Gujarati handwritten words", In Proceedings of international conference on emerging applications of information technology (EAIT 2011), IEEEXplore, pp. 19–20, 2011.
[100] J. Gllavata, R. Ewerth, B. Freisleben, “Text detection in images based on unsupervised classification of high-frequency wavelet coefficients”, in Proceeding IEEE International Conference Pattern Recognition, pp. 425–428, 2004.
[101] Q. Ye, Q. Huang, W. Gao and D. Zhao, “Fast and robust text detection in images and video frames”, Image and Visison Computing, Vol. 23, pp. 565–576, 2005.
[102] Sobel, I., “An Isotropic 3×3 Gradient Operator”, Machine Vision for Three – Dimensional Scenes, Freeman, H., Academic Pres, NY, pp. 376-379, 1990.
[103] Haralick, Robert, M., Linda, G. S., “Computer and Robot Vision”, Addison-Wesley Longman: Boston, MA, USA, pp. 28–48, 1992.
[104] N. Otsu, “A Threshold Selection Method from GreyLevel Histogram”, IEEE transactions on systems, man, and cybernetics, Vol. 9, No. 1, pp. 62-66, 1979.