Open Access   Article Go Back

Holistic Approach of Indian Sign Language Prediction Software with Emotion Detection

Dipankar Mazumder1 , Upamita Das2 , Hillal Kumar Roy3 , Nilava Sarkar4 , Abhishek Kumar Singh5

Section:Research Paper, Product Type: Journal Paper
Volume-11 , Issue-01 , Page no. 155-160, Nov-2023

Online published on Nov 30, 2023

Copyright © Dipankar Mazumder, Upamita Das, Hillal Kumar Roy, Nilava Sarkar, Abhishek Kumar Singh . This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

View this paper at   Google Scholar | DPI Digital Library

XML View
PDF Download

How to Cite this Paper

  • IEEE Citation
  • MLA Citation
  • APA Citation
  • BibTex Citation
  • RIS Citation

IEEE Style Citation: Dipankar Mazumder, Upamita Das, Hillal Kumar Roy, Nilava Sarkar, Abhishek Kumar Singh, “Holistic Approach of Indian Sign Language Prediction Software with Emotion Detection,” International Journal of Computer Sciences and Engineering, Vol.11, Issue.01, pp.155-160, 2023.

MLA Style Citation: Dipankar Mazumder, Upamita Das, Hillal Kumar Roy, Nilava Sarkar, Abhishek Kumar Singh "Holistic Approach of Indian Sign Language Prediction Software with Emotion Detection." International Journal of Computer Sciences and Engineering 11.01 (2023): 155-160.

APA Style Citation: Dipankar Mazumder, Upamita Das, Hillal Kumar Roy, Nilava Sarkar, Abhishek Kumar Singh, (2023). Holistic Approach of Indian Sign Language Prediction Software with Emotion Detection. International Journal of Computer Sciences and Engineering, 11(01), 155-160.

BibTex Style Citation:
@article{Mazumder_2023,
author = {Dipankar Mazumder, Upamita Das, Hillal Kumar Roy, Nilava Sarkar, Abhishek Kumar Singh},
title = {Holistic Approach of Indian Sign Language Prediction Software with Emotion Detection},
journal = {International Journal of Computer Sciences and Engineering},
issue_date = {11 2023},
volume = {11},
Issue = {01},
month = {11},
year = {2023},
issn = {2347-2693},
pages = {155-160},
url = {https://www.ijcseonline.org/full_spl_paper_view.php?paper_id=1427},
publisher = {IJCSE, Indore, INDIA},
}

RIS Style Citation:
TY - JOUR
UR - https://www.ijcseonline.org/full_spl_paper_view.php?paper_id=1427
TI - Holistic Approach of Indian Sign Language Prediction Software with Emotion Detection
T2 - International Journal of Computer Sciences and Engineering
AU - Dipankar Mazumder, Upamita Das, Hillal Kumar Roy, Nilava Sarkar, Abhishek Kumar Singh
PY - 2023
DA - 2023/11/30
PB - IJCSE, Indore, INDIA
SP - 155-160
IS - 01
VL - 11
SN - 2347-2693
ER -

           

Abstract

A real-time AI software solution for a holistic approach to recognizing Indian Sign Language (ISL) where elements of ISL such as hand shape, facial expression, orientation, movement etc. are analyzed, recognized, and converted into written text. Sentences are formed by analyzing each sign one by one and overlapping detections are ignored. It is a software solution that a user can run on their system without installing any dependencies. We also use emotion detection to understand what a person is trying to say as any human being will have emotions while they convey their message. The model is also trained with an ideal state where if no signs are being shown, that is if there are no hand movements, no sign is predicted.

Key-Words / Index Term

Mediapipe, LSTM, CV2, Indian Sign Language, DeepFace, PyInstaller, Keras, CNN

References

[1]. Munib Q., Habeeb M., Takruri B. and Al-Malik H. “A. American Sign Language (ASL) recognition is based on Hough transform and neural networks”, Expert Systems with Applications, Vol.32, pp.24-37, 2007.
[2]. Zeshan U., Vasishta M. M. and Sethna M, “Implementation of Indian Sign Language in Educational Settings”, Asia Pacific Disability Rehabilitation Journal. Vol.1, pp.16-40,2005.
[3]. Vasishta M., Woodward J. and Wilson K, “Sign language in India: regional variation within the deaf population”, Indian Journal of Applied Linguistics. Vol.4, Issue.2, pp.66- 74, 1978.
[4]. Suryapriya A. K., Sumam S. and Idicula M, “Design and Development of a Frame-Based MT System for English- to-ISL”, World Congress on Nature and Biologically Inspired Computing. pp.1382-1387, 2009.
[5]. Kshirsagar K. P. and Doye D, “Object-Based Key Frame Selection for Hand Gesture Recognition”, Advances in Recent Technologies in Communication and Computing (ARTCom) International Conference on. pp.181-185, 2010.
[6]. Davydov M. V., Nikolski I. V. and Pasichnyk V. V, “Real-time Ukrainian sign language recognition system”, Intelligent Computing and Intelligent Systems (ICIS), IEEE International Conference on , pp.875-879, 2010.
[7]. Shanableh T. and Assaleh K, “Arabic sign language recognition in user-independent mode”, Intelligent and Advanced Systems ICIAS 2007 International Conference on. pp.597-600, 2007.
[8]. Xiaolong T., Bian W., Weiwei Y. and Chongqing Liu, “A hand gesture recognition system based on locally linear embedding”, Journal of Visual Languages and amp; Computing, pp.442-454, 2005.
[9]. Rana, S, Liu, W., Lazarescu, M and Venkatesh, S, “A unified tensor framework for face recognition”, Pattern Recognition, First edition, ELSEVIER Publisher, Australia, pp.2850-2862, 2009.
[10]. Wang S., Zhang D., Jia C., Zhang N., Zhou C. and Zhang L, “A Sign Language Recognition Based on Tensor. Multimedia and Information Technology (MMIT) Second International Conference on. Vol.2, pp.192-195, 2009.