Holistic Approach of Indian Sign Language Prediction Software with Emotion Detection

Authors

Keywords:

Mediapipe, LSTM, CV2, Indian Sign Language, DeepFace, PyInstaller

Abstract

A real-time AI software solution for a holistic approach to recognizing Indian Sign Language (ISL) where elements of ISL such as hand shape, facial expression, orientation, movement etc. are analyzed, recognized, and converted into written text. Sentences are formed by analyzing each sign one by one and overlapping detections are ignored. It is a software solution that a user can run on their system without installing any dependencies. We also use emotion detection to understand what a person is trying to say as any human being will have emotions while they convey their message. The model is also trained with an ideal state whereif no signs are being shown, that is if there are no hand movements, no sign is predicted

References

[1] Munib Q., Habeeb M., Takruri B. and Al-Malik H. “A. American Sign Language (ASL) recognition is based on Hough transform and neural networks”, Expert Systems with Applications, Vol.32, pp.24-37, 2007.

[2] Zeshan U., Vasishta M. M. and Sethna M, “Implementation of Indian Sign Language in Educational Settings”, Asia Pacific Disability Rehabilitation Journal. Vol.1, pp.16-40,2005.

[3] Vasishta M., Woodward J. and Wilson K, “Sign language in India: regional variation within the deaf population”, Indian Journal of Applied Linguistics. Vol.4, Issue.2, pp.66- 74, 1978.

[4] Suryapriya A. K., Sumam S. and Idicula M, “Design and Development of a Frame-Based MT System for English- to-ISL”, World Congress on Nature and Biologically Inspired Computing. pp.1382-1387, 2009.

[5] Kshirsagar K. P. and Doye D, “Object-Based Key Frame Selection for Hand Gesture Recognition”, Advances in Recent Technologies in Communication and Computing (ARTCom) International Conference on. pp.181-185, 2010.

[6] Davydov M. V., Nikolski I. V. and Pasichnyk V. V, “Real-time Ukrainian sign language recognition system”, Intelligent Computing and Intelligent Systems (ICIS), IEEE International Conference on , pp.875-879, 2010.

[7] Shanableh T. and Assaleh K, “Arabic sign language recognition in user-independent mode”, Intelligent and Advanced Systems ICIAS 2007 International Conference on. pp.597-600, 2007.

[8] Xiaolong T., Bian W., Weiwei Y. and Chongqing Liu, “A hand gesture recognition system based on locally linear embedding”, Journal of Visual Languages and amp; Computing, pp.442-454, 2005.

[9] Rana, S, Liu, W., Lazarescu, M and Venkatesh, S, “A unified tensor framework for face recognition”, Pattern Recognition, First edition, ELSEVIER Publisher, Australia, pp.2850-2862, 2009.

[10] Wang S., Zhang D., Jia C., Zhang N., Zhou C. and Zhang L, “A Sign Language Recognition Based on Tensor. Multimedia and Information Technology (MMIT) Second International Conference on. Vol.2, pp.192-195, 2009.

Downloads

Published

2026-01-19

How to Cite

[1]
D. Mazumder, U. Das, H. K. Roy, Nilava Sarkar, and A. K. Singh, “Holistic Approach of Indian Sign Language Prediction Software with Emotion Detection”, Int. J. Comp. Sci. Eng., vol. 11, no. 1, pp. 155–160, Jan. 2026.

Issue

Vol. 11 No. 1 (2023): Special Issue-1 Nov Editon

Section

Research Article

License

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Authors contributing to this journal agree to publish their articles under the Creative Commons Attribution 4.0 International License, allowing third parties to share their work (copy, distribute, transmit) and to adapt it, under the condition that the authors are given credit and that in the event of reuse or distribution, the terms of this license are made clear.