Deep Learning-Based Sign Language Communication System with Multi-Language Support

Speech impairment is regarded as one of the significant disabilities. When it comes to communicating with others, people with this limitation employ sign language. However, individuals with speech disabilities are unable to communicate with those who do not understand sign language. Therefore, our initiative aims to bridge this communication gap. The primary goal of this work is to develop a vision-based system that can recognise sign language motions in real-time and provide translation in multiple languages. We trained a CNN model on both spatial and temporal features present in real-time video sequences, achieving an accuracy of 97.5%. For multilingual translation, we employed the T5 transformer, achieving an average BLEU score of 0.193, ROUGE-L score of 0.038, and 0.224 seconds per sentence for English-to-Hindi, and a BLEU score of 0.048, ROUGE-L score of 0.022, and 0.434 seconds per sentence for English-to-Kannada translation.

• Publication year

  2025

• Venue

  IEEE India Conference

• Publication date

  2025-12-18

• Fields of study

  Not labeled

• Identifiers
  DOI 10.1109/INDICON68490.2025.11392980
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Spatial–temporal attention with graph and general neural network-based sign language recognition

  2024cited by this paper
• Spatial-temporal feature-based End-to-end Fourier network for 3D sign language recognition

  2024cited by this paper
• Empowering Communication: A Deep Learning Framework for Arabic Sign Language Recognition with an Attention Mechanism

  2024cited by this paper
• Isolated sign language recognition through integrating pose data and motion history images

  2024cited by this paper
• Sign Language Recognition Using Graph and General Deep Neural Network Based on Large Scale Dataset

  2024cited by this paper
• Natural Language-Assisted Sign Language Recognition

  2023cited by this paper
• SignBERT+: Hand-Model-Aware Self-Supervised Pre-Training for Sign Language Understanding

  2023cited by this paper
• Hindi to English: Transformer-Based Neural Machine Translation

  2021cited by this paper
• Multi-view Subword Regularization

  2021cited by this paper
• LTRC-MT Simple & Effective Hindi-English Neural Machine Translation Systems at WAT 2019

  2019cited by this paper
• Iterative Alignment Network for Continuous Sign Language Recognition

  2019cited by this paper
• Neural Sign Language Translation

  2018cited by this paper
• Attention is All you Need

  2017cited by this paper
• Neural Machine Translation by Jointly Learning to Align and Translate

  2014cited by this paper
• Sequence to Sequence Learning with Neural Networks

  2014cited by this paper

• No citing papers are available for this paper.