Advanced Sign Language Translation: A Holistic Network for Hand Gesture Recognition Using Deep Learning

Sign language recognition (SLR) requires interpreting dynamic hand gestures with complex variations in shape, orientation, motion, and spatial configuration. Conventional models such as U‐Net and ResNet offer strengths in segmentation and feature extraction, respectively, but face critical limitations. U‐Net struggles with retaining fine spatial details in cluttered backgrounds and lacks temporal modeling, while ResNet can lose motion continuity and suffers from vanishing gradient issues in deeper architectures. To overcome these challenges, we propose the holistic sign language interpretation network (HSLIN), a novel deep learning framework tailored for Indian sign language (ISL) recognition. HSLIN incorporates three key innovations: Uniformed frame isolation and augmentation (UFIA) for standardized preprocessing and noise removal, synaptic gesture movement analysis (SGMA) for capturing detailed motion using keypoint detection and optical flow, and a hybrid architecture combining U‐Net‐based segmentation with an enhanced ResNet‐TC50V2 backbone. The novelty lies in fusing spatial precision with deep temporal modeling through bottleneck layers and temporal convolutional layers (TCL), enabling the model to effectively learn gesture patterns over time. Experimental results on the ISL‐CSLTR dataset demonstrate that the proposed method achieves an accuracy of 99.9%, a precision of 100%, recall of 99.9%, and an F1‐score of 100% across 14 word‐level sign classes. Furthermore, an ablation study confirms the critical role of each architectural component in achieving optimal performance. These outcomes clearly establish the robustness, efficiency, and uniqueness of the proposed HSLIN framework, positioning it as a powerful solution for real‐world ISL recognition and communication accessibility for the deaf and hard‐of‐hearing community.

• Publication year

  2026

• Venue

  Comput. Animat. Virtual Worlds

• Publication date

  2026-01-01

• Fields of study

  Linguistics, Computer Science

• Identifiers
  DOI 10.1002/cav.70084
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Estimation of human body 3D pose for parent-infant interaction settings using azure Kinect and OpenPose

  2024cited by this paper
• Signlanghub: Empowering Communication Through Sign Language Recognition and Voice-to Action Conversion

  2024cited by this paper
• Unraveling a Decade: A Comprehensive Survey on Isolated Sign Language Recognition

  2023cited by this paper
• Word-level Sign Language Recognition with Multi-stream Neural Networks Focusing on Local Regions

  2022cited by this paper
• An Instrumented Glove for Recognition of Brazilian Sign Language Alphabet

  2022cited by this paper
• Sentence2SignGesture: a hybrid neural machine translation network for sign language video generation

  2022cited by this paper
• American Sign Language Words Recognition of Skeletal Videos Using Processed Video Driven Multi-Stacked Deep LSTM

  2022cited by this paper
• Sign Pose-based Transformer for Word-level Sign Language Recognition

  2022cited by this paper
• Deepsign: Sign Language Detection and Recognition Using Deep Learning

  2022cited by this paper
• Development of an End-to-End Deep Learning Framework for Sign Language Recognition, Translation, and Video Generation

  2022cited by this paper
• A Systematic Review on Systems-Based Sensory Gloves for Sign Language Pattern Recognition: An Update From 2017 to 2022

  2022cited by this paper
• Deep Learning for Sign Language Recognition: Current Techniques, Benchmarks, and Open Issues

  2021cited by this paper
• Dynamic GAN for high-quality sign language video generation from skeletal poses using generative adversarial networks

  2021cited by this paper
• American sign language recognition and training method with recurrent neural network

  2021cited by this paper
• Hand Pose Guided 3D Pooling for Word-level Sign Language Recognition

  2021cited by this paper
• ASL-3DCNN: American sign language recognition technique using 3-D convolutional neural networks

  2021cited by this paper
• Self-Mutual Distillation Learning for Continuous Sign Language Recognition

  2021cited by this paper
• Robust hand gesture recognition using multiple shape-oriented visual cues

  2021cited by this paper
• Advances in machine translation for sign language: approaches, limitations, and challenges

  2021cited by this paper
• Methods, Databases and Recent Advancement of Vision-Based Hand Gesture Recognition for HCI Systems: A Review

  2021cited by this paper
• CNN based feature extraction and classification for sign language

  2020cited by this paper
• Multi-Information Spatial–Temporal LSTM Fusion Continuous Sign Language Neural Machine Translation

  2020cited by this paper
• Understanding vision-based continuous sign language recognition

  2020cited by this paper
• Perspective and Evolution of Gesture Recognition for Sign Language: A Review

  2020cited by this paper
• A new framework for sign language alphabet hand posture recognition using geometrical features through artificial neural network (part 1)

  2020cited by this paper
• Movement Trajectory Recognition of Sign Language Based on Optimized Dynamic Time Warping

  2020cited by this paper
• Hand sign language recognition using multi-view hand skeleton

  2020cited by this paper
• Hand Gesture Recognition for Sign Language Using 3DCNN

  2020cited by this paper
• DeepArSLR: A Novel Signer-Independent Deep Learning Framework for Isolated Arabic Sign Language Gestures Recognition

  2020cited by this paper
• Advances, Challenges and Opportunities in Continuous Sign Language Recognition

  2019cited by this paper
• Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparison

  2019cited by this paper
• Neural Machine Translation by Jointly Learning to Align and Translate

  2014cited by this paper
• Sign language translation system

  2014cited by this paper

• No citing papers are available for this paper.