Thai finger spelling localization and classification under complex background using a YOLO-based deep learning

Sign language recognition has been actively studied and remains a challenge in computer vision. The finger spelling is an integral part of a sign language. This study focuses on Thai finger spelling(TFS), especially TFS single hand schema under complex background condition. We proposed a YOLO-based Thai finger spelling(Y-TFS) that used the convolution neural network architecture to localize and classify 25 TFS signs. The experiment on the training dataset of 15,000 images and test dataset of 15,000 images shows that our system has performed well and is robust against various background conditions. For the Thai fingerspelling recognition, our Y-TFS achieved the mAPs of 82.06% under a complex background and 84.99 % under a plain background.

• Publication year

  2019

• Venue

  International Conference on Computer Modeling and Simulation

• Publication date

  2019-01-16

• Fields of study

  Computer Science

• Identifiers
  DOI 10.1145/3307363.3307403
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Hand Sign Recognition for Thai Finger Spelling: an Application of Convolution Neural Network

  2018cited by this paper
• Thai Finger-Spelling Recognition Using a Cascaded Classifier Based on Histogram of Orientation Gradient Features

  2017cited by this paper
• YOLO9000: Better, Faster, Stronger

  2016influential reference
• Reliability and accuracy of Thai sign language recognition with Kinect sensor

  2016cited by this paper
• Hand Gesture Recognition for Thai Sign Language in Complex Background Using Fusion of Depth and Color Video

  2016influential reference
• Depth Sensor Based Automatic Hand Region Extraction by Using Time-series Curve and Its Application to Japanese Finger-spelled Sign Language Recognition

  2015cited by this paper
• You Only Look Once: Unified, Real-Time Object Detection

  2015cited by this paper
• Hand detection based on depth information and color information of the Kinect

  2015cited by this paper
• Alphabetic hand sign interpretation using geometric invariance

  2014cited by this paper
• Microsoft COCO: Common Objects in Context

  2014cited by this paper
• Very Deep Convolutional Networks for Large-Scale Image Recognition

  2014cited by this paper
• English Letters Finger Spelling Sign Language Recognition System

  2014cited by this paper
• TFRS: Thai finger-spelling sign language recognition system

  2012cited by this paper
• American Sign Language word recognition with a sensory glove using artificial neural networks

  2011cited by this paper
• The Pascal Visual Object Classes (VOC) Challenge

  2010cited by this paper
• Automatic recognition of fingerspelled words in British Sign Language

  2009cited by this paper
• Intelligent Thai text - Thai sign translation for language learning

  2008cited by this paper
• Written language of deaf and hard-of-hearing students in public schools.

  2005cited by this paper
• Histograms of oriented gradients for human detection

  2005cited by this paper
• A Linguistic Feature Vector for the Visual Interpretation of Sign Language

  2004cited by this paper
• Learning-based hand sign recognition using SHOSLIF-M

  1995cited by this paper
• Real-time American Sign Language recognition from video using hidden Markov models

  1995cited by this paper
• Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position

  1980cited by this paper
• ThinkIR: The University of Louisville's Institutional Repository ThinkIR: The University of Louisville's Institutional Repository

  year unknowncited by this paper

• Mathematical and computational perspectives on next-generation neural networks for sign language recognition: A systematic review of advances, challenges, and assistive applications

  2026cites this paper
• Deep Multimodal-Based Recognition of Thai Finger Spelling with Two-Handed Postures

  2024influential citation
• A Comprehensive Benchmark and Evaluation of Thai Finger Spelling in Multi-Modal Deep Learning Models

  2024cites this paper
• BdSpell: A YOLO-based Real-time Finger Spelling System for Bangla Sign Language

  2023cites this paper
• Thai Finger-Spelling using Vision Transformer

  2023cites this paper
• TFS Recognition: Investigating MPH]{Thai Finger Spelling Recognition: Investigating MediaPipe Hands Potentials

  2022cites this paper
• Automatic Thai Finger Spelling Transcription

  2021cites this paper
• Deep Learning for Sign Language Recognition: Current Techniques, Benchmarks, and Open Issues

  2021cites this paper
• Thai Fingerspelling Recognition Using Hand Landmark Clustering

  2021cites this paper
• Real-Time Bangla Sign Language Detection with Sentence and Speech Generation

  2020influential citation