Detecting Face-Touching Gestures with Smartwatches and Deep Learning Networks

Everyday actions like scratching one's nose or resting the chin on one's hand may facilitate the spread of germs and diseases. Detecting these gestures holds the potential for innovative health monitoring and disease prevention applications. However, the identification of face-touching poses challenges due to the variety of human gestures and the limitations in accuracy associated with wearable sensors. This study introduces an original deep-learning system to identify face-touching gestures using standard smartwatches' inertial measurement unit sensors. The system on the smartwatch captures and pre-processes multi-channel time-series data from the accelerometer to generate robust input features. We utilize a benchmark, the Face Touching dataset, to evaluate the recognition performance of deep learning networks, including our proposed network. Our approach proposes a hybrid deep residual network architecture tailored for sequence-based gesture classification using signals from the smartwatch sensors. In our experiments, the developed deep learning framework achieves an impressive F1-score of 97.53% in detecting face-touch gestures. The suggested system takes a step forward in advancing the practical applications of face-touch gesture detection to smartwatch-based health sensing and disease prevention.

• Publication year

  2024

• Venue

  International Conference on Telecommunications and Signal Processing

• Publication date

  2024-07-10

• Fields of study

  Medicine, Computer Science

• Identifiers
  DOI 10.1109/TSP63128.2024.10605968
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Enhanced Complex Human Activity Recognition System: A Proficient Deep Learning Framework Exploiting Physiological Sensors and Feature Learning

  2023cited by this paper
• Attention-Based Hybrid Deep Learning Network for Human Activity Recognition Using WiFi Channel State Information

  2023cited by this paper
• Hybrid convolution neural network with channel attention mechanism for sensor-based human activity recognition

  2023cited by this paper
• A Deep Learning Network with Aggregation Residual Transformation for Human Activity Recognition Using Inertial and Stretch Sensors

  2023cited by this paper
• Physical Activity Recognition Based on Deep Learning Using Photoplethysmography and Wearable Inertial Sensors

  2023cited by this paper
• Automatic Recognition of Construction Worker Activities Using Deep Learning Approaches and Wearable Inertial Sensors

  2023cited by this paper
• A CNN-MLP Deep Model for Sensor-based Human Activity Recognition

  2023cited by this paper
• Hyperparameter Tuning in Convolutional Neural Network for Face Touching Activity Recognition using Accelerometer Data

  2022cited by this paper
• FallNeXt: A Deep Residual Model based on Multi-Branch Aggregation for Sensor-based Fall Detection

  2022cited by this paper
• A Deep Learning-based Model for Human Activity Recognition using Biosensors embedded into a Smart Knee Bandage

  2022cited by this paper
• Badminton Activity Recognition and Player Assessment based on Motion Signals using Deep Residual Network

  2022cited by this paper
• A Deep Residual-based Model on Multi-Branch Aggregation for Stress and Emotion Recognition through Biosignals

  2022cited by this paper
• Enhancement of Human Complex Activity Recognition using Wearable Sensors Data with InceptionTime Network

  2021cited by this paper
• Biometrics in the Era of COVID-19: Challenges and Opportunities

  2021cited by this paper
• FaceGuard: A Wearable System To Avoid Face Touching

  2021cited by this paper
• Computer-Vision based Face Mask Detection using CNN

  2021cited by this paper
• Sensor-based Complex Human Activity Recognition from Smartwatch Data using Hybrid Deep Learning Network

  2021cited by this paper
• Using Smartwatches to Detect Face Touching

  2021influential reference
• Bidirectional Gate Recurrent Unit Neural Network for Recognizing Face Touching Activities using Smartwatch Sensors

  2021cited by this paper
• Multimodal hand gesture recognition using single IMU and acoustic measurements at wrist

  2020cited by this paper
• Application of Deep Learning to IMU sensor motion

  2019cited by this paper
• Real-time Arm Skeleton Tracking and Gesture Inference Tolerant to Missing Wearable Sensors

  2019cited by this paper
• Leveraging CNN and Transfer Learning for Vision-based Human Activity Recognition

  2019cited by this paper
• End-to-End Learning Framework for IMU-Based 6-DOF Odometry

  2019cited by this paper
• A systematic literature review on vision based gesture recognition techniques

  2018cited by this paper
• Real-Time and Embedded Detection of Hand Gestures with an IMU-Based Glove

  2018cited by this paper
• Recognition of Human Hand Activities Based on a Single Wrist IMU Using Recurrent Neural Networks

  2017cited by this paper
• Combining low-cost Inertial Measurement Unit (IMU) and deep learning algorithm for predicting vehicle attitude

  2017cited by this paper
• Face touching: A frequent habit that has implications for hand hygiene

  2015cited by this paper
• Recent methods and databases in vision-based hand gesture recognition: A review

  2015cited by this paper

• Orthopedic Walker Fall Detection and Motion Classification Using Bidirectional Gated Recurrent Unit Neural Network

  2025cites this paper