Emotion recognition with multiple physiological parameters based on ensemble learning

Emotion recognition is a key research area in artificial intelligence, playing a critical role in enhancing human-computer interaction and optimizing user experience design. This study explores the application and effectiveness of ensemble learning methods for emotion recognition based on multiple physiological parameters. A dataset was systematically constructed by preprocessing data from electroencephalogram (EEG), galvanic skin response (GSR), skin temperature (ST), and heart rate (HR) collected from 38 subjects while watching short videos. We proposed a hybrid model framework combining Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) networks, trained and optimized using a random seed initialization strategy and a cosine annealing warm restart strategy. To further enhance performance, various strategies were designed and evaluated. The results showed that applying advanced preprocessing techniques significantly improved data quality, while the hybrid model effectively leveraged the advantages of both CNN and LSTM. Incorporating the cosine annealing warm restart strategy further boosted model performance. Using a soft voting ensemble method, the proposed approach achieved a 96.21% accuracy rate in classifying seven emotions—calm, happy, disgust, surprise, anger, sad, and fear, indicating its ability to accurately capture emotional responses to short videos. This study presents an innovative approach to emotion recognition using multiple physiological parameters, demonstrating the potential of ensemble learning for complex tasks. It offers valuable insights for the development of effective applications.

• Publication year

  2025

• Venue

  Scientific Reports

• Publication date

  2025-06-06

• Fields of study

  Medicine, Computer Science

• Identifiers
  DOI 10.1038/s41598-025-96616-0PMID 40473884PMCID 12141732
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• Exploring emotional experiences and dataset construction in the era of short videos based on physiological signals

  2024cited by this paper
• A computationally efficient speech emotion recognition system employing machine learning classifiers and ensemble learning

  2024cited by this paper
• Prompt Consistency for Multi-Label Textual Emotion Detection

  2024cited by this paper
• Research on Multimodal Emotion Recognition Based on Fusion of Electroencephalogram and Electrooculography

  2024cited by this paper
• Correction to: Convolutional neural network and ensemble machine learning model for optimizing performance of emotion recognition in wild

  2024cited by this paper
• A Novel Emotion Recognition Method Based on the Feature Fusion of Single-Lead EEG and ECG Signals

  2024cited by this paper
• Emotion Recognition System Based on Two-Level Ensemble of Deep-Convolutional Neural Network Models

  2023cited by this paper
• Multimodal Emotion Recognition From EEG Signals and Facial Expressions

  2023cited by this paper
• Transformer-Based Self-Supervised Multimodal Representation Learning for Wearable Emotion Recognition

  2023cited by this paper
• Knowledge-based BERT word embedding fine-tuning for emotion recognition

  2023cited by this paper
• Research on Emotion Recognition Based on Parameter Transfer and Res2Net

  2023influential reference
• EEG-based emotion recognition using modified covariance and ensemble classifiers

  2023cited by this paper
• Convolutional neural network and ensemble machine learning model for optimizing performance of emotion recognition in wild

  2023cited by this paper
• Emotion Arousal Assessment Based on Multimodal Physiological Signals for Game Users

  2023cited by this paper
• Emotion Recognition from Multimodal Data: a machine learning approach combining classical and hybrid deep architectures

  2023cited by this paper
• A novel driver emotion recognition system based on deep ensemble classification

  2023cited by this paper
• Facial expression recognition using bidirectional LSTM - CNN

  2023cited by this paper
• Developing a Physiological Signal-Based, Mean Threshold and Decision-Level Fusion Algorithm (PMD) for Emotion Recognition

  2023cited by this paper
• EEG-based emotion recognition using hybrid CNN and LSTM classification

  2022cited by this paper
• One-dimensional convolutional neural networks for low/high arousal classification from electrodermal activity

  2022cited by this paper
• Motor imagery EEG classification algorithm based on CNN-LSTM feature fusion network

  2022cited by this paper
• Emotion Recognition using Short-Term Multi-Physiological Signals

  2022cited by this paper
• CNN and LSTM based ensemble learning for human emotion recognition using EEG recordings

  2022cited by this paper
• Multimodal emotion recognition based on manifold learning and convolution neural network

  2022cited by this paper
• Robust Speech Emotion Recognition Using CNN+LSTM Based on Stochastic Fractal Search Optimization Algorithm

  2022cited by this paper
• Multi-modal emotion recognition using EEG and speech signals

  2022cited by this paper
• Face masks impair facial emotion recognition and induce specific emotion confusions

  2022cited by this paper
• LSTM-Modeling of Emotion Recognition Using Peripheral Physiological Signals in Naturalistic Conversations

  2022cited by this paper
• Uncertain Facial Expression Recognition via Multi-Task Assisted Correction

  2022cited by this paper
• A survey on long short-term memory networks for time series prediction

  2021cited by this paper
• Working Memory Connections for LSTM

  2021cited by this paper
• Real time human action recognition using triggered frame extraction and a typical CNN heuristic

  2020cited by this paper
• Machine-Learning-Based Diagnostics of EEG Pathology

  2020cited by this paper
• The Design of CNN Architectures for Optimal Six Basic Emotion Classification Using Multiple Physiological Signals

  2020cited by this paper
• Self-Supervised ECG Representation Learning for Emotion Recognition

  2020cited by this paper
• LSTM-Based Emotion Detection Using Physiological Signals: IoT Framework for Healthcare and Distance Learning in COVID-19

  2020cited by this paper
• Introducing WESAD, a Multimodal Dataset for Wearable Stress and Affect Detection

  2018cited by this paper
• Snapshot Ensembles: Train 1, get M for free

  2017cited by this paper
• Emotion recognition based on the multiple physiological signals

  2016cited by this paper
• EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces

  2016cited by this paper
• The SWELL Knowledge Work Dataset for Stress and User Modeling Research

  2014cited by this paper

• Transferable and Noise-Resilient Detection of Fear Emotion from Speech via Hybrid Acoustic-Temporal Modeling

  2025cites this paper
• Embodied AI: Multimodal Integration of Facial Expressions and Biometric Signals

  2025cites this paper
• A Comprehensive Review of Multimodal Emotion Recognition: Techniques, Challenges, and Future Directions

  2025cites this paper
• Research of Physiological Monitoring Models in the Military Domain

  2025cites this paper
• A survey on video emotion recognition: Segmentation, classification, and explainable AI techniques

  2025cites this paper
• Stress Classification using Vanilla Transformer Encoder on Multi-Channel Physiological Sensor Time Series

  2025cites this paper
• DiscBioSem-LM: A Framework for Anxiety Detection via Semantic Fusion of Multimodal Physiological Signals and Large Language Models

  2025cites this paper
• Emotional Analysis of a Subject using the GSR and ECG by Applied Machine Learning Techniques

  2025cites this paper
• Mental States and Machine: Enhancing Driver Engagement in Automated Vehicles for Safer Transitions

  year unknowncites this paper