Statistical Selection of CNN-based Audiovisual Features for Instantaneous Estimation of Human Emotional States

Automatic prediction of continuous-level emotional state requires selection of suitable affective features to develop a regression system based on supervised machine learning. This paper investigates the performance of features statistically learned using convolutional neural networks for instantaneously predicting the continuous dimensions of emotional states. Features with minimum redundancy and maximum relevancy are chosen by using the mutual information-based selection process. The performance of frame-by-frame prediction of emotional state using the moderate length features as proposed in this paper is evaluated on spontaneous and naturalistic human-human conversation of RECOLA database. Experimental results show that the proposed model can be used for instantaneous prediction of emotional state with an accuracy higher than traditional audio or video features that are used for affective computation.

• Publication year

  2017

• Venue

  Italian Conference on Theoretical Computer Science

• Publication date

  2017-08-23

• Fields of study

  Computer Science

• Identifiers
  DOI 10.1109/ICTCS.2017.14arXiv 1708.07021
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Mutual information-based selection of audiovisual affective features to predict instantaneous emotional state

  2016cited by this paper
• How deep neural networks can improve emotion recognition on video data

  2016cited by this paper
• Differential components of discriminative 2D Gaussian-Hermite moments for recognition of facial expressions

  2016cited by this paper
• Video Affective Content Analysis: A Survey of State-of-the-Art Methods

  2015cited by this paper
• Deep Learning

  2015cited by this paper
• Dropout: a simple way to prevent neural networks from overfitting

  2014cited by this paper
• Mutual Information-Based Emotion Recognition

  2013cited by this paper
• Introducing the RECOLA multimodal corpus of remote collaborative and affective interactions

  2013cited by this paper
• LSTM-Modeling of continuous emotions in an audiovisual affect recognition framework

  2013cited by this paper
• Robust continuous prediction of human emotions using multiscale dynamic cues

  2012cited by this paper
• Continuous Emotion Recognition Using Gabor Energy Filters

  2011cited by this paper
• Naturalistic Affective Expression Classification by a Multi-stage Approach Based on Hidden Markov Models

  2011cited by this paper
• Emotional and Conversational Nonverbal Signals

  2004cited by this paper
• Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy

  2003cited by this paper
• Emotional expressions in audiovisual human computer interaction

  2000cited by this paper
• Support Vector Regression Machines

  1996influential reference
• Affective judgment and psychophysiological response: Dimensional covariation in the evaluation of pictorial stimuli.

  1989cited by this paper
• Learning representations by back-propagating errors

  1986cited by this paper
• Grundzüge der physiologischen psyhcologie

  1893cited by this paper

• Prediction of instantaneous likeability of advertisements using deep learning

  2021cites this paper
• An Improved Multimodal Dimension Emotion Recognition Based on Different Fusion Methods

  2020cites this paper
• Estimation of affective dimensions using CNN-based features of audiovisual data

  2019cites this paper