VESUS: A Crowd-Annotated Database to Study Emotion Production and Perception in Spoken English

We introduce the Varied Emotion in Syntactically Uniform Speech (VESUS) repository as a new resource for the speech community. VESUS is a lexically controlled database, in which a semantically neutral script is portrayed with different emotional inflections. In total, VESUS contains over 250 distinct phrases, each read by ten actors in five emotional states. We use crowd sourcing to obtain ten human ratings for the perceived emotional content of each utterance. Our unique database construction enables a multitude of scientific and technical ex-plorations. To jumpstart this effort, we provide benchmark performance on three distinct emotion recognition tasks us-ing VESUS: longitudinal speaker analysis, extrapolating across syntactical complexity, and generalization to a new speaker.

• Publication year

  2019

• Venue

  Interspeech

• Publication date

  2019-09-15

• Fields of study

  Linguistics, Computer Science

• Identifiers
  DOI 10.21437/interspeech.2019-1413
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS): A dynamic, multimodal set of facial and vocal expressions in North American English

  2018influential reference
• MSP-IMPROV: An Acted Corpus of Dyadic Interactions to Study Emotion Perception

  2017influential reference
• Adam: A Method for Stochastic Optimization

  2014cited by this paper
• Learning Salient Features for Speech Emotion Recognition Using Convolutional Neural Networks

  2014cited by this paper
• Speech emotion recognition using deep neural network and extreme learning machine

  2014cited by this paper
• Evaluating the Impact of Syntax and Semantics on Emotion Recognition from Text

  2013cited by this paper
• Long Term Suboxone™ Emotional Reactivity As Measured by Automatic Detection in Speech

  2013cited by this paper
• Deep learning for robust feature generation in audiovisual emotion recognition

  2013cited by this paper
• The Wisdom of the Crowd in Combinatorial Problems

  2012cited by this paper
• Survey on speech emotion recognition: Features, classification schemes, and databases

  2011cited by this paper
• Emotion recognition using a hierarchical binary decision tree approach

  2011cited by this paper
• Analysis of Emotionally Salient Aspects of Fundamental Frequency for Emotion Detection

  2009cited by this paper
• IEMOCAP: interactive emotional dyadic motion capture database

  2008cited by this paper
• Statistical Parametric Speech Synthesis

  2007cited by this paper
• Speech Emotion Recognition Using Eigen-FFT in Clean and Noisy Environments

  2007cited by this paper
• Two-stream emotion recognition for call center monitoring

  2007cited by this paper
• Toward A Speaker-Independent Real-Time Affect Detection System

  2006cited by this paper
• Real vs. acted emotional speech

  2006cited by this paper
• ICARUS: Source generator based real-time recognition of speech in noisy stressful and Lombard effect environments

  1995cited by this paper
• Edinburgh Research Explorer Analysis of statistical parametric and unit selection speech synthesis systems applied to emotional speech

  year unknowncited by this paper

• AudioMarathon: A Comprehensive Benchmark for Long-Context Audio Understanding and Efficiency in Audio LLMs

  2025cites this paper
• Re-ENACT: Reinforcement Learning for Emotional Speech Generation using Actor-Critic Strategy

  2024influential citation
• Towards Generalizable SER: Soft Labeling and Data Augmentation for Modeling Temporal Emotion Shifts in Large-Scale Multilingual Speech

  2023cites this paper
• You're Not You When You're Angry: Robust Emotion Features Emerge by Recognizing Speakers

  2023cites this paper
• Adaptive Duration Modification of Speech using Masked Convolutional Networks and Open-Loop Time Warping

  2023cites this paper
• KBES: A dataset for realistic Bangla speech emotion recognition with intensity level

  2023cites this paper
• Emotions and virality: Social transmission of political messages on Twitter

  2022cites this paper
• Data in Brief

  2022cites this paper
• Speech emotion recognition for the Urdu language

  2022cites this paper
• A Diffeomorphic Flow-Based Variational Framework for Multi-Speaker Emotion Conversion

  2022cites this paper
• A Comparative Study of Data Augmentation Techniques for Deep Learning Based Emotion Recognition

  2022influential citation
• VocalTurk: Exploring Feasibility of Crowdsourced Speaker Identification

  2021cites this paper
• Contrasting the Emotions identified in Spanish TV debates and in Human-Machine Interactions

  2021cites this paper
• SEMOUR: A Scripted Emotional Speech Repository for Urdu

  2021cites this paper
• Learning Paralinguistic Features from Audiobooks through Style Voice Conversion

  2021cites this paper
• Emotional Voice Conversion: Theory, Databases and ESD

  2021cites this paper
• A Deep-Bayesian Framework for Adaptive Speech Duration Modification

  2021cites this paper
• GANtron: Emotional Speech Synthesis with Generative Adversarial Networks

  2021cites this paper
• Mapping Discrete Emotions in the Dimensional Space: An Acoustic Approach

  2021cites this paper
• Non-parallel Emotion Conversion using a Deep-Generative Hybrid Network and an Adversarial Pair Discriminator

  2020cites this paper
• Multi-speaker Emotion Conversion via Latent Variable Regularization and a Chained Encoder-Decoder-Predictor Network

  2020cites this paper
• End-to-end recurrent denoising autoencoder embeddings for speaker identification

  2020cites this paper
• A Multi-Speaker Emotion Morphing Model Using Highway Networks and Maximum Likelihood Objective

  2019influential citation
• Automated Emotion Morphing in Speech Based on Diffeomorphic Curve Registration and Highway Networks

  2019influential citation