Analysis of Temporal Features for Interaction Quality Estimation

Many different approaches for estimating the Interaction Quality (IQ) of Spoken Dialogue Systems have been investigated. While dialogues clearly have a sequential nature, statistical classification approaches designed for sequential problems do not seem to work better on automatic IQ estimation than static approaches, i.e., regarding each turn as being independent of the corresponding dialogue. Hence, we analyse this effect by investigating the subset of temporal features used as input for statistical classification of IQ. We extend the set of temporal features to contain the system and the user view. We determine the contribution of each feature sub-group showing that temporal features contribute most to the classification performance. Furthermore, for the feature sub-group modeling the temporal effects with a window, we modify the window size increasing the overall performance significantly by +15.69%.

• Publication year

  2016

• Venue

  International Workshop on Spoken Dialogue Systems Technology

• Publication date

  2016-04-07

• Fields of study

  Computer Science

• Identifiers
  DOI 10.1007/978-981-10-2585-3_30arXiv 1604.01985
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Analysis of an Extended Interaction Quality Corpus

  2015cited by this paper
• User-centred adaptive spoken dialogue modelling

  2015cited by this paper
• Interaction Quality: Assessing the quality of ongoing spoken dialog interaction by experts - And how it relates to user satisfaction

  2015cited by this paper
• The proof and measurement of association between two things.

  2015cited by this paper
• First Insight into Quality-Adaptive Dialogue

  2014cited by this paper
• Application and Evaluation of a Conditioned Hidden Markov Model for Estimating Interaction Quality of Spoken Dialogue Systems

  2014cited by this paper
• Interaction Quality Estimation in Spoken Dialogue Systems Using Hybrid-HMMs

  2014influential reference
• Dialogue Management for User-Centered Adaptive Dialogue

  2014cited by this paper
• Improving Interaction Quality Recognition Using Error Correction

  2013cited by this paper
• On Quality Ratings for Spoken Dialogue Systems – Experts vs. Users

  2013cited by this paper
• A Parameterized and Annotated Spoken Dialog Corpus of the CMU Let’s Go Bus Information System

  2012influential reference
• Towards Quality-Adaptive Spoken Dialogue Management

  2012cited by this paper
• A Theoretical Framework for a User-Centered Spoken Dialog Manager

  2011cited by this paper
• Modeling and Predicting Quality in Spoken Human-Computer Interaction

  2011cited by this paper
• A statistical approach for estimating user satisfaction in spoken human-machine interaction

  2011cited by this paper
• Modeling User Satisfaction Transitions in Dialogues from Overall Ratings

  2010cited by this paper
• Issues in Predicting User Satisfaction Transitions in Dialogues: Individual Differences, Evaluation Criteria, and Prediction Models

  2010cited by this paper
• Estimation Method of User Satisfaction Using N-gram-based Dialog History Model for Spoken Dialog System

  2010cited by this paper
• Spoken Dialogue Systems for Ambient Environments

  2010cited by this paper
• Modeling User Satisfaction with Hidden Markov Models

  2009influential reference
• Doing research on a deployed spoken dialogue system: one year of let's go! experience

  2006cited by this paper
• Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data

  2001cited by this paper
• The Nature of Statistical Learning Theory

  2000cited by this paper
• PARADISE: A Framework for Evaluating Spoken Dialogue Agents

  1997cited by this paper
• Long Short-Term Memory

  1997cited by this paper
• The measurement of observer agreement for categorical data.

  1977cited by this paper
• Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit.

  1968cited by this paper
• A Coefficient of Agreement for Nominal Scales

  1960cited by this paper
• Individual Comparisons by Ranking Methods

  1945cited by this paper

• Modeling Performance in Open-Domain Dialogue with PARADISE

  2021cites this paper
• Towards Continuous Estimation of Dissatisfaction in Spoken Dialog

  2021cites this paper
• Multimodal User Satisfaction Recognition for Non-task Oriented Dialogue Systems

  2021cites this paper
• Interaction Quality Estimation Using Long Short-Term Memories

  2017influential citation
• Significance of Interaction Parameter Levels in Interaction Quality Modelling for Human-Human Conversation

  2017cites this paper
• Analysis of Interaction Parameter Levels in Interaction Quality Modelling for Human-Human Conversation

  2017cites this paper
• 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue

  2014cites this paper