Strategy and Policy Learning for Non-Task-Oriented Conversational Systems

We propose a set of generic conversational strategies to handle possible sys-tem breakdowns in non-task-oriented dialog systems. We also design dialog policies to select among these strategies with respect to different dialog contexts. We combine expert knowledge and the statistical findings that derived from previous collected data in designing these policies. The dialog policy learned via reinforcement learning outperforms the random selection policy and the locally greedy policy in both the simulated and the real-world settings. In addition, we propose three metrics, which consider both the lo-cal and global quality of the conversation, to evaluate conversation quality.

• Publication year

  2016

• Venue

  SIGDIAL Conference

• Publication date

  2016-09-01

• Fields of study

  Computer Science

• Identifiers
  DOI 10.18653/v1/W16-3649
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Towards an Open-Domain Social Dialog System

  2016cited by this paper
• Chatbot Evaluation and Database Expansion via Crowdsourcing

  2016cited by this paper
• A Neural Conversational Model

  2015cited by this paper
• TickTock: A Non-Goal-Oriented Multimodal Dialog System with Engagement Awareness

  2015cited by this paper
• Towards an open-domain conversational system fully based on natural language processing

  2014cited by this paper
• VADER: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text

  2014cited by this paper
• Chat-Like Conversational System Based on Selection of Reply Generating Module with Reinforcement Learning

  2014cited by this paper
• Efficient Estimation of Word Representations in Vector Space

  2013cited by this paper
• Reinforcement learning for parameter estimation in statistical spoken dialogue systems

  2012cited by this paper
• IRIS: a Chat-oriented Dialogue System based on the Vector Space Model

  2012cited by this paper
• Reinforcement Learning of Question-Answering Dialogue Policies for Virtual Museum Guides

  2012cited by this paper
• POMDP-based Let's Go system for spoken dialog challenge

  2012cited by this paper
• Data-Driven Response Generation in Social Media

  2011cited by this paper
• LIBSVM: A library for support vector machines

  2011cited by this paper
• Empirically evaluating the application of reinforcement learning to the induction of effective and adaptive pedagogical strategies

  2011cited by this paper
• Reinforcement Learning of Argumentation Dialogue Policies in Negotiation

  2011cited by this paper
• Modeling Spoken Decision Making Dialogue and Optimization of its Dialogue Strategy

  2010cited by this paper
• Submission Category : Applications , Preference : ORAL Reinforcement Learning for Spoken Dialogue Systems

  2007cited by this paper
• Partially observable Markov decision processes for spoken dialog systems

  2007cited by this paper
• Learning Optimal Dialogue Strategies: A Case Study of a Spoken Dialogue Agent for Email

  1998cited by this paper
• Learning dialogue strategies within the Markov decision process framework

  1997cited by this paper

• Personality Dialogue Agent Based on Personality Description and Conversation History

  2025cites this paper
• Intelligent chatbot dialogue breakdown solutions and challenges: A systematic literature review

  2025cites this paper
• Multi-User Chat Assistant (MUCA): a Framework Using LLMs to Facilitate Group Conversations

  2024cites this paper
• Inductive-Deductive Strategy Reuse for Multi-Turn Instructional Dialogues

  2024cites this paper
• A Survey of Personality, Persona, and Profile in Conversational Agents and Chatbots

  2023cites this paper
• IRWoZ: Constructing an Industrial Robot Wizard-of-OZ Dialoguing Dataset

  2023cites this paper
• DECAF: A Modular and Extensible Conversational Search Framework

  2023cites this paper
• When to generate hedges in peer-tutoring interactions

  2023cites this paper
• Towards Hybrid Dialog Management Strategies for a Health Coach Chatbot

  2023cites this paper
• A speech-enabled virtual assistant for efficient human-robot interaction in industrial environments

  2023cites this paper
• Policy-driven Knowledge Selection and Response Generation for Document-grounded Dialogue

  2023cites this paper
• Context-Aware Language Modeling for Goal-Oriented Dialogue Systems

  2022cites this paper
• CHAI: A CHatbot AI for Task-Oriented Dialogue with Offline Reinforcement Learning

  2022cites this paper
• Learning towards conversational AI: A survey

  2022cites this paper
• ToD4IR: A Humanised Task-Oriented Dialogue System for Industrial Robots

  2022cites this paper
• Chatbot-mediated Learning: Conceptual Framework for the Design of Chatbot Use Cases in Education

  2022cites this paper
• Investigating the relationship between dialogue and exchange-level impression

  2022cites this paper
• PEACE: A Model of Key Social and Emotional Qualities of Conversational Chatbots

  2022cites this paper
• A Natural Language-enabled Virtual Assistant for Human-Robot Interaction in Industrial Environments

  2022cites this paper
• Double-meaning agreements by two robots to conceal incoherent agreements to user's opinions

  2021cites this paper
• Towards Automatic Evaluation of Dialog Systems: A Model-Free Off-Policy Evaluation Approach

  2021influential citation
• How can I help you? An Intelligent Virtual Assistant for Industrial Robots

  2021cites this paper
• Twitter-based selection of topics that users are interested in but are not familiar with

  2021cites this paper
• Key Qualities of Conversational Chatbots – the PEACE Model

  2021cites this paper
• Gathering Information and Engaging the User ComBot: A Task-Based, Serendipitous Dialog Model for Patient-Doctor Interactions

  2021cites this paper
• GTM: A Generative Triple-wise Model for Conversational Question Generation

  2021cites this paper
• Smart integration of sensors, computer vision and knowledge representation for intelligent monitoring and verbal human-computer interaction

  2021cites this paper
• Hierarchical Dependence-aware Evaluation Measures for Conversational Search

  2021cites this paper
• Discovering Dialog Structure Graph for Coherent Dialog Generation

  2021cites this paper
• One Arm to Rule Them All: Online Learning with Multi-armed Bandits for Low-Resource Conversational Agents

  2021cites this paper
• Coherent Dialog Generation with Query Graph

  2021cites this paper
• Persona Authentication through Generative Dialogue

  2021cites this paper
• Predicting Success of a Persuasion through Joint Modeling of Utterance Categorization

  2021cites this paper
• Building A User-Centric and Content-Driven Socialbot

  2020influential citation
• Accelerating Natural Language Understanding in Task-Oriented Dialog

  2020cites this paper
• ClarQ: A large-scale and diverse dataset for Clarification Question Generation

  2020cites this paper
• Learning parametric policies and transition probability models of markov decision processes from data

  2020cites this paper
• Adding Chit-Chat to Enhance Task-Oriented Dialogues

  2020cites this paper
• Improvements in the Utterance Database for Enhancing System Utterances in Chat-oriented Dialogue Systems

  2020cites this paper
• Data-Efficient Methods for Dialogue Systems

  2020influential citation
• An Evaluation of a Chat-oriented Dialogue System that Remembers and Uses User Information over Multiple Days

  2020cites this paper
• Selection of Evaluation Metrics for Dialogue Breakdown Detection in Dialogue Breakdown Detection Challenge 3

  2020cites this paper
• Emotional Dialogue Generation with Generative Adversarial Networks

  2020cites this paper
• Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Good

  2019cites this paper
• Survey on Intelligent Chatbots: State-of-the-Art and Future Research Directions

  2019cites this paper
• Overview of the sixth dialog system technology challenge: DSTC6

  2019cites this paper
• Learning in Text Streams: Discovery and Disambiguation of Entity and Relation Instances

  2019cites this paper
• Data fusion methods in multimodal human computer dialog

  2019cites this paper
• Answer-guided and Semantic Coherent Question Generation in Open-domain Conversation

  2019cites this paper
• A Repository of Conversational Datasets

  2019cites this paper
• Challenges in Building Intelligent Open-domain Dialog Systems

  2019cites this paper
• Time to Take Emoji Seriously: They Vastly Improve Casual Conversational Models

  2019cites this paper
• Exploiting Persona Information for Diverse Generation of Conversational Responses

  2019cites this paper
• Emotion-Aware and Human-Like Autonomous Agents

  2019cites this paper
• Towards Coherent and Engaging Spoken Dialog Response Generation Using Automatic Conversation Evaluators

  2019cites this paper
• SEMAPRO 2019 Proceedings

  2019cites this paper
• Knowledge-based Intelligence and Strategy Learning for Personalised Virtual Assistance in the Healthcare Domain

  2019influential citation
• Chat More If You Like: Dynamic Cue Words Planning to Flow Longer Conversations

  2018cites this paper
• Proposal of Implicit Confirmation toward Lexical Acquisition during Dialogues

  2018cites this paper
• 1 System Overview Natural Language Understanding ASR Dialog Manager Context Intent Classifier Feedback Topic Dialog Modules Natural Language Generation Segmentation Noun Phrase

  2018cites this paper
• A Deep Reinforcement Learning Chatbot (Short Version)

  2018cites this paper
• Learning to Ask Questions in Open-domain Conversational Systems with Typed Decoders

  2018cites this paper
• Natural language generation for commercial applications

  2018cites this paper
• Analyses of Example Sentences Collected by Conversation for Example-Based Non-Task-Oriented Dialog System

  2018cites this paper
• Development of a customizable open domain chat-oriented dialogue system

  2018cites this paper
• Evaluating dialogue breakdown detection in chat-oriented dialogue systems

  2018cites this paper
• Chunks in Multiparty Conversation - Building Blocks for Extended Social Talk

  2018cites this paper
• ChaD: Chat-Oriented Dialog Systems

  2018cites this paper
• Emotional Conversation Generation Orientated Syntactically Constrained Bidirectional-Asynchronous Framework

  2018cites this paper
• Dialogue Modeling Via Hash Functions

  2018cites this paper
• The Bottleneck Simulator: A Model-based Deep Reinforcement Learning Approach

  2018cites this paper
• Assigning Personality/Profile to a Chatting Machine for Coherent Conversation Generation

  2018cites this paper
• Generating Informative Responses with Controlled Sentence Function

  2018cites this paper
• An Analysis of the Effect of Emotional Speech Synthesis on Non-Task-Oriented Dialogue System

  2018cites this paper
• Challenges in Exploiting Conversational Memory in Human-Agent Interaction

  2018cites this paper
• Explorations in multiparty casual social talk and its relevance for social human machine dialogue

  2018cites this paper
• Refinement of Utterance Database and Concatenation of Utterances for Enhancing System Utterances in Chat-oriented Dialogue System

  2018cites this paper
• Neural Response Ranking for Social Conversation: A Data-Efficient Approach

  2018cites this paper
• Incorporating Relevant Knowledge in Context Modeling and Response Generation

  2018cites this paper
• Dialogue Generation: From Imitation Learning to Inverse Reinforcement Learning

  2018cites this paper
• Overview of Dialogue Breakdown Detection Challenge 3

  2017cites this paper
• Towards an Automatic Turing Test: Learning to Evaluate Dialogue Responses

  2017cites this paper
• Learning Conversational Systems that Interleave Task and Non-Task Content

  2017influential citation
• A review of evaluation techniques for social dialogue systems

  2017cites this paper
• Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders

  2017cites this paper
• Alana: Social Dialogue using an Ensemble Model and a Ranker trained on User Feedback

  2017influential citation
• A Deep Reinforcement Learning Chatbot

  2017cites this paper
• An Ensemble Model with Ranking for Social Dialogue

  2017influential citation
• Online Learning for Conversational Agents

  2017cites this paper
• Assigning personality/identity to a chatting machine for coherent conversation generation

  2017cites this paper
• Combining Search with Structured Data to Create a More Engaging User Experience in Open Domain Dialogue

  2017cites this paper
• Lexical Acquisition through Implicit Confirmations over Multiple Dialogues

  2017cites this paper
• Exploring Multiparty Casual Talk for Social Human-Machine Dialogue

  2017cites this paper
• The Octopus Approach to the Alexa Competition : A Deep Ensemble-based Socialbot

  2017cites this paper
• Adversarial Learning for Neural Dialogue Generation

  2017influential citation
• How Generic Can Dialogue Breakdown Detection Be? The KTH entry to DBDC3

  2017cites this paper
• Online Sequence-to-Sequence Reinforcement Learning for Open-Domain Conversational Agents

  2016cites this paper
• Online Sequence-to-Sequence Active Learning for Open-Domain Dialogue Generation

  2016cites this paper
• Generative Deep Neural Networks for Dialogue: A Short Review

  2016cites this paper
• Deep Active Learning for Dialogue Generation

  2016cites this paper