Deep Active Learning for Dialogue Generation

We propose an online, end-to-end, neural generative conversational model for open-domain dialogue. It is trained using a unique combination of offline two-phase supervised learning and online human-in-the-loop active learning. While most existing research proposes offline supervision or hand-crafted reward functions for online reinforcement, we devise a novel interactive learning mechanism based on hamming-diverse beam search for response generation and one-character user-feedback at each step. Experiments show that our model inherently promotes the generation of semantically relevant and interesting responses, and can be used to train agents with customized personas, moods and conversational styles.

• Publication year

  2016

• Venue

  International Workshop on Semantic Evaluation

• Publication date

  2016-12-12

• Fields of study

  Computer Science

• Identifiers
  DOI 10.18653/v1/S17-1008
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Investigation of Language Understanding Impact for Reinforcement Learning Based Dialogue Systems

  2017cited by this paper
• Agent-Agnostic Human-in-the-Loop Reinforcement Learning

  2017cited by this paper
• Key-Value Retrieval Networks for Task-Oriented Dialogue

  2017cited by this paper
• Composite Task-Completion Dialogue System via Hierarchical Deep Reinforcement Learning

  2017cited by this paper
• A Copy-Augmented Sequence-to-Sequence Architecture Gives Good Performance on Task-Oriented Dialogue

  2017cited by this paper
• End-to-End Task-Completion Neural Dialogue Systems

  2017cited by this paper
• A Conditional Variational Framework for Dialog Generation

  2017cited by this paper
• Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue

  2016cited by this paper
• Dialogue Learning With Human-In-The-Loop

  2016influential reference
• Dialog-based Language Learning

  2016cited by this paper
• Strategy and Policy Learning for Non-Task-Oriented Conversational Systems

  2016cited by this paper
• A Persona-Based Neural Conversation Model

  2016cited by this paper
• Deep Reinforcement Learning for Dialogue Generation

  2016influential reference
• Towards End-to-End Learning for Dialog State Tracking and Management using Deep Reinforcement Learning

  2016cited by this paper
• Multiresolution Recurrent Neural Networks: An Application to Dialogue Response Generation

  2016cited by this paper
• A Network-based End-to-End Trainable Task-oriented Dialogue System

  2016cited by this paper
• End-to-end LSTM-based dialog control optimized with supervised and reinforcement learning

  2016cited by this paper
• A Simple, Fast Diverse Decoding Algorithm for Neural Generation

  2016cited by this paper
• Deep Reinforcement Learning for Multi-Domain Dialogue Systems

  2016cited by this paper
• Continuously Learning Neural Dialogue Management

  2016cited by this paper
• Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Models

  2016cited by this paper
• Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models

  2015cited by this paper
• A Neural Conversational Model

  2015cited by this paper
• A Neural Network Approach to Context-Sensitive Generation of Conversational Responses

  2015cited by this paper
• Intelligent Affect: Rational Decision Making for Socially Aligned Agents

  2015cited by this paper
• A Diversity-Promoting Objective Function for Neural Conversation Models

  2015influential reference
• Neural Responding Machine for Short-Text Conversation

  2015cited by this paper
• Sequence to Sequence Learning with Neural Networks

  2014cited by this paper
• Adam: A Method for Stochastic Optimization

  2014cited by this paper
• Association for Computational Linguistics: Human Language Technologies

  2011cited by this paper
• Chameleons in Imagined Conversations: A New Approach to Understanding Coordination of Linguistic Style in Dialogs

  2011cited by this paper
• Statistical Phrase-Based Translation

  2003influential reference
• Conference

  1969cited by this paper

• A Methodology for Identifying Evaluation Items for Practical Dialogue Systems Based on Business-Dialogue System Alignment Models

  2026cites this paper
• A continual active learning approach to adapt neural networks to distribution shifts in quality monitoring applications

  2025cites this paper
• A Survey on the Recent Advancements in Human-Centered Dialog Systems

  2025cites this paper
• Enhancing Cost Efficiency in Active Learning with Candidate Set Query

  2025cites this paper
• Active Prompt Learning with Vision-Language Model Priors

  2024cites this paper
• Perturbation-based Active Learning for Question Answering

  2023cites this paper
• Active Learning for Semantic Segmentation with Multi-class Label Query

  2023cites this paper
• An reinforcement learning-based speech censorship chatbot system

  2022cites this paper
• Combating Label Distribution Shift for Active Domain Adaptation

  2022cites this paper
• A Reinforcement Learning-based Offensive semantics Censorship System for Chatbots

  2022cites this paper
• A Static and Dynamic Attention Framework for Multi Turn Dialogue Generation

  2022cites this paper
• Interactive Machine Learning for Image Captioning

  2022cites this paper
• TrustAL: Trustworthy Active Learning using Knowledge Distillation

  2022cites this paper
• Exploring the online learning response to predict students’ satisfaction

  2021cites this paper
• Uncertainty and Utility Sampling with Pre-Clustering

  2021cites this paper
• Towards Exploring the Limitations of Active Learning: An Empirical Study

  2021cites this paper
• SEQ2SEQ++: A Multitasking-Based Seq2seq Model to Generate Meaningful and Relevant Answers

  2021cites this paper
• Active Learning Strategy for COVID-19 Annotated Dataset

  2021cites this paper
• Local Explanation of Dialogue Response Generation

  2021cites this paper
• Combination of Natural Language Understanding and Reinforcement Learning for Booking Bot

  2021cites this paper
• Recent advances in deep learning based dialogue systems: a systematic survey

  2021influential citation
• Incremental Beam Manipulation for Natural Language Generation

  2021cites this paper
• Non-Autoregressive Neural Dialogue Generation

  2020cites this paper
• Human-machine dialogue modelling with the fusion of word- and sentence-level emotions

  2020cites this paper
• Would you Like to Talk about Sports Now?: Towards Contextual Topic Suggestion for Open-Domain Conversational Agents

  2020cites this paper
• Enhancements to the Sequence-to-Sequence-Based Natural Answer Generation Models

  2020cites this paper
• A Survey of Deep Active Learning

  2020influential citation
• Item Response Theory for Efficient Human Evaluation of Chatbots

  2020cites this paper
• AMUSED: A Multi-Stream Vector Representation Method for Use in Natural Dialogue

  2019cites this paper
• Towards Best Experiment Design for Evaluating Dialogue System Output

  2019cites this paper
• Emotion-Aware and Human-Like Autonomous Agents

  2019cites this paper
• Augmented Memory Networks for Streaming-Based Active One-Shot Learning

  2019cites this paper
• Active Learning for Domain Classification in a Commercial Spoken Personal Assistant

  2019cites this paper
• A Survey of Natural Language Generation Techniques with a Focus on Dialogue Systems - Past, Present and Future Directions

  2019influential citation
• Improving Neural Conversational Models with Entropy-Based Data Filtering

  2019cites this paper
• TION METHOD FOR USE IN NATURAL DIALOGUE

  2019cites this paper
• Diversifying Topic-Coherent Response Generation for Natural Multi-turn Conversations

  2019cites this paper
• Knowledge Graph Construction and Applications for Web Search and Beyond

  2019cites this paper
• Aiming to Know You Better Perhaps Makes Me a More Engaging Dialogue Partner

  2018cites this paper
• Context-Sensitive Generation of Open-Domain Conversational Responses

  2018cites this paper
• Dialogue Modeling Via Hash Functions

  2018cites this paper
• Conversational Model Adaptation via KL Divergence Regularization

  2018cites this paper
• Active learning and input space analysis for deep networks

  2018cites this paper
• What's to Know? Uncertainty as a Guide to Asking Goal-Oriented Questions

  2018cites this paper
• SHAPED: Shared-Private Encoder-Decoder for Text Style Adaptation

  2018cites this paper
• Diverse Beam Search for Increased Novelty in Abstractive Summarization

  2018cites this paper
• Adversarial Active Learning for Deep Networks: a Margin Based Approach

  2018cites this paper
• Modeling Psychotherapy Dialogues with Kernelized Hashcode Representations: A Nonparametric Information-Theoretic Approach.

  2018cites this paper
• Infogain-Driven Dialogue Modeling with Hashcode Representations

  2018cites this paper
• Utterance Censorship of Online Reinforcement Learning Chatbot

  2018cites this paper
• Explore-Exploit: A Framework for Interactive and Online Learning

  2018cites this paper
• A Neural Generation-based Conversation Model Using Fine-grained Emotion-guide Attention

  2018cites this paper
• Better Conversations by Modeling, Filtering, and Optimizing for Coherence and Diversity

  2018cites this paper
• Data Distillation for Controlling Specificity in Dialogue Generation

  2017cites this paper
• Adversarial Learning for Neural Dialogue Generation

  2017cites this paper
• Affective Neural Response Generation

  2017cites this paper
• A Survey on Dialogue Systems: Recent Advances and New Frontiers

  2017cites this paper