A Deep Reinforcement Learning Chatbot (Short Version)

We present MILABOT: a deep reinforcement learning chatbot developed by the Montreal Institute for Learning Algorithms (MILA) for the Amazon Alexa Prize competition. MILABOT is capable of conversing with humans on popular small talk topics through both speech and text. The system consists of an ensemble of natural language generation and retrieval models, including neural network and template-based models. By applying reinforcement learning to crowdsourced data and real-world user interactions, the system has been trained to select an appropriate response from the models in its ensemble. The system has been evaluated through A/B testing with real-world users, where it performed significantly better than other systems. The results highlight the potential of coupling ensemble systems with deep reinforcement learning as a fruitful path for developing real-world, open-domain conversational agents.

• Publication year

  2018

• Venue

  arXiv.org

• Publication date

  2018-01-20

• Fields of study

  Mathematics, Computer Science

• Identifiers
  arXiv 1801.06700
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Towards an Automatic Turing Test: Learning to Evaluate Dialogue Responses

  2017cited by this paper
• A Deep Reinforcement Learning Chatbot

  2017cited by this paper
• How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation

  2016cited by this paper
• Strategy and Policy Learning for Non-Task-Oriented Conversational Systems

  2016cited by this paper
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation

  2016cited by this paper
• HALEF: An Open-Source Standard-Compliant Telephony-Based Modular Spoken Dialog System: A Review and An Outlook

  2015cited by this paper
• Playing Atari with Deep Reinforcement Learning

  2013cited by this paper
• Building Watson: An Overview of the DeepQA Project

  2010cited by this paper
• Matrix Factorization Techniques for Recommender Systems

  2009cited by this paper
• Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning

  2004cited by this paper
• Off-Policy Temporal Difference Learning with Function Approximation

  2001cited by this paper
• Eligibility Traces for Off-Policy Policy Evaluation

  2000cited by this paper
• The Philips automatic train timetable information system

  1995cited by this paper
• Black box and glass box evaluation of the SUNDIAL system

  1993cited by this paper
• Dialogue Management for Telephone Information Systems

  1992cited by this paper
• Reinforcement Learning for Robots Using Neural Networks

  1992cited by this paper
• ELIZA—a computer program for the study of natural language communication between man and machine

  1966cited by this paper

• Optimizing Conversational Quality in Spoken Dialogue Systems with Reinforcement Learning from AI Feedback

  2026cites this paper
• Integrated Learning with AI Enabled Interactions with Customers by Fashion Retailers

  2025cites this paper
• Revolutionizing Campus Communication: NLP-Powered University Chatbots

  2024cites this paper
• Improving a sequence-to-sequence nlp model using a reinforcement learning policy algorithm

  2022cites this paper
• An Overview of Chatbot Structure and Source Algorithms

  2022cites this paper
• Intents Categorization for Chatbot Development Using Recurrent Neural Network (RNN) Learning

  2021influential citation
• Leveraging User Paraphrasing Behavior In Dialog Systems To Automatically Collect Annotations For Long-Tail Utterances

  2020cites this paper
• Automation of Admission Enquiry Process Through Chatbot—A Feedback-Enabled Learning System

  2020cites this paper
• A Qualitative Evaluation of Language Models on Automatic Question-Answering for COVID-19

  2020cites this paper
• Intelligent Agent to Negotiate on Goal Oriented Conversations

  2020cites this paper
• Ensemble-Based Deep Reinforcement Learning for Chatbots

  2019influential citation
• Deep Reinforcement Learning for Chatbots Using Clustered Actions and Human-Likeness Rewards

  2019cites this paper
• A Survey on Reinforcement Learning for Dialogue Systems

  2019cites this paper
• A Study on Dialogue Reward Prediction for Open-Ended Conversational Agents

  2018influential citation
• Sozial interagierende Roboter in der Pflege

  2018cites this paper