Optimizing Dialogue Management with Reinforcement Learning: Experiments with the NJFun System

Designing the dialogue policy of a spoken dialogue system involves many nontrivial choices. This paper presents a reinforcement learning approach for automatically optimizing a dialogue policy, which addresses the technical challenges in applying reinforcement learning to a working dialogue system with human users. We report on the design, construction and empirical evaluation of NJFun, an experimental spoken dialogue system that provides users with access to information about fun things to do in New Jersey. Our results show that by optimizing its performance via reinforcement learning, NJFun measurably improves system performance.

• Publication year

  2011

• Venue

  Journal of Artificial Intelligence Research

• Publication date

  2011-06-03

• Fields of study

  Computer Science

• Identifiers
  DOI 10.1613/jair.859arXiv 1106.0676
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Computational Models of Mixed-Initiative Interaction

  2007cited by this paper
• Submission Category : Applications , Preference : ORAL Reinforcement Learning for Spoken Dialogue Systems

  2007cited by this paper
• SPoT: A Trainable Sentence Planner

  2001cited by this paper
• Spoken Dialog Management for Robots

  2000cited by this paper
• Predicting and Adapting to Poor Speech Recognition in a Spoken Dialogue System

  2000cited by this paper
• A stochastic model of human-machine interaction for learning dialog strategies

  2000cited by this paper
• An Application of Reinforcement Learning to Dialogue Strategy Selection in a Spoken Dialogue System for Email

  2000cited by this paper
• SPOKEN LANGUAGE DIALOGUE : FROM THEORY TO PRACTICE

  1999cited by this paper
• Automatic Detection of Poor Speech Recognition at the Dialogue Level

  1999cited by this paper
• From novice to expert: the effect of tutorials on user expertise with spoken dialogue systems

  1998cited by this paper
• Evaluating spoken dialogue agents with PARADISE: Two case studies

  1998cited by this paper
• Evaluating Response Strategies in a Web-Based Spoken Dialogue Agent

  1998cited by this paper
• Learning Optimal Dialogue Strategies: A Case Study of a Spoken Dialogue Agent for Email

  1998cited by this paper
• An evaluation of strategies for selectively verifying utterance meanings in spoken natural language dialog

  1998cited by this paper
• Evaluation of the Dutch train timetable information system developed in the ARISE project

  1998cited by this paper
• Introduction to Reinforcement Learning

  1998cited by this paper
• Learning Optimal Dialogue Strategies: A Case Study of a Spoken Dialogue Agent for Email

  1998cited by this paper
• Tracking Initiative in Collaborative Dialogue Interactions

  1997cited by this paper
• Metrics for Evaluating Dialogue Strategies in a Spoken Language System

  1996influential reference
• Reinforcement Learning: A Survey

  1996cited by this paper
• A dialog control strategy based on the reliability of speech recognition

  1996cited by this paper
• Neuro-dynamic programming

  1996cited by this paper
• CHRONUS, The next generation

  1995cited by this paper
• Improving Elevator Performance Using Reinforcement Learning

  1995cited by this paper
• Temporal Difference Learning and TD-Gammon

  1995cited by this paper
• Voice communication between humans and machines

  1994cited by this paper
• User Interfaces for voice applications

  1994cited by this paper
• Intelligent dialogues in automated telephone services

  1992cited by this paper
• Mixed Initiative in Dialogue: An Investigation into Discourse Segmentation

  1990cited by this paper

• Best Policy Learning from Trajectory Preference Feedback

  2025cites this paper
• Permanent Magnet Synchronous Motor Control Based on the TD3 Reinforcement Learning Algorithm

  2025cites this paper
• Exploring the conversational landscape: A survey of artificial intelligence-based chatbot development

  2025cites this paper
• Reviewing chatbot algorithms: methods for intelligent dialogue systems

  2025cites this paper
• Negotiation games with structured post-hoc intents

  2025cites this paper
• Digital Revolution and Management Control Practices: The Impact of Digitalization in Moroccan Private Companies

  2024cites this paper
• Dynamic Reward Adjustment in Multi-Reward Reinforcement Learning for Counselor Reflection Generation

  2024cites this paper
• Reinforcement learning-based autonomous attacker to uncover computer network vulnerabilities

  2024cites this paper
• A review on the role of various machine learning algorithms in microwave-assisted pyrolysis of lignocellulosic biomass waste.

  2024cites this paper
• Deep reinforcement learning challenges and opportunities for urban water systems.

  2024cites this paper
• Conversational Ai: A Treatise About Vying Chatbots

  2024cites this paper
• Chatbots & Dialogue Systems

  2023cites this paper
• Offline Reinforcement Learning for Mixture-of-Expert Dialogue Management

  2023cites this paper
• “What do others think?”: Task-Oriented Conversational Modeling with Subjective Knowledge

  2023cites this paper
• Multi-Agent Reinforcement Learning for Highway Platooning

  2023cites this paper
• Reinforcement learning algorithms: A brief survey

  2023cites this paper
• A Review of AI-Driven Conversational Chatbots Implementation Methodologies and Challenges (1999–2022)

  2023influential citation
• Development of a Trust-Aware User Simulator for Statistical Proactive Dialog Modeling in Human-AI Teams

  2023cites this paper
• Workflow-Guided Response Generation for Task-Oriented Dialogue

  2023cites this paper
• Multi-Agent Reinforcement Learning for Traffic Signal Control: A Cooperative Approach

  2023cites this paper
• Can Humans Be out of the Loop?

  2022cites this paper
• E2HRL: An Energy-efficient Hardware Accelerator for Hierarchical Deep Reinforcement Learning

  2022cites this paper
• Ethical conversational agent for performing in-game user experience questionnaires

  2022cites this paper
• On learning history based policies for controlling Markov decision processes

  2022cites this paper
• Context-Aware Language Modeling for Goal-Oriented Dialogue Systems

  2022cites this paper
• Learning Natural Language Generation with Truncated Reinforcement Learning

  2022cites this paper
• Experience Replay-based Deep Reinforcement Learning for Dialogue Management Optimisation

  2022cites this paper
• Efficient Dialog Policy Learning by Reasoning with Contextual Knowledge

  2022cites this paper
• A Survey on Recent Advances and Challenges in Reinforcement Learning Methods for Task-oriented Dialogue Policy Learning

  2022cites this paper
• A Mixture-of-Expert Approach to RL-based Dialogue Management

  2022cites this paper
• Dynamic Planning in Open-Ended Dialogue using Reinforcement Learning

  2022cites this paper
• Dynamic Policy Programming with Descending Regularization for Efficient Reinforcement Learning Control

  2022cites this paper
• Dialogue Management in Conversational Systems: A Review of Approaches, Challenges, and Opportunities

  2022cites this paper
• Simheuristics Approaches for Efficient Decision-Making Support in Materials Trading Networks

  2021cites this paper
• Learning Natural Language Generation from Scratch

  2021cites this paper
• Predictable and Adaptive Goal-oriented Dialog Policy Generation

  2021cites this paper
• Recent advances in deep learning based dialogue systems: a systematic survey

  2021cites this paper
• User Satisfaction Reward Estimation Across Domains: Domain-independent Dialogue Policy Learning

  2021influential citation
• Survey on reinforcement learning for language processing

  2021cites this paper
• Maximum Entropy Dueling Network Architecture

  2021cites this paper
• Shiftable Dynamic Policy Programming for Efficient and Robust Reinforcement Learning Control

  2021cites this paper
• A survey on deep reinforcement learning for audio-based applications

  2021cites this paper
• Buffer evaluation model and scheduling strategy for video streaming services in 5G-powered drone using machine learning

  2021cites this paper
• Weighted User Goal Sampling for Dialog Policy Learning

  2021cites this paper
• A Survey on Adversarial Attacks for Malware Analysis

  2021cites this paper
• Generating Predictable and Adaptive Dialog Policies in Single- and Multi-domain Goal-oriented Dialog Systems

  2021cites this paper
• Dueling RL: Reinforcement Learning with Trajectory Preferences

  2021cites this paper
• One Arm to Rule Them All: Online Learning with Multi-armed Bandits for Low-Resource Conversational Agents

  2021cites this paper
• Resource‐Aware Intrusion Response Based on Deep Reinforcement Learning for Software‐Defined Internet‐of‐Battle‐Things

  2021cites this paper
• An empirical assessment of deep learning approaches to task-oriented dialog management

  2021cites this paper
• An Exploration-driven Reinforcement Learning Model for Video Streaming Scheduling in 5G-Powered Drone

  2021cites this paper
• Towards versatile conversations with data-driven dialog management and its integration in commercial platforms

  2021cites this paper
• An Energy Efficient EdgeAI Autoencoder Accelerator for Reinforcement Learning

  2021cites this paper
• Document-editing Assistants and Model-based Reinforcement Learning as a Path to Conversational AI

  2020cites this paper
• Beyond Domain APIs: Task-oriented Conversational Modeling with Unstructured Knowledge Access

  2020cites this paper
• Deep Reinforcement Learning with Pre-training for Time-efficient Training of Automatic Speech Recognition

  2020cites this paper
• B&Anet: Combining bidirectional LSTM and self-attention for end-to-end learning of task-oriented dialogue system

  2020cites this paper
• A Reinforcement Learning and IoT based System to Assist Patients with Disabilities

  2020cites this paper
• Transforming Robot Programs Based on Social Context

  2020cites this paper
• Building A User-Centric and Content-Driven Socialbot

  2020cites this paper
• Deep reinforcement learning for permanent magnet synchronous motor speed control systems

  2020cites this paper
• Data-Efficient Methods for Dialogue Systems

  2020cites this paper
• Investigating the Effects of Double Deep Q-Learning in Video Game Environment

  2020cites this paper
• Conversational AI: Dialogue Systems, Conversational Agents, and Chatbots

  2020cites this paper
• Strategic Exploration in Reinforcement Learning - New Algorithms and Learning Guarantees

  2020cites this paper
• A Conversational Movie Recommender System

  2020cites this paper
• Differential variable speed limits control for freeway recurrent bottlenecks via deep actor-critic algorithm

  2020cites this paper
• Task-Completion Dialogue Policy Learning via Monte Carlo Tree Search with Dueling Network

  2020influential citation
• Autonomous Visual Navigation using Deep Reinforcement Learning: An Overview

  2019cites this paper
• Neural Assistant: Joint Action Prediction, Response Generation, and Latent Knowledge Reasoning

  2019cites this paper
• Investigation of Error Simulation Techniques for Learning Dialog Policies for Conversational Error Recovery

  2019cites this paper
• Towards Personalized Dialog Policies for Conversational Skill Discovery

  2019cites this paper
• Cas-GANs: An Approach of Dialogue Policy Learning based on GAN and RL Techniques

  2019cites this paper
• Clustering subspace generalization to obtain faster reinforcement learning

  2019cites this paper
• Domain Transfer in Dialogue Systems without Turn-Level Supervision

  2019cites this paper
• Influence of Time and Risk on Response Acceptability in a Simple Spoken Dialogue System

  2019cites this paper
• Augmenting Knowledge through Statistical, Goal-oriented Human-Robot Dialog

  2019cites this paper
• Discussing with a computer to practice a foreign language: research synthesis and conceptual framework of dialogue-based CALL

  2019cites this paper
• Pre-training in Deep Reinforcement Learning for Automatic Speech Recognition

  2019cites this paper
• I love your chain mail! Making knights smile in a fantasy game world: Open-domain goal-oriented dialogue agents

  2019cites this paper
• Ambient Assisted Living with Deep Learning

  2019cites this paper
• Uncertainty-based Decision Making Using Deep Reinforcement Learning

  2019cites this paper
• Planning for a Corpus of Continuous Ratings of Spoken Dialog Quality

  2019cites this paper
• A Survey of Reinforcement Learning Informed by Natural Language

  2019cites this paper
• MAKING KNIGHTS SMILE IN A FANTASY GAME WORLD

  2019cites this paper
• A Learning Automata based Solution for Optimizing Dialogue Strategy in Spoken Dialogue System

  2019cites this paper
• A Reinforcement Learning Based Intelligent System for the Healthcare Treatment Assistance of Patients with Disabilities

  2019cites this paper
• Les systèmes de dialogue orientés-but : état de l’art et perspectives d’amélioration (Goal-oriented dialog systems : a recent overview and research prospects )

  2019cites this paper
• A Learning Automata based Solution for Optimizing Dialogue Strategy in Spoken Dialogue System

  2019cites this paper
• A Learning Automata based Solution for Optimizing Dialogue Strategy in Spoken Dialogue System

  2019cites this paper
• Towards Integration of Cognitive Models in Dialogue Management: Designing the Virtual Negotiation Coach Application

  2019cites this paper
• A Learning Automata based Solution for Optimizing Dialogue Strategy in Spoken Dialogue System

  2019cites this paper
• Quantized Dialog - A general approach for conversational systems

  2019cites this paper
• Reinforcement learning for Dialogue Systems optimization with user adaptation. (Apprentissage par Renforcement pour l'optimisation des systèmes de dialogue via l'adaptation à l'utilisateur)

  2019cites this paper
• Importance Sampling to Identify Empirically Valid Policies and their Critical Decisions

  2019cites this paper
• Self-Improving Generative Adversarial Reinforcement Learning

  2019cites this paper
• Few-Shot Generalization Across Dialogue Tasks

  2018cites this paper
• Survey of biometric pattern recognition via machine learning techniques

  2018cites this paper
• Reinforcement Learning Based Speech Enhancement for Robust Speech Recognition

  2018cites this paper
• Multimodal Hierarchical Reinforcement Learning Policy for Task-Oriented Visual Dialog

  2018cites this paper