Conversational In-Vehicle Dialog Systems: The past, present, and future

Automotive technology rapidly advances with increasing connectivity and automation. These advancements aim to assist safe driving and improve user travel experience. Before the realization of a full automation, in-vehicle dialog systems may reduce the driver distraction from many services available through connectivity. Even when a full automation is realized, in-vehicle dialog systems still play a special role in assisting vehicle occupants to perform various tasks. On the other hand, in-vehicle use cases need to address very different user conditions, environments, and industry requirements than other uses. This makes the development of effective and efficient in-vehicle dialog systems challenging; it requires multidisciplinary expertise in automatic speech recognition, spoken language understanding, dialog management (DM), natural language generation, and application management, as well as field system and safety testing. In this article, we review research and development (R&D) activities for in-vehicle dialog systems from both academic and industrial perspectives, examine findings, discuss key challenges, and share our visions for voice-enabled interaction and intelligent assistance for smart vehicles over the next decade.

• Publication year

  2016

• Venue

  IEEE Signal Processing Magazine

• Publication date

  2016-11-01

• Fields of study

  Computer Science, Engineering

• Identifiers
  DOI 10.1109/MSP.2016.2599201
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Improving teenage driver perceptions regarding the impact of distracted driving in the Pacific Northwest

  2016cited by this paper
• On using heterogeneous data for vehicle-based speech recognition: A DNN-based approach

  2015cited by this paper
• Deep Learning

  2015cited by this paper
• Lexicon-Free Conversational Speech Recognition with Neural Networks

  2015cited by this paper
• Computationally-efficient endpointing features for natural spoken interaction with personal-assistant systems

  2014cited by this paper
• Deep Speech: Scaling up end-to-end speech recognition

  2014cited by this paper
• An investigation of deep neural networks for noise robust speech recognition

  2013cited by this paper
• Online whole-word and stroke-based modeling for hand-written letter recognition in in-car environments

  2013cited by this paper
• Deep Neural Networks for Acoustic Modeling in Speech Recognition

  2012cited by this paper
• International Large-Scale Vehicle Corpora for Research on Driver Behavior on the Road

  2011cited by this paper
• Multimodal Input in the Car, Today and Tomorrow

  2011cited by this paper
• A Progressive Feature Selection Algorithm for Ultra Large Feature Spaces

  2006cited by this paper
• CHAT: a conversational helper for automotive tasks

  2006cited by this paper
• Combining missing-feature theory, speech enhancement, and speaker-dependent/-independent modeling for speech separation

  2006cited by this paper
• D4. 1: Integration of Learning and Adaptivity with the ISU approach

  2005cited by this paper
• Construction and Analysis of a Multi-Layered In-car Spoken Dialogue Corpus

  2005cited by this paper
• A Wizard of Oz Framework for Collecting Spoken Human Computer Dialogs: An Experiment Procedure for the Design and Testing of Natural Language In-Vehicle Technology Systems

  2005cited by this paper
• CSA-BF: a constrained switched adaptive beamformer for speech enhancement and recognition in real car environments

  2003cited by this paper
• Design of the VICO Spoken Dialogue System: Evaluation of User Expectations by Wizard-of-Oz Experiments

  2002cited by this paper
• Is the speaker done yet? faster and more accurate end-of-utterance detection using prosody

  2002cited by this paper
• "CU-move" : analysis & corpus development for interactive in-vehicle speech systems

  2001cited by this paper
• To ‘errrr’ is human: ecology and acoustics of speech disfluencies

  2001cited by this paper
• "CU-move": robust speech processing for in-vehicle speech systems

  2000cited by this paper
• Prosodic features which cue back-channel responses in English and Japanese

  2000cited by this paper
• The AT&t-DARPA communicator mixed-initiative spoken dialog system

  2000cited by this paper
• Robustness to telephone handset distortion in speaker recognition by discriminative feature design

  2000cited by this paper
• Hands-free human-machine dialogue - corpora, technology and evaluation

  2000cited by this paper
• Adaptive microphone array employing calibration signals: an analytical evaluation

  1999cited by this paper
• The speechdat-car multilingual speech databases for in-car applications: some first validation results

  1999cited by this paper
• NONLINEAR DISCRIMINANT FEATURE EXTRACTION FOR ROBUST TEXT-INDEPENDENT SPEAKER RECOGNITION

  1997cited by this paper
• Continuous speech recognition

  1995cited by this paper
• Managing problems in speaking

  1994cited by this paper
• This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. 1 The CALO Meeting Assistant System

  year unknowncited by this paper

• ProVoice: Designing Proactive Functionality for In-Vehicle Conversational Assistants using Multi-Objective Bayesian Optimization to Enhance Driver Experience

  2026cites this paper
• Looking and Listening Inside and Outside: Multimodal Artificial Intelligence Systems for Driver Safety Assessment and Intelligent Vehicle Decision-Making

  2026cites this paper
• Speech Recognition of High Impact Model Using Deep Learning Technique: A Review

  2025cites this paper
• What type of intelligent cockpit can help more in traffic accident scenarios? Considering the effects of gender and anthropomorphism in voice interface, and their interaction with text length

  2025cites this paper
• In Car Speech Detection with mmWave Radars

  2025cites this paper
• CabinSep: IR-Augmented Mask-Based MVDR for Real-Time In-Car Speech Separation with Distributed Heterogeneous Arrays

  2025cites this paper
• ChatGPT on the Road: Leveraging Large Language Model-Powered In-vehicle Conversational Agents for Safer and More Enjoyable Driving Experience

  2025influential citation
• IDAS: Intelligent Driving Assistance System Using RAG

  2024cites this paper
• Fine-Grained Satisfaction Analysis of In-Vehicle Infotainment Systems Using Improved Kano Model and Cumulative Prospect Theory

  2024cites this paper
• DriVLMe: Enhancing LLM-based Autonomous Driving Agents with Embodied and Social Experiences

  2024cites this paper
• Zoneformer: On-device Neural Beamformer For In-car Multi-zone Speech Separation, Enhancement and Echo Cancellation

  2023cites this paper
• Voice orientation of conversational interfaces in vehicles

  2023cites this paper
• Embodying a Virtual Agent in a Self-Driving Car

  2023cites this paper
• AutoVis: Enabling Mixed-Immersive Analysis of Automotive User Interface Interaction Studies

  2023cites this paper
• Deep Neural Mel-Subband Beamformer for in-Car Speech Separation

  2022cites this paper
• DOROTHIE: Spoken Dialogue for Handling Unexpected Situations in Interactive Autonomous Driving Agents

  2022cites this paper
• A Novel Human-Vehicle Interaction Assistive Device for Arab Drivers Using Speech Recognition

  2022cites this paper
• Detecting Persian speaker-independent voice commands based on LSTM and ontology in communicating with the smart home appliances

  2022cites this paper
• Advanced Data Expoitation in Speech Analysis

  2021cites this paper
• Empathy consideration in the design of natural language interfaces for future vehicles

  2021cites this paper
• Voicing Concerns: User-Specific Pitfalls of Favoring Voice over Text in Conversational Recommender Systems (Short paper)

  2021cites this paper
• Vision-Cloud Data Fusion for ADAS: A Lane Change Prediction Case Study

  2021cites this paper
• Performance Improvement of Adaptive Wavelet Thresholding for Speech Enhancement Using Generalized Gaussian Priors and Frame-Wise Context Modeling

  2020cites this paper
• Risk, Trust, and Bias: Causal Regulators of Biometric-Enabled Decision Support

  2020cites this paper
• Smart Automotive Mobility: Reliable Technology for the Mobile Human

  2020cites this paper
• Adapting In-Vehicle Voice Output:A User- and Situation-Adaptive Approach

  2020cites this paper
• KoFFI—The New Driving Experience: How to Cooperate with Automated Driving Vehicles

  2020cites this paper
• Study on Consumer Requirements for Automotive Infotainment Systems

  2019cites this paper
• A QoE Test System for Vehicular Voice Cloud Services

  2019cites this paper
• Ensemble-Based Deep Reinforcement Learning for Chatbots

  2019cites this paper
• A Dialogue manager for task-oriented agents based on dialogue building-blocks and generic cognitive processing

  2019cites this paper
• Internet of Speech: A Conceptual Model

  2018cites this paper
• Adasa: A Conversational In-Vehicle Digital Assistant for Advanced Driver Assistance Features

  2018cites this paper
• Enhancing Trust in Autonomous Vehicles through Intelligent User Interfaces That Mimic Human Behavior

  2018cites this paper
• An Advanced Driver Risk Measurement System for Usage-Based Insurance on Big Driving Data

  2018cites this paper
• Design and Evaluation of a Real-Time Speech Recognition System

  2018cites this paper
• The impact of information systems on stress and behavioral outcomes

  2018influential citation
• Intent detection and semantic parsing for navigation dialogue language processing

  2017cites this paper
• Advanced Data Exploitation in Speech Analysis: An overview

  2017cites this paper
• Neural Network Configurations Analysis for Multilevel Speech Pattern Recognition System with Mixture of Experts

  2017cites this paper
• Object Referring in Visual Scene with Spoken Language

  2017cites this paper
• Driver Modeling for Detection and Assessment of Driver Distraction: Examples from the UTDrive Test Bed

  2017cites this paper
• Driver Modeling for Detection & Assessment of Distraction : Examples from the UTDrive testbed

  2017cites this paper
• Natural Language Processing, Moving from Rules to Data

  2017cites this paper
• Ethical Challenges in Data-Driven Dialogue Systems

  2017cites this paper
• An Inference Method for Personalized Automotive Service Based on Rough Set and Evidential Reasoning

  2017cites this paper
• Investigating a two-way-audio query-response command interface with navigation data extraction for driver assistance

  2017cites this paper
• Neural Network Configurations Analysis for Identification of Speech Pattern with Low Order Parameters

  2016cites this paper
• Evaluating Virtual Reality 360 Video to Inform Interface Design for Driverless Taxis

  year unknowncites this paper