Predicting human visuomotor behaviour in a driving task

The sequential deployment of gaze to regions of interest is an integral part of human visual function. Owing to its central importance, decades of research have focused on predicting gaze locations, but there has been relatively little formal attempt to predict the temporal aspects of gaze deployment in natural multi-tasking situations. We approach this problem by decomposing complex visual behaviour into individual task modules that require independent sources of visual information for control, in order to model human gaze deployment on different task-relevant objects. We introduce a softmax barrier model for gaze selection that uses two key elements: a priority parameter that represents task importance per module, and noise estimates that allow modules to represent uncertainty about the state of task-relevant visual information. Comparisons with human gaze data gathered in a virtual driving environment show that the model closely approximates human performance.

• Publication year

  2014

• Venue

  Philosophical Transactions of the Royal Society B: Biological Sciences

• Publication date

  2014-02-19

• Fields of study

  Medicine, Computer Science

• Identifiers
  DOI 10.1098/rstb.2013.0044PMID 24395971PMCID 3886332
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• What/Where to Look Next? Modeling Top-Down Visual Attention in Complex Interactive Environments

  2014cited by this paper
• A soft barrier model for predicting human visuomotor behavior in a driving task

  2013cited by this paper
• Gaze Allocation Analysis for a Visually Guided Manipulation Task

  2012cited by this paper
• The role of uncertainty and reward on eye movements in a virtual driving task.

  2012influential reference
• A modular reinforcement learning model for human visuomotor behavior in a driving task

  2011cited by this paper
• 2010 Special Issue: Detecting contingencies: An infomax approach

  2010cited by this paper
• Influence of encoding context on face recognition

  2010cited by this paper
• Saliency, attention, and visual search: an information theoretic approach.

  2009cited by this paper
• Looking and Acting: Vision and eye movements in natural behaviour

  2009cited by this paper
• The prominence of behavioural biases in eye guidance

  2009cited by this paper
• The Speed and Accuracy of a Simple Perceptual Decision: A Mathematical Primer.

  2007cited by this paper
• Where to look next? Eye movements reduce local uncertainty.

  2007cited by this paper
• Trade-offs between gaze and working memory use.

  2007cited by this paper
• Modeling embodied visual behaviors

  2007cited by this paper
• Bayesian brain : probabilistic approaches to neural coding

  2006cited by this paper
• Optimal eye movement strategies in visual search

  2005cited by this paper
• Eye Movements for Reward Maximization

  2003cited by this paper
• A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking

  2002cited by this paper
• Computational modelling of visual attention

  2001cited by this paper
• Reinforcement Learning: An Introduction

  1998cited by this paper
• The Roles of Vision and Eye Movements in the Control of Activities of Daily Living

  1998cited by this paper
• On Information and Sufficiency

  1997cited by this paper
• Memory Representations in Natural Tasks

  1995cited by this paper
• Please Scroll down for Article Visual Cognition Sun: Top-down Saliency Using Natural Statistics

  year unknowncited by this paper

• Vision Control of a Vehicle Intended for Tourist Routes Designed for People with Special Needs

  2025cites this paper
• When knowing the activity is not enough to predict gaze

  2024cites this paper
• We steer where we stare

  2024cites this paper
• SCOUT+: Towards Practical Task-Driven Drivers’ Gaze Prediction

  2024cites this paper
• Evaluation of Visual Risk Perception of Automated Driving Tasks by Analyzing Gaze Pattern Dispersion

  2024cites this paper
• Attentional heterogeneity in social anxiety disorder: Evidence from Hidden Markov Models.

  2023cites this paper
• Effects of conventional and high-definition transcranial direct current stimulation (tDCS) on driving abilities: A tDCS-driving simulator study

  2023cites this paper
• How much situation awareness does the driver have when driving autonomously? A study based on driver attention allocation

  2023cites this paper
• Resolving uncertainty on the fly: modeling adaptive driving behavior as active inference

  2023cites this paper
• Modeling eye movement in dynamic interactive tasks for maximizing situation awareness based on Markov decision process

  2022cites this paper
• What are the visuo-motor tendencies of omnidirectional scene free-viewing in virtual reality?

  2022cites this paper
• Gaze Strategies in Driving–An Ecological Approach

  2022cites this paper
• Comparing Human and Machine Attention in Visuomotor Tasks

  2021cites this paper
• Dynamic scan paths investigations under manual and highly automated driving

  2021cites this paper
• Look-ahead fixations during visuomotor behavior: Evidence from assembling a camping tent

  2021cites this paper
• The Hierarchical Evolution in Human Vision Modeling

  2021cites this paper
• Inferring Goals with Gaze during Teleoperated Manipulation

  2021cites this paper
• Inattention and Uncertainty in the Predictive Brain

  2021cites this paper
• Design Strategy Network: A deep hierarchical framework to represent generative design strategies in complex action spaces

  2021cites this paper
• Inferring Intent and Action from Gaze in Naturalistic Behavior

  2020cites this paper
• Drivers use active gaze to monitor waypoints during automated driving

  2020cites this paper
• Eye Gaze for Assistive Manipulation

  2020cites this paper
• Machine versus Human Attention in Deep Reinforcement Learning Tasks

  2020cites this paper
• Visual attention for high speed driving

  2019cites this paper
• Attention Guided Imitation Learning and Reinforcement Learning

  2019cites this paper
• Improving Autonomous Driving Agents using Bio-Inspired Visual Attention

  2019cites this paper
• Gaze behavior and cognitive states during fingerprint target group localization

  2019cites this paper
• Sensitivity to the Focusing Significance of the Gaze Direction of the Partner at Mentally Retarded Preschoolers

  2019cites this paper
• Highly Automated Driving Impact on Drivers’ Gaze Behaviors during a Car-Following Task

  2019cites this paper
• Atari-HEAD: Atari Human Eye-Tracking and Demonstration Dataset

  2019cites this paper
• Periphery-Fovea Multi-Resolution Driving Model Guided by Human Attention

  2019cites this paper
• Atari-HEAD: Atari Human Eye-Tracking and Demonstration Dataset

  2019cites this paper
• Effects of an Active Visuomotor Steering Task on Covert Attention

  2019cites this paper
• Attention, Information-Seeking, and Active Sampling

  2019cites this paper
• AGIL: Learning Attention from Human for Visuomotor Tasks

  2018cites this paper
• Modeling the Human Visuo-Motor System to Support Remote-Control Operation

  2018cites this paper
• Gaze doesn't always lead steering.

  2018cites this paper
• A computational model for driver's cognitive state, visual perception and intermittent attention in a distracted car following task

  2018cites this paper
• Modeling sensory-motor decisions in natural behavior

  2018cites this paper
• Towards a neuroscience of active sampling and curiosity

  2018cites this paper
• Hand-Eye Coordination and Visual Attention in Infancy

  2018cites this paper
• Davida Teller Award Lecture 2017: What can be learned from natural behavior?

  2018cites this paper
• Recentering bias for temporal saccades only: Evidence from binocular recordings of eye movements.

  2018cites this paper
• Visuomotor Control, Eye Movements, and Steering: A Unified Approach for Incorporating Feedback, Feedforward, and Internal Models

  2018cites this paper
• Quaddles: A multidimensional 3-D object set with parametrically controlled and customizable features

  2017cites this paper
• Modeling Attention Division When Switching Between Tasks of Different Difficulty and Priority

  2017cites this paper
• Inferring Intent and Action from Gaze in Naturalistic Behavior: A Review

  2017cites this paper
• Parietal neurons encode expected gains in instrumental information

  2017cites this paper
• Differential Contribution of Low- and High-level Image Content to Eye Movements in Monkeys and Humans

  2017cites this paper
• Information s eeking and simulation : R oles of attention in guiding a goal-directed behavior

  2017cites this paper
• Information seeking and simulation: Roles of attention in guiding a goal-directed behavior

  2017cites this paper
• Systematic Observation of an Expert Driver's Gaze Strategy—An On-Road Case Study

  2017cites this paper
• Understanding active sampling strategies: Empirical approaches and implications for attention and decision research.

  2017cites this paper
• Perisaccadic Updating of Visual Representations and Attentional States: Linking Behavior and Neurophysiology

  2016cites this paper
• Optimizing gaze direction in a visual navigation task

  2016influential citation
• Predicting lane keeping behavior of visually distracted drivers using inverse suboptimal control

  2016influential citation
• Research on driving style recognition method based on driver’s dynamic demand

  2016cites this paper
• Modeling of Human Decision Making via Direct and Optimization-based Methods for Semi-Autonomous Systems

  2015cites this paper
• Modeling visual sampling on in-car displays: The challenge of predicting safety-critical lapses of control

  2015cites this paper
• Eye movements and hazard perception in active and passive driving

  2015cites this paper
• Investigation of Human First-Person Guidance Strategy from Gaze Tracking Data

  2015cites this paper
• Experimental Framework for Investigating First Person Guidance and Perception

  2015cites this paper
• Classification of Human Gaze in Spatial Guidance and Control

  2015cites this paper
• Risk-aversive optimal planning of sensing

  2015cites this paper
• The role of visual information in the steering behaviour of young and adult bicyclists

  2015cites this paper
• The Effect of Familiarity and Complexity of Environments, and Mode of Wayfinding on Gaze and Memory

  2015cites this paper
• Predicting lapses in attention: a study of brain oscillations, neural synchrony and eye measures

  2014cites this paper
• Seeing and doing: how vision shapes animal behaviour

  2014cites this paper
• Systematic Observation of an Expert Driver ' s Gaze þÿ �

  year unknowncites this paper