When do we interact multimodally?: cognitive load and multimodal communication patterns

No abstract is available for this paper.

• Publication year

  2004

• Venue

  International Conference on Multimodal Interaction

• Publication date

  2004-10-13

• Fields of study

  Computer Science, Psychology

• Identifiers
  DOI 10.1145/1027933.1027957
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Foreground and background interaction with sensor-enhanced mobile devices

  2005cited by this paper
• MULTIMODAL INTERFACES THAT Flex, Adapt, and Persist

  2004cited by this paper
• Isolating the effects of visual impairment: exploring the effect of AMD on the utility of multimodal feedback

  2004cited by this paper
• The Handbook of Multisensory Processing

  2004cited by this paper
• Toward a theory of organized multimodal integration patterns during human-computer interaction

  2003cited by this paper
• Adaptive interfaces and agents

  2002cited by this paper
• An analysis of audio-visual crossmodal integration by means of event-related potential (ERP) recordings.

  2002cited by this paper
• Handbook of Multimodal and Spoken Dialogue Systems: Resources, Terminology and Product Evaluation

  2001cited by this paper
• Recognizing Time Pressure and Cognitive Load on the Basis of Speech: An Experimental Study

  2001cited by this paper
• Multimodal system processing in mobile environments

  2000cited by this paper
• Audio-visual and multimodal speech-based systems

  2000cited by this paper
• Designing the User Interface for Multimodal Speech and Pen-Based Gesture Applications: State-of-the-Art Systems and Future Research Directions

  2000cited by this paper
• Mutual disambiguation of recognition errors in a multimodel architecture

  1999cited by this paper
• Ten myths of multimodal interaction

  1999cited by this paper
• Noun-phrase anaphora and focus : The informational load hypothesis

  1999cited by this paper
• A Split-Attention Effect in Multimedia Learning: Evidence for Dual Processing Systems in Working Memory

  1998cited by this paper
• Mulitmodal Interactive Maps: Designing for Human Performance

  1997influential reference
• A mechanism for multimodal presentation planning based on agent cooperation and negotiation

  1997cited by this paper
• When two sensory modes are better than one

  1997cited by this paper
• Reducing cognitive load by mixing auditory and visual presentation modes

  1995cited by this paper
• Predicting spoken disfluencies during human-computer interaction

  1995cited by this paper
• Cognitive Load Theory and the Format of Instruction

  1991cited by this paper
• Modality effects and the structure of short-term verbal memory

  1989cited by this paper
• Cognitive Load During Problem Solving: Effects on Learning

  1988cited by this paper
• Working memory

  1984cited by this paper
• Compatibility and Resource Competition between Modalities of Input, Central Processing, and Output

  1983cited by this paper
• Toward a taxonomy of given-new information

  1981cited by this paper
• Logic and conversation

  1975cited by this paper
• SPEECH INTELLIGIBILITY DERIV ASYNCHRONOUS PROCESSING OF AUDITORY -

  year unknowncited by this paper

• From Visual to Multimodal Programming: Designing an Interface to Externalize Decomposition Thinking for Novice Learners

  2026cites this paper
• A Comparative Study of Speech-and-Pointing and Disocclusion Mini-Map Techniques for Object Selection in Virtual Reality

  2025cites this paper
• Towards Deeper Understanding of Natural User Interactions in Virtual Reality Based Assembly Tasks

  2025cites this paper
• Investigating the Impact of Voice-only and Embodied Conversational Virtual Agents on Mixed Reality Puzzle Solving

  2025cites this paper
• MindSpace: Improving Relaxation Break and Performance Using a Device Exterting Tactile Sensation of Life-like Breathing Movement with Squeeze-like Deep Touch Pressure on Users' Upper Chest Area

  2025cites this paper
• Light My Way: Developing and Exploring a Multimodal Interface to Assist People With Visual Impairments to Exit Highly Automated Vehicles

  2025cites this paper
• Social Media Addiction and Adolescent Mental Health: The Mediating Role of Cognitive Interference in Digital Communication

  2025cites this paper
• Bloom: Designing for LLM-Augmented Behavior Change Interactions

  2025cites this paper
• SwarmChat: An LLM-Based, Context-Aware Multimodal Interaction System for Robotic Swarms

  2025cites this paper
• Debate with Images: Detecting Deceptive Behaviors in Multimodal Large Language Models

  2025cites this paper
• Understanding Latency Sensitivity in Thermal and Tactile Feedback for Multimodal Haptics in VR

  2025cites this paper
• Feasibility and Utility of Multimodal Micro Ecological Momentary Assessment on a Smartwatch

  2025cites this paper
• Large Language Model-assisted Speech and Pointing Benefits Multiple 3D Object Selection in Virtual Reality

  2024cites this paper
• Dimensional Design of Emotive Sounds for Robots

  2024cites this paper
• Integrating Representational Gestures into Automatically Generated Embodied Explanations and its Effects on Understanding and Interaction Quality

  2024cites this paper
• Virtual voices: Exploring individual differences in chat and verbal participation in virtual meetings

  2024cites this paper
• Immersive virtual reality for learning exoskeleton-like virtual walking: a feasibility study

  2024cites this paper
• ExpressEdit: Video Editing with Natural Language and Sketching

  2024cites this paper
• Make it or draw it? Investigating the communicative trade-offs between sketches and prototypes

  2023cites this paper
• Assessing the Value of Multimodal Interfaces: A Study on Human–Machine Interaction in Weld Inspection Workstations

  2023cites this paper
• A Qualitative Study on the Expectations and Concerns Around Voice and Gesture Interactions in Vehicles

  2023cites this paper
• Seeing eye to eye: trustworthy embodiment for task-based conversational agents

  2023cites this paper
• Autonomous is Not Enough: Designing Multisensory Mid-Air Gestures for Vehicle Interactions Among People with Visual Impairments

  2023cites this paper
• Cognitive and motor cortex activation during robot-assisted multi-sensory interactive motor rehabilitation training: An fNIRS based pilot study

  2023cites this paper
• Evaluation of a Multimodal Interaction System for Big Displays

  2023cites this paper
• A Human-Computer Collaborative Editing Tool for Conceptual Diagrams

  2023cites this paper
• The Impacts of Referent Display on Gesture and Speech Elicitation

  2022cites this paper
• The Effects of Multimodal Communication Classroom on the Learning Performance of EFL Learners from the Perspective of Translanguaging

  2022cites this paper
• A Case Study on the Rapid Development of Natural and Synergistic Multimodal Interfaces for XR Use-Cases

  2022cites this paper
• Effects of vibrotactile feedback on yoga practice

  2022cites this paper
• Understanding Acoustic Patterns of Human Teachers Demonstrating Manipulation Tasks to Robots

  2022cites this paper
• Use Ping Wisely: A Study of Team Communication and Performance under Lean Affordance

  2022cites this paper
• MCTK: a Multi-modal Conversational Troubleshooting Kit for supporting users in web applications

  2022cites this paper
• Does Using Voice Authentication in Multimodal Systems Correlate With Increased Speech Interaction During Non-critical Routine Tasks?

  2022cites this paper
• Reducing the Cognitive Load of Playing a Digital Tabletop Game with a Multimodal Interface

  2022cites this paper
• What Is Happening Behind The Wall?

  2022cites this paper
• Losing Its Touch: Understanding User Perception of Multimodal Interaction and Smart Assistance

  2022influential citation
• Multimodal Error Correction for Speech-to-Text in a Mobile Office Automated Vehicle: Results From a Remote Study

  2022cites this paper
• A Proposed Design of a Lecture Material to Reduce Learning Complexity

  2022cites this paper
• A framework for fidelity evaluation of immersive virtual reality systems

  2022cites this paper
• Comparison of Uni- and Multimodal Interfaces for Spatial Interaction

  2022cites this paper
• Integrating Eye- and Mouse-Tracking with Assistant Based Speech Recognition for Interaction at Controller Working Positions

  2021cites this paper
• Dependability and Safety: Two Clouds in the Blue Sky of Multimodal Interaction

  2021cites this paper
• Synthesising augmentation and automation

  2021cites this paper
• Improvement of Autonomous Vehicles Trust Through Synesthetic-Based Multimodal Interaction

  2021cites this paper
• I Know What You Know: What Hand Movements Reveal about Domain Expertise

  2021influential citation
• Multisensory Proximity and Transition Cues for Improving Target Awareness in Narrow Field of View Augmented Reality Displays

  2021cites this paper
• Exploring students' cognitive and affective states during problem solving through multimodal data: Lessons learned from a programming activity

  2021cites this paper
• A Concise Guide to Elicitation Methodology

  2021cites this paper
• Operational Feasibility Analysis of the Multimodal Controller Working Position “TriControl”

  2020cites this paper
• Transfer from action to perception: The effect of motor-perceptual enrichment

  2020cites this paper
• Singing Robots: How Embodiment Affects Emotional Responses to Non-Linguistic Utterances

  2020cites this paper
• Multimodality on the Road Towards Evidence-Based Cognitive Modelling of Human Interactions in Everyday Roadside Situations

  2020cites this paper
• Multimodal Coordination Measures to Understand Users and Tasks

  2020cites this paper
• Why Interoperability at Data Level Is Not Sufficient for Enabling pHealth?

  2020cites this paper
• Alpaca: AR Graphics Extensions for Web Applications

  2020cites this paper
• the psychophysics of workload: Why bother with subjective measures?

  2020influential citation
• Understanding Gesture and Speech Multimodal Interactions for Manipulation Tasks in Augmented Reality Using Unconstrained Elicitation

  2020cites this paper
• Finally on Par?! Multimodal and Unimodal Interaction for Open Creative Design Tasks in Virtual Reality

  2020influential citation
• Talking to a pedagogical agent in a smart TV: modality matching effect in human-TV interaction

  2019cites this paper
• Investigating the effects of modality switches on driver distraction and interaction efficiency in the car

  2019cites this paper
• Report of 2017 NSF Workshop on Multimedia Challenges, Opportunities and Research Roadmaps

  2019cites this paper
• Estimating cognitive load from speech gathered in a complex real-life training exercise

  2019cites this paper
• Trust and Cognitive Load During Human-Robot Interaction

  2019cites this paper
• Physical dimensions of children's touchscreen interactions: Lessons from five years of study on the MTAGIC project

  2019cites this paper
• Dynamic Adaptive Gesturing Predicts Domain Expertise in Mathematics

  2019cites this paper
• Choosing a Dialogue System's Modality in Order to Minimize User's Workload

  2018cites this paper
• Cognitive Load Changes during Music Listening and its Implication in Earcon Design in Public Environments: An fNIRS Study

  2018cites this paper
• The TESTMED Project Experience. Process-aware Enactment of Clinical Guidelines through Multimodal Interfaces

  2018cites this paper
• Multimodal and Context-Aware Interaction in Augmented Reality for Active Assistance

  2018cites this paper
• Orchestration Load Indicators and Patterns: In-the-Wild Studies Using Mobile Eye-Tracking

  2018cites this paper
• Designing Multi-modal Interaction – A Basic Operations Approach

  2018cites this paper
• Multimodal behavioral and physiological signals as indicators of cognitive load

  2018cites this paper
• Multimodal learning analytics: assessing learners' mental state during the process of learning

  2018cites this paper
• Rethinking the interactions between people and cars

  2018cites this paper
• Facilitating Spreadsheet Manipulation on Mobile Devices Leveraging Speech

  2018cites this paper
• A systematic approach for the integration of emotional context in interactive systems

  2018cites this paper
• Emotionally-Aware Multimodal Interfaces: Preliminary Work on a Generic Affective Modality

  2018cites this paper
• Symmetric Evaluation of Multimodal Human-Robot Interaction with Gaze and Standard Control

  2018cites this paper
• Head-Mounted Display-Based Virtual Reality Social Story as a Tool to Teach Social Skills to Children Diagnosed with Autism Spectrum Disorder

  2018cites this paper
• A Review of Intent Detection, Arbitration, and Communication Aspects of Shared Control for Physical Human-Robot Interaction

  2018cites this paper
• The effects of substitute multisensory feedback on task performance and the sense of presence in a virtual reality environment

  2018cites this paper
• Metrics to Evaluate Human Teaching Engagement From a Robot's Point of View

  2018cites this paper
• The State of Speech in HCI: Trends, Themes and Challenges

  2018cites this paper
• Consistent categorization of multimodal integration patterns during human–computer interaction

  2017cites this paper
• Perspectives on learning with multimodal technology

  2017cites this paper
• Effective quality factors of multimodal interaction in simple and complex tasks of using a smart television

  2017cites this paper
• MiNT: Multimodal Interaction for Modeling and Model Refactoring

  2017cites this paper
• The role of multisensory feedback in the objective and subjective evaluations of fidelity in virtual reality environments

  2017cites this paper
• Technical support in collaborative decision making: an explorative study

  2017influential citation
• A Scheme for Multimodal Component Recommendation

  2017cites this paper
• Interaction History in Adaptive Multimodal Interaction

  2017cites this paper
• Theoretical foundations of multimodal interfaces and systems

  2017cites this paper
• The Handbook of Multimodal-Multisensor Interfaces: Foundations, User Modeling, and Common Modality Combinations - Volume 1

  2017cites this paper
• Engineering Psychology and Cognitive Ergonomics: Cognition and Design

  2017influential citation
• Explorer Learning human multimodal dialogue strategies

  2017cites this paper
• Understanding and supporting modality choices

  2017cites this paper
• Natural, Multi-modal Interfaces for Unmanned Systems

  2017cites this paper
• Daily Living Skills Training in Virtual Reality to Help Children with Autism Spectrum Disorder in a Real Shopping Scenario

  2017cites this paper
• Multimodal speech and pen interfaces

  2017cites this paper