Multisensory Integration of Sounds and Vibrotactile Stimuli in Processing Streams for “What” and “Where”

The segregation between cortical pathways for the identification and localization of objects is thought of as a general organizational principle in the brain. Yet, little is known about the unimodal versus multimodal nature of these processing streams. The main purpose of the present study was to test whether the auditory and tactile dual pathways converged into specialized multisensory brain areas. We used functional magnetic resonance imaging (fMRI) to compare directly in the same subjects the brain activation related to localization and identification of comparable auditory and vibrotactile stimuli. Results indicate that the right inferior frontal gyrus (IFG) and both left and right insula were more activated during identification conditions than during localization in both touch and audition. The reverse dissociation was found for the left and right inferior parietal lobules (IPL), the left superior parietal lobule (SPL) and the right precuneus-SPL, which were all more activated during localization conditions in the two modalities. We propose that specialized areas in the right IFG and the left and right insula are multisensory operators for the processing of stimulus identity whereas parts of the left and right IPL and SPL are specialized for the processing of spatial attributes independently of sensory modality.

• Publication year

  2009

• Venue

  Journal of Neuroscience

• Publication date

  2009-09-02

• Fields of study

  Biology, Medicine, Psychology

• Identifiers
  DOI 10.1523/JNEUROSCI.0910-09.2009PMID 19726653PMCID PMC3343457
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• Neuroimaging studies of working memory: A meta-analysis

  2010cited by this paper
• An Auditory Region in the Primate Insular Cortex Responding Preferentially to Vocal Communication Sounds

  2009cited by this paper
• Maps and streams in the auditory cortex: nonhuman primates illuminate human speech processing

  2009cited by this paper
• The Contribution of the Inferior Parietal Lobe to Auditory Spatial Working Memory

  2008cited by this paper
• Visual and spatial working memory: from boxes to networks.

  2008cited by this paper
• Domain-dependent activation during spatial and nonspatial auditory working memory

  2008cited by this paper
• Modality and domain specific components in auditory and visual working memory tasks

  2008cited by this paper
• Common coding of auditory and visual spatial information in working memory.

  2008cited by this paper
• Auditory, somatosensory, and multisensory insular cortex in the rat.

  2008cited by this paper
• Effects of feature-selective attention on auditory pattern and location processing

  2008cited by this paper
• Processing of sound location in human cortex

  2008cited by this paper
• Visual modulation of neurons in auditory cortex.

  2008cited by this paper
• Attention and Sensory Interactions within the Occipital Cortex in the Early Blind: An fMRI Study

  2007cited by this paper
• Multisensory convergence in auditory cortex, I. Cortical connections of the caudal superior temporal plane in macaque monkeys

  2007cited by this paper
• Audiovisual Temporal Correspondence Modulates Human Multisensory Superior Temporal Sulcus Plus Primary Sensory Cortices

  2007cited by this paper
• Processing of location and pattern changes of natural sounds in the human auditory cortex

  2007cited by this paper
• Multisensory processing of naturalistic objects in motion: A high-density electrical mapping and source estimation study

  2007cited by this paper
• Activity and effective connectivity of parietal and occipital cortical regions during haptic shape perception.

  2007cited by this paper
• Multisensory convergence in auditory cortex, II. Thalamocortical connections of the caudal superior temporal plane

  2007influential reference
• Getting in touch: Segregated somatosensory what and where pathways in humans revealed by electrical neuroimaging

  2007cited by this paper
• Neural correlates of spatial working memory in humans: a functional magnetic resonance imaging study comparing visual and tactile processes.

  2006cited by this paper
• Multisensory Activation of the Intraparietal Area When Classifying Grating Orientation: A Functional Magnetic Resonance Imaging Study

  2006cited by this paper
• Auditory temporal processing deficits in patients with insular stroke

  2006cited by this paper
• Processing of auditory spatial cues in human cortex: an fMRI study.

  2006cited by this paper
• Top-down task effects overrule automatic multisensory responses to letter-sound pairs in auditory association cortex

  2006cited by this paper
• Evidence of vibrotactile input to human auditory cortex

  2006cited by this paper
• Reading embossed capital letters: An fMRI study in blind and sighted individuals

  2006cited by this paper
• Integration of touch and sound in auditory cortex.

  2005cited by this paper
• What vs. where in touch: an fMRI study

  2005influential reference
• Impaired Cross-Modal Inhibition in Alzheimer Disease

  2005cited by this paper
• Multisensory contributions to low-level, 'unisensory' processing.

  2005cited by this paper
• Tactile discrimination of grating orientation: fMRI activation patterns

  2005cited by this paper
• The Functional Organization of Auditory Working Memory as Revealed by fMRI

  2005cited by this paper
• Cross-modal activation of visual cortex during depth perception using auditory substitution of vision

  2005cited by this paper
• Visual cortical activity during tactile perception in the sighted and the visually deprived.

  2005influential reference
• Tactile form and location processing in the human brain.

  2005influential reference
• Behavioral / Systems / Cognitive Functionally Segregated Neural Substrates for Arbitrary Audiovisual Paired-Association Learning

  2005cited by this paper
• Asymmetrical Neural Substrates of Tactile Discrimination in Humans: A Functional Magnetic Resonance Imaging Study

  2004cited by this paper
• Task-specific recruitment of dorsal and ventral visual areas during tactile perception.

  2004cited by this paper
• The attentional selection of spatial and non-spatial attributes in touch: ERP evidence for parallel and independent processes.

  2004cited by this paper
• Feeling by sight or seeing by touch?

  2004cited by this paper
• Children’s Health: The Opposite of Obesity: Undernutrition Overwhelms the World’s Children

  2004cited by this paper
• Integration of letters and speech sounds in the human brain.

  2004cited by this paper
• A revised view of sensory cortical parcellation

  2004cited by this paper
• Is there a role of visual cortex in spatial hearing?

  2004cited by this paper
• Assessing the auditory dual-pathway model in humans

  2004cited by this paper
• Beyond sensory images: Object-based representation in the human ventral pathway.

  2004cited by this paper
• Integration of auditory and visual information about objects in superior temporal sulcus.

  2004cited by this paper
• Unraveling multisensory integration: patchy organization within human STS multisensory cortex

  2004cited by this paper
• The contribution of the insula to motor aspects of speech production: a review and a hypothesis.

  2004cited by this paper
• Multisensory cortical processing of object shape and its relation to mental imagery

  2004cited by this paper
• Auditory Cortical Neurons Respond to Somatosensory Stimulation

  2003cited by this paper
• Functional Mapping of the Primate Auditory System

  2003cited by this paper
• Polysensory interactions along lateral temporal regions evoked by audiovisual speech.

  2003cited by this paper
• Differences between auditory evoked responses recorded during spatial and nonspatial working memory tasks

  2003cited by this paper
• Distinct Mechanisms for Processing Spatial Sequences and Pitch Sequences in the Human Auditory Brain

  2003cited by this paper
• Neural networks active during tactile form perception: common and differential activity during macrospatial and microspatial tasks.

  2003cited by this paper
• Neuroimaging studies of working memory:

  2003cited by this paper
• The insula (Island of Reil) and its role in auditory processing. Literature review.

  2003cited by this paper
• On the hemispheric specialization for categorical and coordinate spatial relations: a review of the current evidence.

  2003cited by this paper
• Where is 'where' in the human auditory cortex?

  2002cited by this paper
• Functional Magnetic Resonance Imaging of the Human Sensorimotor Cortex Using a Novel Vibrotactile Stimulator

  2002cited by this paper
• Auditory-somatosensory multisensory processing in auditory association cortex: an fMRI study.

  2002cited by this paper
• Deactivation of Sensory-Specific Cortex by Cross-Modal Stimuli

  2002cited by this paper
• An auditory domain in primate prefrontal cortex

  2002cited by this paper
• Convergence of visual and tactile shape processing in the human lateral occipital complex.

  2002influential reference
• What and Where in human audition: selective deficits following focal hemispheric lesions

  2002cited by this paper
• Functional Specialization in Rhesus Monkey Auditory Cortex

  2001cited by this paper
• Polymodal motion processing in posterior parietal and premotor cortex: a human fMRI study strongly implies equivalencies between humans and monkeys.

  2001cited by this paper
• Somatosensory input to auditory association cortex in the macaque monkey.

  2001cited by this paper
• Neuroimaging of cognitive functions in human parietal cortex.

  2001cited by this paper
• Crossmodal processing in the human brain: insights from functional neuroimaging studies.

  2001cited by this paper
• Categorical and coordinate spatial processing: more on contributions of the transient/magnocellular visual system.

  2001cited by this paper
• “What” and “where” in the human auditory system

  2001influential reference
• The metamodal organization of the brain.

  2001cited by this paper
• Visuo-haptic object-related activation in the ventral visual pathway

  2001influential reference
• The functional anatomy of visual-tactile integration in man: a study using positron emission tomography.

  2000cited by this paper
• Distinct Pathways Involved in Sound Recognition and Localization: A Human fMRI Study

  2000influential reference
• Working memory of auditory localization.

  2000cited by this paper
• A Positron Emission Tomographic Study of Auditory Localization in the Congenitally Blind

  2000cited by this paper
• Mechanisms and streams for processing of "what" and "where" in auditory cortex.

  2000cited by this paper
• Automated Talairach Atlas labels for functional brain mapping

  2000influential reference
• A multimodal cortical network for the detection of changes in the sensory environment

  2000cited by this paper
• Involvement of visual cortex in tactile discrimination of orientation

  1999cited by this paper
• Dual streams of auditory afferents target multiple domains in the primate prefrontal cortex

  1999cited by this paper
• Modality-specific frontal and parietal areas for auditory and visual spatial localization in humans

  1999influential reference
• Neural pathways in tactile object recognition

  1999cited by this paper
• A PET study of human auditory spatial processing.

  1999cited by this paper
• Cortical processing of complex sounds.

  1998cited by this paper
• Spatial Processing and Hemispheric Asymmetry: Contributions of the Transient/Magnocellular Visual System

  1998cited by this paper
• Neuroimaging analyses of human working memory.

  1998cited by this paper
• Parallel Processing in the Auditory Cortex of Primates

  1998cited by this paper
• Cross-Modal Transfer of Information between the Tactile and the Visual Representations in the Human Brain: A Positron Emission Tomographic Study

  1998cited by this paper
• Functional MRI studies of spatial and nonspatial working memory.

  1998cited by this paper
• Serial and parallel processing in rhesus monkey auditory cortex

  1997cited by this paper
• Processing of complex sounds in the auditory cortex of cat, monkey, and man.

  1997cited by this paper
• Linear Systems Analysis of Functional Magnetic Resonance Imaging in Human V1

  1996cited by this paper
• Circuitry and functional aspects of the insular lobe in primates including humans

  1996cited by this paper
• Eye-centered, head-centered, and intermediate coding of remembered sound locations in area LIP.

  1996cited by this paper
• Characterizing evoked hemodynamics with fMRI.

  1995cited by this paper

• Age-related neural dynamics revealed by time-domain fNIRS decoding of audiovisual dual-task processing.

  2026cites this paper
• Somatosensory cortical representations of the assimilation effect for vibrotactile stimulation

  2025cites this paper
• Multisensory vs. unisensory learning: how they shape effective connectivity networks subserving unimodal and multimodal integration

  2025cites this paper
• Resting‐State Network Plasticity Following Category Learning Depends on Sensory Modality

  2024cites this paper
• Effects of computerized working memory training on neuroplasticity in healthy individuals: A combined neuroimaging and neurotransmitter study

  2024cites this paper
• Resting-state functional connectivity changes following audio-tactile speech training

  2024cites this paper
• Somatosensory cortical representations of assimilation effect by vibrotactile stimulation

  2024cites this paper
• fNIRS a novel neuroimaging tool to investigate olfaction, olfactory imagery, and crossmodal interactions: a systematic review

  2024cites this paper
• The neural representations underlying asymmetric cross‐modal prediction of words

  2023cites this paper
• Unraveling tactile categorization and decision-making in the subregions of supramarginal gyrus via direct cortical stimulation.

  2023cites this paper
• Language Teaching Methods for Visually Impaired people: Integrating Psycholinguistics Approach

  2023cites this paper
• Investigating cortical networks from vibrotactile stimulation in young adults using independent component analysis: an fMRI study

  2023cites this paper
• Cortical thickness is related to cognitive-motor automaticity and attention allocation in individuals with Alzheimer’s disease: a regions of interest study

  2023cites this paper
• Modulation of perception by visual, auditory, and audiovisual reward predicting cues

  2023cites this paper
• Source Localization of Audiovisual Multisensory Neural Generators in Young Adults with Attention-Deficit/Hyperactivity Disorder

  2022cites this paper
• Neural correlates of multisensory integration in the human brain: an ALE meta-analysis

  2022cites this paper
• An fMRI-study of leading and following using rhythmic tapping

  2022cites this paper
• Brain responses to high frequencies (270 Hz-480 Hz) changes due to vibratory stimulation of human fingertips: An fMRI study

  2020cites this paper
• Multisensory Integration and Sense Modalism

  2020influential citation
• Neural Correlates of Audiovisual Temporal Binding Window in Individuals With Schizotypal and Autistic Traits: Evidence From Resting‐State Functional Connectivity

  2020cites this paper
• Sensory and Working Memory

  2019cites this paper
• Properties of cross-modal occipital responses in early blindness: An ALE meta-analysis

  2019cites this paper
• Audio–visual and olfactory–visual integration in healthy participants and subjects with autism spectrum disorder

  2019cites this paper
• Overt Spatial Attention Modulates Multisensory Selection

  2019cites this paper
• Spatial Patterns of Brain Activity Preferentially Reflecting Transient Pain and Stimulus Intensity

  2019cites this paper
• Long-Term Memory Structure

  2019cites this paper
• Haptic and Auditory-Haptic Attentional Blink in Spatial and Object-Based Tasks.

  2019cites this paper
• Theta-burst stimulation causally affects side perception in the Deutsch’s octave illusion

  2018cites this paper
• Audio-Tactile and Peripersonal Space Processing Around the Trunk in Human Parietal and Temporal Cortex: An Intracranial EEG Study

  2018cites this paper
• Taking two to tango: fMRI analysis of improvised joint action with physical contact

  2018cites this paper
• Involvement of ordinary what and where auditory cortical areas during illusory perception

  2018influential citation
• Multisensory integration processing during olfactory‐visual stimulation—An fMRI graph theoretical network analysis

  2018cites this paper
• Semantic Congruence Alters Functional Connectivity during Olfactory-Visual Perception

  2018cites this paper
• Schizophrenia affects speech-induced functional connectivity of the superior temporal gyrus under cocktail-party listening conditions.

  2017cites this paper
• High‐resolution functional MRI identified distinct global intrinsic functional networks of nociceptive posterior insula and S2 regions in squirrel monkey brain

  2017cites this paper
• Event-Related Potentials Associated with Somatosensory Effect in Audio-Visual Speech Perception

  2017cites this paper
• Multisensory Interplay Within Human Auditory Cortex: New Evidence from Intraoperative Optical Imaging of Intrinsic Signal.

  2017cites this paper
• Hearing, feeling or seeing a beat recruits a supramodal network in the auditory dorsal stream

  2017cites this paper
• Neuronal interactions in areas of spatial attention reflect avoidance of disgust, but orienting to danger

  2016cites this paper
• The Structural and Functional Organization of Cognition

  2016cites this paper
• Simultaneous Assessment of White Matter Changes in Microstructure and Connectedness in the Blind Brain

  2016cites this paper
• Blindness alters the microstructure of the ventral but not the dorsal visual stream

  2016cites this paper
• Cross-modal transfer of statistical information benefits from sleep.

  2016cites this paper
• Analysis of haptic information in the cerebral cortex.

  2016cites this paper
• Same Intervention–Different Reorganization

  2016cites this paper
• Increased resting state connectivity between ipsilesional motor cortex and contralesional premotor cortex after transcranial direct current stimulation with physical therapy

  2016cites this paper
• Neural correlates associated with superior tactile symmetry perception in the early blind.

  2015cites this paper
• The role of the left inferior parietal lobule in reading.

  2015cites this paper
• Space-time interdependence: evidence against asymmetric mapping between time and space.

  2015cites this paper
• General Psychology: An Introduction

  2015cites this paper
• Aural localization of silent objects by active human biosonar: neural representations of virtual echo‐acoustic space

  2015cites this paper
• Brain substrates of perceived spatial separation between speech sources under simulated reverberant listening conditions in schizophrenia

  2015cites this paper
• Dissecting neural circuits for multisensory integration and crossmodal processing

  2015cites this paper
• Features of migraine aura as "Holy Grail" for studying pathophysiology of migraine with aura

  2015cites this paper
• Audiotactile interactions: psychophysical and neuroimaging approaches

  2015cites this paper
• Prestimulus oscillatory alpha power and connectivity patterns predispose perceptual integration of an audio and a tactile stimulus

  2015cites this paper
• The role of the left inferior parietal lobule in reading.

  2015cites this paper
• Cortical spreading depression: origins and paths as inferred from the sequence of events during migraine aura.

  2014cites this paper
• Cortical plasticity and preserved function in early blindness.

  2014cites this paper
• Neuroplasticity in the blind and sensory substitution for vision

  2014cites this paper
• Compensatory plasticity and cross-modal reorganization following early visual deprivation.

  2014cites this paper
• Features of migraine aura in teenagers

  2014cites this paper
• Crossmodal effects on texture perception and processing

  2014cites this paper
• Learning to abstract : How short-term experience shapes neural sound representations

  2013cites this paper
• Just how important is spatial coincidence to multisensory integration? Evaluating the spatial rule

  2013cites this paper
• Do you see what I'm singing? Visuospatial movement biases pitch perception

  2013cites this paper
• Multisensory interactions between auditory and haptic object recognition.

  2013cites this paper
• Struktur-Funktionsbeziehungen in Assoziationsgebieten des Gehirns am Beispiel des unteren Parietallappens

  2013cites this paper
• Interoceptive and multimodal functions of the operculo-insular cortex: Tactile, nociceptive and vestibular representations

  2013cites this paper
• Evoked Multisensory Cortical Representations During Unisensory Stimulation

  2013cites this paper
• Spatio-temporal measures of electrophysiological correlates for behavioral multisensory enhancement during visual, auditory and somatosensory stimulation: A behavioral and ERP study

  2013cites this paper
• The effect of object processing in content-dependent source memory

  2013cites this paper
• Looking into brain activation using a sensory substitution prosthesis : seeing with sound

  2013cites this paper
• Multisensory integration processes in people with attention deficits

  2013cites this paper
• The large-scale organization of "visual" streams emerges without visual experience.

  2012cites this paper
• Altered cortical morphology in sensorimotor processing regions in adolescents and adults with attention-deficit/hyperactivity disorder.

  2012cites this paper
• Looking into Task-Specific Activation Using a Prosthesis Substituting Vision with Audition

  2012cites this paper
• International Evaluation of Research and Doctoral Training at the University of Helsinki 2005-2010: RC-Specific Evaluation of NEUROMED - Neuroscience research at the Institute of Biomedicine

  2012cites this paper
• What can multisensory processing tell us about the functional organization of auditory cortex

  2012cites this paper
• BOLD Responses in Somatosensory Cortices Better Reflect Heat Sensation than Pain

  2012cites this paper
• Dual processing streams in chemosensory perception

  2012influential citation
• Early and Sustained Supramarginal Gyrus Contributions to Phonological Processing

  2012cites this paper
• Across-study and within-subject functional connectivity of a right temporo-parietal junction subregion involved in stimulus-context integration

  2012cites this paper
• Audiotactile interactions in temporal perception

  2011cites this paper
• The Sound of Consciousness: Neural Underpinnings of Auditory Perception

  2011influential citation
• Tinnitus School – An Integrated Management of Somatic Tinnitus

  2011cites this paper
• Dual pathways for haptic and visual perception of spatial and texture information

  2011influential citation
• Functional specialization of mouse higher visual cortical areas.

  2011cites this paper
• Puddles, Parties, and Professors: Linking Word Categorization to Neural Patterns of Visuospatial Coding

  2011cites this paper
• Gamma-Band Activity as a Signature for Cross-Modal Priming of Auditory Object Recognition by Active Haptic Exploration

  2011cites this paper
• The auditory dorsal pathway: orienting vision.

  2011cites this paper
• Episodic retrieval and the cortical binding of relational activity

  2011cites this paper
• Impaired sensitivity to Pavlovian stimulus-outcome learning after excitotoxic lesion of the ventrolateral neostriatum.

  2011cites this paper
• Behavioral and neural correlates of auditory encoding and memory functions in Rhesus Macaques

  2011cites this paper
• Up to Date on Tinnitus

  2011cites this paper
• Adaptation, Expertise, and Giftedness: Towards an Understanding of Cortical, Subcortical, and Cerebellar Network Contributions

  2010cites this paper
• Vision substitution and depth perception: Early blind subjects experience visual perspective through their ears

  2010cites this paper
• Epileptic feeling of multiple presences in the frontal space.

  2010cites this paper
• Preserved functional specialization for spatial processing in the middle occipital gyrus of the early blind.

  2010cites this paper
• Assessing audiotactile interactions: Spatiotemporal factors and role of visual experience

  2009cites this paper