From maps to models: A survey on the reliability of small studies of task-based fMRI

Abstract Task-based functional magnetic resonance imaging (fMRI) is a powerful tool for studying brain function. However, the reliability and viability of small-sample studies remain a concern. While it is well understood that larger samples are preferable, researchers often need to interpret findings from small studies (e.g., when reviewing the literature, analyzing pilot data, or assessing subsamples). However, quantitative guidance for making these judgments remains scarce. To address this gap, we leverage the UK Biobank and the Human Connectome Project’s Young Adult dataset to survey a range of standard task-based fMRI analyses, from obtaining regional activation maps to performing predictive modeling. These analyses are repeated using volumetric and two types of cortical surface data. For classic mass-univariate analyses (e.g., regional activation detection or cluster peak localization), studies with as few as 40 participants can be adequate depending on the effect size. For predictive modeling, similar sample sizes can be used to detect whether a feature is predictable, but developing stable, generalizable models typically requires cohorts at least an order of magnitude larger, and possibly two (hundreds or thousands). Together, these results clarify how reliability depends on the interplay of effect size, sample size, and analysis type, offering practical guidance for designing and interpreting small-scale task-fMRI studies.

• Publication year

  2026

• Venue

  Imaging neuroscience

• Publication date

  2026-01-07

• Fields of study

  Biology, Medicine

• Identifiers
  DOI 10.1162/IMAG.a.1076PMID 41523208PMCID 12779752
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• Overlooked features lead to divergent neurobiological interpretations of brain-based machine learning biomarkers

  2025cited by this paper
• Sampling inequalities affect generalization of neuroimaging-based diagnostic classifiers in psychiatry

  2023cited by this paper
• Reproducible brain-wide association studies require thousands of individuals

  2022cited by this paper
• Brain–phenotype models fail for individuals who defy sample stereotypes

  2022cited by this paper
• Highlight Results, Don’t Hide Them: Enhance interpretation, reduce biases and improve reproducibility

  2022cited by this paper
• Performance reserves in brain-imaging-based phenotype prediction

  2022cited by this paper
• Effect sizes and test-retest reliability of the fMRI-based Neurologic Pain Signature

  2021cited by this paper
• Insights from an autism imaging biomarker challenge: promises and threats to biomarker discovery

  2021cited by this paper
• Moderated t-tests for group-level fMRI analysis

  2021cited by this paper
• Functional MRI Can Be Highly Reliable, but It Depends on What You Measure: A Commentary on Elliott et al. (2020)

  2021cited by this paper
• A neuroimaging biomarker for sustained experimental and clinical pain

  2021cited by this paper
• Confidence Sets for Cohen’s d effect size images

  2020cited by this paper
• Variability may limit the translation of neuroimaging findings comment on "Variability in the analysis of a single neuroimaging dataset by many teams".

  2020cited by this paper
• A guide to the measurement and interpretation of fMRI test-retest reliability.

  2020cited by this paper
• NeuroMark: An automated and adaptive ICA based pipeline to identify reproducible fMRI markers of brain disorders

  2020cited by this paper
• Neuroimaging results altered by varying analysis pipelines

  2020cited by this paper
• The empirical replicability of task-based fMRI as a function of sample size

  2020cited by this paper
• Fine-grain atlases of functional modes for fMRI analysis

  2020cited by this paper
• Deep neural networks and kernel regression achieve comparable accuracies for functional connectivity prediction of behavior and demographics

  2020cited by this paper
• Reply to: fMRI replicability depends upon sufficient individual-level data

  2019cited by this paper
• Variability in the analysis of a single neuroimaging dataset by many teams

  2019cited by this paper
• Cluster failure or power failure? Evaluating sensitivity in cluster-level inference

  2019cited by this paper
• Establishment of Best Practices for Evidence for Prediction: A Review.

  2019cited by this paper
• Identifying reproducible individual differences in childhood functional brain networks: An ABCD study

  2019cited by this paper
• Combining multiple connectomes improves predictive modeling of phenotypic measures

  2019cited by this paper
• Probabilistic TFCE: A generalized combination of cluster size and voxel intensity to increase statistical power

  2019influential reference
• Standardized Effect Sizes and Image-Based Meta-Analytical Approaches for fMRI Data

  2019cited by this paper
• Large-Scale Gradients in Human Cortical Organization.

  2018cited by this paper
• Small sample sizes reduce the replicability of task-based fMRI studies

  2018influential reference
• Exploring the impact of analysis software on task fMRI results

  2018cited by this paper
• The impact of traditional neuroimaging methods on the spatial localization of cortical areas

  2018cited by this paper
• Task-induced brain state manipulation improves prediction of individual traits

  2018cited by this paper
• fMRI replicability depends upon sufficient individual-level data

  2018cited by this paper
• Local-Global Parcellation of the Human Cerebral Cortex From Intrinsic Functional Connectivity MRI

  2017cited by this paper
• The relation between statistical power and inference in fMRI

  2017cited by this paper
• The conception of the ABCD study: From substance use to a broad NIH collaboration

  2017cited by this paper
• Multimodal Surface Matching with Higher-Order Smoothness Constraints☆

  2017cited by this paper
• Cross‐validation failure: Small sample sizes lead to large error bars

  2017cited by this paper
• Shared High Value Research Resources: The CamCAN Human Lifespan Neuroimaging Dataset Processed on the Open Science Grid

  2017cited by this paper
• Effect Size Estimation in Neuroimaging.

  2017cited by this paper
• All-Resolutions Inference for Brain Imaging

  2017cited by this paper
• Inflated false negative rates undermine reproducibility in task-based fMRI

  2017cited by this paper
• Image processing and Quality Control for the first 10,000 brain imaging datasets from UK Biobank

  2017cited by this paper
• What reliability can and cannot tell us about pain report and pain neuroimaging

  2016cited by this paper
• Scanning the horizon: towards transparent and reproducible neuroimaging research

  2016cited by this paper
• Multimodal population brain imaging in the UK Biobank prospective epidemiological study

  2016cited by this paper
• Between‐ and within‐site variability of fMRI localizations

  2016cited by this paper
• A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research.

  2016cited by this paper
• Evaluation of highly accelerated simultaneous multi-slice EPI for fMRI

  2015cited by this paper
• Data-analytical stability of cluster-wise and peak-wise inference in fMRI data analysis.

  2015cited by this paper
• Effects of thresholding on correlation-based image similarity metrics

  2015cited by this paper
• Permutation inference for the general linear model

  2014cited by this paper
• MSM: A new flexible framework for Multimodal Surface Matching

  2014cited by this paper
• NeuroVault.org: a web-based repository for collecting and sharing unthresholded statistical maps of the human brain

  2014cited by this paper
• Cluster-extent based thresholding in fMRI analyses: Pitfalls and recommendations

  2014cited by this paper
• Machine learning for neuroimaging with scikit-learn

  2014cited by this paper
• The Autism Brain Imaging Data Exchange: Towards Large-Scale Evaluation of the Intrinsic Brain Architecture in Autism

  2013cited by this paper
• The minimal preprocessing pipelines for the Human Connectome Project

  2013cited by this paper
• Function in the human connectome: Task-fMRI and individual differences in behavior

  2013influential reference
• Power failure: why small sample size undermines the reliability of neuroscience

  2013cited by this paper
• An improved framework for confound regression and filtering for control of motion artifact in the preprocessing of resting-state functional connectivity data

  2013cited by this paper
• Variability of clinical functional MR imaging results: a multicenter study.

  2013cited by this paper
• The cost of using exact confidence intervals for a binomial proportion

  2013cited by this paper
• The WU-Minn Human Connectome Project: An overview

  2013cited by this paper
• An fMRI-based neurologic signature of physical pain.

  2013cited by this paper
• Ironing out the statistical wrinkles in "ten ironic rules"

  2013cited by this paper
• Blipped‐controlled aliasing in parallel imaging for simultaneous multislice echo planar imaging with reduced g‐factor penalty

  2012cited by this paper
• Whole-brain, time-locked activation with simple tasks revealed using massive averaging and model-free analysis

  2012cited by this paper
• Parcellations and hemispheric asymmetries of human cerebral cortex analyzed on surface-based atlases.

  2012cited by this paper
• Scikit-learn: Machine Learning in Python

  2011cited by this paper
• Informatics and Data Mining Tools and Strategies for the Human Connectome Project

  2011cited by this paper
• The organization of the human cerebellum estimated by intrinsic functional connectivity.

  2011cited by this paper
• The organization of the human cerebral cortex estimated by intrinsic functional connectivity.

  2011cited by this paper
• A common, high-dimensional model of the representational space in human ventral temporal cortex.

  2011cited by this paper
• Mapping anterior temporal lobe language areas with fMRI: A multicenter normative study

  2011cited by this paper
• Encoding and decoding in fMRI

  2011cited by this paper
• The organization of the human cerebral cortex estimated by intrinsic functional connectivity

  2011cited by this paper
• Multiplexed Echo Planar Imaging for Sub-Second Whole Brain FMRI and Fast Diffusion Imaging

  2010cited by this paper
• Reliability of fMRI motor tasks in structures of the corticostriatal circuitry: Implications for future studies and circuit function

  2010cited by this paper
• Multiband multislice GE‐EPI at 7 tesla, with 16‐fold acceleration using partial parallel imaging with application to high spatial and temporal whole‐brain fMRI

  2010cited by this paper
• Coordinate‐based activation likelihood estimation meta‐analysis of neuroimaging data: A random‐effects approach based on empirical estimates of spatial uncertainty

  2009cited by this paper
• Threshold-free cluster enhancement: Addressing problems of smoothing, threshold dependence and localisation in cluster inference

  2009cited by this paper
• Big Correlations in Little Studies: Inflated fMRI Correlations Reflect Low Statistical Power—Commentary on Vul et al. (2009)

  2009cited by this paper
• Type I and Type II error concerns in fMRI research: re-balancing the scale.

  2009cited by this paper
• Analysis of a large fMRI cohort: Statistical and methodological issues for group analyses

  2007cited by this paper
• Localizing the rostrolateral prefrontal cortex at the individual level

  2007cited by this paper
• A rapid fMRI task battery for mapping of visual, motor, cognitive, and emotional function

  2006cited by this paper
• A well-conditioned estimator for large-dimensional covariance matrices

  2004cited by this paper
• Advances in functional and structural MR image analysis and implementation as FSL.

  2004cited by this paper
• Repeatability of motor and working-memory tasks in healthy older volunteers: assessment at functional MR imaging.

  2004cited by this paper
• Intrasubject reproducibility of presurgical language lateralization and mapping using fMRI

  2003cited by this paper
• Valence, gender, and lateralization of functional brain anatomy in emotion: a meta-analysis of findings from neuroimaging

  2003cited by this paper
• Meta-Analysis of the Functional Neuroanatomy of Single-Word Reading: Method and Validation

  2002cited by this paper
• Test-retest precision of functional magnetic resonance imaging processed with independent component analysis

  2002cited by this paper
• Estimating sample size in functional MRI (fMRI) neuroimaging studies: statistical power analyses.

  2002cited by this paper
• Sensorimotor mapping of the human cerebellum: fMRI evidence of somatotopic organization

  2001cited by this paper
• Reliability of functional MR imaging with word-generation tasks for mapping Broca's area.

  2001cited by this paper
• Movement and mind: a functional imaging study of perception and interpretation of complex intentional movement patterns.

  2000cited by this paper
• Tracking the hemodynamic responses to reward and punishment in the striatum.

  2000cited by this paper
• Within-subject reproducibility of visual activation patterns with functional magnetic resonance imaging using multislice echo planar imaging.

  1998cited by this paper

• No citing papers are available for this paper.