Beyond differences in means: robust graphical methods to compare two groups in neuroscience

If many changes are necessary to improve the quality of neuroscience research, one relatively simple step could have great pay-offs: to promote the adoption of detailed graphical methods, combined with robust inferential statistics. Here we illustrate how such methods can lead to a much more detailed understanding of group differences than bar graphs and t-tests on means. To complement the neuroscientist’s toolbox, we present two powerful tools that can help us understand how groups of observations differ: the shift function and the difference asymmetry function. These tools can be combined with detailed visualisations to provide complementary perspectives about the data. We provide implementations in R and Matlab of the graphical tools, and all the examples in the article can be reproduced using R scripts.

• Publication year

  2017

• Venue

  bioRxiv

• Publication date

  2017-05-16

• Fields of study

  Biology, Medicine, Computer Science

• Identifiers
  DOI 10.1111/ejn.13610PMID 28544058
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• Robust regression: an inferential method for determining which independent variables are most important

  2018cited by this paper
• ggplot2 - Elegant Graphics for Data Analysis (2nd Edition)

  2017cited by this paper
• UvA-DARE ( Digital Academic Repository ) Detecting and avoiding likely false-positive findings

  2017cited by this paper
• 2. Introduction

  2017cited by this paper
• A manifesto for reproducible science

  2017cited by this paper
• Data Visualization and Statistical Graphics in Big Data Analysis

  2016cited by this paper
• Scanning the horizon: towards transparent and reproducible neuroimaging research

  2016cited by this paper
• Comparative effects of adaptation on layers II–III and V–VI neurons in cat V1

  2016cited by this paper
• A few simple steps to improve the description of group results in neuroscience

  2016influential reference
• Modern graphical methods to compare two groups of observations

  2016cited by this paper
• A statistical framework for neuroimaging data analysis based on mutual information estimated via a gaussian copula

  2016influential reference
• Neuronal Diversity : Categorizing Types of Neurons Categorically distinct types of receptive fields in early visual cortex

  2016cited by this paper
• Beyond Bar and Line Graphs: Time for a New Data Presentation Paradigm

  2015influential reference
• Seeing is believing: good graphic design principles for medical research

  2015cited by this paper
• Streamlined Plot Theme and Plot Annotations for 'ggplot2'

  2015cited by this paper
• A robust and representative lower bound on object processing speed in humans

  2015cited by this paper
• R: A language and environment for statistical computing.

  2014influential reference
• Comparing two independent groups via the lower and upper quantiles

  2014influential reference
• Power failure: why small sample size undermines the reliability of neuroscience

  2013cited by this paper
• Comparing Two Independent Groups Via a Quantile Generalization of the Wilcoxon-Mann-Whitney Test

  2012influential reference
• Data visualization in the neurosciences: overcoming the curse of dimensionality.

  2012cited by this paper
• Neuron NeuroView Data Visualization in the Neurosciences : Overcoming the Curse of Dimensionality

  2012influential reference
• Comparing two dependent groups via quantiles

  2012influential reference
• An Agenda for Purely Confirmatory Research

  2012cited by this paper
• ggplot2: Elegant Graphics for Data Analysis

  2010cited by this paper
• Exploring the differences in distributional properties between Stroop and Simon effects using delta plots

  2010cited by this paper
• Graphs in Statistical Analysis

  2010cited by this paper
• Relative Distribution Methods

  2010cited by this paper
• Basic Statistics: Understanding Conventional Methods and Modern Insights

  2009cited by this paper
• When all hypotheses are right: A multifocal account of dyslexia

  2009influential reference
• Brain classification reveals the right cerebellum as the best biomarker of dyslexia

  2009influential reference
• Delta Plots and Coherent Distribution Ordering

  2008influential reference
• Modern robust statistical methods: an easy way to maximize the accuracy and power of your research.

  2008cited by this paper
• Graphical Methods for Assessing Effect Size: Some Alternatives to Cohen's d

  2006cited by this paper
• Delta plots in the study of individual differences: new tools reveal response inhibition deficits in AD/Hd that are eliminated by methylphenidate treatment.

  2005cited by this paper
• Introduction to Robust Estimation and Hypothesis Testing

  2005cited by this paper
• Introduction to Robust Estimation and Hypothesis Testing (2nd ed.)

  2005influential reference
• Modern robust data analysis methods: measures of central tendency.

  2003cited by this paper
• Modern Insights About Pearson’s Correlation and Least Squares Regression

  2001cited by this paper
• Measuring effect size: a non-parametric analogue of omega 2.

  1999influential reference
• Measuring effect size: A non-parametric analogue of ω2

  1999cited by this paper
• Comparing Two Independent Groups Via Multiple Quantiles

  1995cited by this paper
• Conditional and unconditional automaticity: a dual-process model of effects of spatial stimulus-response correspondence.

  1994cited by this paper
• A new distribution-free quantile estimator

  1982cited by this paper
• Some graphical methods in statistics: A review and some extensions

  1977influential reference
• Plotting with confidence: Graphical comparisons of two populations

  1976cited by this paper
• Empirical Probability Plots and Statistical Inference for Nonlinear Models in the Two-Sample Case

  1974influential reference
• Acquisition of resistance in guinea pigs infected with different doses of virulent tubercle bacilli.

  1960influential reference

• Language and cognition revisited: Labels alter post-sensory processing in visual and auditory categorization

  2026cites this paper
• Comparing Various Descriptive Statistics for Two Independent Groups in Agricultural Experiments

  2026cites this paper
• From significant to meaningful: ATOMizing the study of sex differences and similarities.

  2025cites this paper
• Cross-frequency coupling patterns during oddball processing in disorders of consciousness: a delta–gamma predominance

  2025cites this paper
• A custom analysis of R-R interval properties reveals alterations in the brain-heart axis in experimental animals with metabolic syndrome.

  2025cites this paper
• Leveraging LLMs for User Rating Prediction from Textual Reviews: A Hospitality Data Annotation Case Study

  2025cites this paper
• Research outcome expectations questionnaire: Validation in undergraduate Hispanic students

  2025cites this paper
• Evaluating Interaction Capability in a Serious Game for Children with ASD: An Operability-Based Approach Aligned with ISO/IEC 25010:2023

  2025cites this paper
• Notch signaling blockade links transcriptome heterogeneity in quiescent neural stem cells with reactivation routes and potential

  2025cites this paper
• Assessing individual physiological variability and future performance phenotypes is essential for predicting the resilience of fish populations to anthropogenic climate change

  2025cites this paper
• Gaze behavior in face-to-face interaction: A cross-cultural investigation between Japan and The Netherlands.

  2025cites this paper
• Diagnostic tools for exploring differences in distributional properties between two samples: nonparametric approach

  2025influential citation
• Discrete Repetition Effects for Visual Words Compared to Faces and Animals, but No Modulation by Expectation: An Event‐Related Potential Study

  2025cites this paper
• Change in exercise capacity, physical activity and motivation for physical activity at 12 months after a cardiac rehabilitation program in coronary heart disease patients: a prospective, monocentric and observational study

  2025cites this paper
• Perception of Attachment Security, Neurodynamics of Emotion Recognition, and Social Skills in Late Childhood and Early Adolescence: A Multilinear Modeling Study

  2025cites this paper
• Distributional moderation analysis: Unpacking moderation effects in intervention research.

  2025cites this paper
• The fundamentals of eye tracking part 4: Tools for conducting an eye tracking study

  2025cites this paper
• Combination of rosuvastatin and curcumin outperforms monotherapies in alleviating gentamicin-induced nephrotoxicity, audiotoxicity and vestibulotoxicity in a rat animal model.

  2025cites this paper
• ODC and ROC Curves, Comparison Curves and Stochastic Dominance

  2024cites this paper
• Gaze-action coupling, gaze-gesture coupling, and exogenous attraction of gaze in dyadic interactions

  2024cites this paper
• Early- and Late-Stage Auditory Processing of Speech Versus Non-Speech Sounds in Children With Autism Spectrum Disorder: An ERP and Oscillatory Activity Study.

  2024cites this paper
• Impact of prenatal delta-9-tetrahydrocannabinol exposure on mouse brain development: a fetal-to-adulthood magnetic resonance imaging study

  2024cites this paper
• Exploring the spatial interference effects elicited by social and non‐social targets: A conditional accuracy function approach

  2024cites this paper
• The Timing of Gaze Direction Perception: ERP Decoding and Task Modulation

  2024cites this paper
• Optimasi Pembelajaran: Strategi Meningkatkan Pemahaman Statistik melalui Pemahaman Kecerdasan Emosi, Spiritual, dan Intelektual Mahasiswa

  2024cites this paper
• Modality-specific impacts of distractors on visual and auditory categorical decision-making: an evidence accumulation perspective

  2024cites this paper
• Communicating with Machines: Future Trajectories in the Field of Human-Machine Communication

  2024cites this paper
• From means to meaning in the study of sex/gender differences and similarities.

  2024influential citation
• Adapting to Covariate Shift in Real-time by Encoding Trees with Motion Equations

  2024cites this paper
• Autonomy-supportive instructional language does not enhance skill acquisition compared to controlling instructional language

  2024influential citation
• Using cluster‐based permutation tests to estimate MEG/EEG onsets: How bad is it?

  2024cites this paper
• Laboratory Automated Interrogation of Data: an interactive web application for visualization of multilevel data from biological experiments

  2024cites this paper
• Audiovisual interactions outside of visual awareness during motion adaptation

  2024cites this paper
• Measurement and Comparison of Distributional Shift with Applications to Ecology, Economics, and Image Analysis

  2024cites this paper
• The effect of using group-averaged or individualized brain parcellations when investigating connectome dysfunction in psychosis

  2023cites this paper
• Cultural facial expressions dynamically convey emotion category and intensity information.

  2023cites this paper
• Association of Endotoxemia with Low-Grade Inflammation, Metabolic Syndrome and Distinct Response to Lipopolysaccharide in Type 1 Diabetes

  2023cites this paper
• Empirical Likelihood-Based Confidence Regions for the Structural Relationship Parameter

  2023cites this paper
• An introduction to the bootstrap: a versatile method to make inferences by using data-driven simulations.

  2023influential citation
• Managing the Assumption of Normality within the General Linear Model with Small Samples: Guidelines for Researchers Regarding If, When and How.

  2023cites this paper
• Behavioral as well as hippocampal transcriptomic and microglial responses differ across sexes in adult mouse offspring exposed to a dual genetic and environmental challenge.

  2023cites this paper
• Effect sizes for equivalence testing: Incorporating the equivalence interval

  2023cites this paper
• Task-related gaze behaviour in face-to-face dyadic collaboration: Toward an interactive theory?

  2023cites this paper
• Are gender-science stereotypes barriers for women in science, technology, engineering, and mathematics? Exploring when, how, and to whom in an experimentally-controlled setting

  2023cites this paper
• Agricultural drought severity in NE Italy: Variability, bias, and future scenarios

  2023cites this paper
• Relative density clouds: Visualizing and exploring multivariate patterns of group differences

  2023cites this paper
• Analysis of distributions reveals real differences on dichotic listening scores between left- and right-handers

  2023influential citation
• A quantile shift approach to main effects and interactions in a 2-by-2 design

  2023cites this paper
• An Updated Guide to Robust Statistical Methods in Neuroscience

  2023influential citation
• The S-index: Summarizing patterns of sex differences at the distribution extremes

  2023cites this paper
• The effect of using group-averaged or individualized brain parcellations when investigating connectome dysfunction: A case study in psychosis

  2023cites this paper
• What Is the Support for Conspiracy Beliefs About COVID-19 Vaccines in Latin America? A Prospective Exploratory Study in 13 Countries

  2022cites this paper
• Explainable Machine Learning Approach to Predict and Explain the Relationship between Task-based fMRI and Individual Differences in Cognition

  2022cites this paper
• Inter-Individual Variability in tDCS Effects: A Narrative Review on the Contribution of Stable, Variable, and Contextual Factors

  2022cites this paper
• Inter‐trial theta phase consistency during face processing in infants is associated with later emerging autism

  2022cites this paper
• Low Prevalence of A Priori Power Analyses in Motor Behavior Research

  2022cites this paper
• Setting Targets for Wetland Restoration to Mitigate Climate Change Effects on Watershed Hydrology

  2022cites this paper
• Beyond ANOVA and MANOVA for repeated measures: Advantages of generalized estimated equations and generalized linear mixed models and its use in neuroscience research

  2022cites this paper
• Quantifying neuro-motor correlations during awake deep brain stimulation surgery using markerless tracking

  2022cites this paper
• Physiological bases of cultivar differences in average grain weight in wheat: Scaling down from plot to individual grain in elite material

  2022cites this paper
• The Development of Creative Search Strategies

  2022cites this paper
• The magnitude of heterogeneity in individual-cell growth dynamics is an inherent characteristic of Salmonella enterica ser. Typhimurium strains.

  2022cites this paper
• Exercising choice over feedback schedules during practice is not advantageous for motor learning

  2022cites this paper
• The place of probability distributions in statistical learning. A commented book review of ‘Distributions for modeling location, scale, and shape using GAMLSS in R’ by Rigby et al. (2021)

  2022cites this paper
• Analysis of the cognitive aspects of the preference elicitation process in the compensatory context: a neuroscience experiment with FITradeoff

  2022cites this paper
• Enlarged Interior Built Environment Scale Modulates High-Frequency EEG Oscillations

  2022cites this paper
• Eye contact avoidance in crowds: A large wearable eye-tracking study

  2022cites this paper
• Role of Perineuronal nets in the cerebellar cortex in cocaine-induced conditioned preference, extinction, and reinstatement.

  2022cites this paper
• Comparing Two Groups

  2022cites this paper
• ElectroEncephaloGraphy robust statistical linear modelling using a single weight per trial

  2022cites this paper
• Electroencephalography robust statistical linear modelling using a single weight per trial

  2022cites this paper
• The third party will make a difference - A study on the impact of dyadic and triadic social situations on the relationship with a voice-based personal agent

  2022cites this paper
• Inter-Trial Formant Variability in Speech Production Is Actively Controlled but Does Not Affect Subsequent Adaptation to a Predictable Formant Perturbation

  2022cites this paper
• A modified variant of coyote optimization algorithm for solving ordinary differential equations and oscillatory mechanical problems

  2022cites this paper
• At a second glance: cognitive and oculomotor neural activity of refixation planning

  2022cites this paper
• Shoulder Pain, Function, and Ultrasound-Determined Structure in Elite Wheelchair-Using Para Athletes: An Observational Study

  2022cites this paper
• The Relative Contribution of Executive Functions and Aging on Attentional Control During Road Crossing

  2022cites this paper
• The relationship between self, value-based reward, and emotion prioritisation effects

  2022influential citation
• Beyond "sex prediction": Estimating and interpreting multivariate sex differences and similarities in the brain

  2021cites this paper
• Users’ Response to Critical Situations in Automated Driving: Rear-Ends, Sideswipes, and False Warnings

  2021cites this paper
• Tropical cyclone simulations over Bangladesh at convection permitting 4.4 km & 1.5 km resolution

  2021cites this paper
• Modulation of Peak Alpha Frequency Oscillations During Working Memory Is Greater in Females Than Males

  2021cites this paper
• The influence of the respiratory cycle on reaction times in sensory-cognitive paradigms

  2021cites this paper
• Phenotyping of Klf14 mouse white adipose tissue enabled by whole slide segmentation with deep neural networks

  2021cites this paper
• Normal colour perception in developmental prosopagnosia

  2021cites this paper
• Differential effects of early or late exposure to prenatal maternal immune activation on mouse embryonic neurodevelopment

  2021cites this paper
• Diurnal variations in multi-sensor wearable-derived sleep characteristics in morning- and evening-type shift workers under naturalistic conditions

  2021cites this paper
• Disrupted basal ganglia output during movement preparation in hemi-parkinsonian mice is consistent with behavioral deficits.

  2021cites this paper
• Inverse Sampling of Degenerate Datasets from a Linear Regression Line

  2021cites this paper
• Perception of the Potential for Interaction in Social Scenes

  2021cites this paper
• A model for naturalistic glance behavior around Tesla Autopilot disengagements.

  2021cites this paper
• Summary Plots With Adjusted Error Bars: The superb Framework With an Implementation in R

  2021cites this paper
• Failure modelling of glass plates in biaxial loading: using flaw-size based weakest-link systems

  2021cites this paper
• Medial Entorhinal Cortex Excitatory Neurons Are Necessary for Accurate Timing

  2021cites this paper
• Subtle alterations in neonatal neurodevelopment following early or late exposure to prenatal maternal immune activation in mice

  2021cites this paper
• Individual pupil size changes as a robust indicator of cognitive familiarity differences

  2021cites this paper
• A data‐driven geospatial workflow to map species distributions for conservation assessments

  2021cites this paper
• Emergence of perceptuomotor relationships during paleolithic stone toolmaking learning: intersections of observation and practice

  2021cites this paper
• Diffusion MRI analysis methods

  2021cites this paper
• Two graphs walk into a bar: Readout-based measurement reveals the Bar-Tip Limit error, a common, categorical misinterpretation of mean bar graphs

  2021cites this paper