Rank-based multiple test procedures and simultaneous confidence intervals

We study simultaneous rank procedures for unbalanced designs with independent observations. The hypotheses are formulated in terms of purely nonparametric treatment effects. In this context, we derive rankbased multiple contrast test procedures and simultaneous confidence intervals which take the correlation between the test statistics into account. Hereby, the individual test decisions and the simultaneous confidence intervals are compatible. This means, whenever an individual hypothesis has been rejected by the multiple contrast test, the corresponding simultaneous confidence interval does not include the null, i.e. the hypothetical value of no treatment effect. The procedures allow for testing arbitrary purely nonparametric multiple linear hypotheses (e.g. many-to-one, all-pairs, changepoint, or even average comparisons). We do not assume homogeneous variances of the data; in particular, the distributions can have different shapes even under the null hypothesis. Thus, a solution to the multiple nonparametric Behrens-Fisher problem is presented in this unified framework.

• Publication year

  2012

• Venue

  Electronic Journal of Statistics

• Publication date

  2012-12-01

• Fields of study

  Mathematics

• Identifiers
  DOI 10.1214/12-EJS691
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Multiple Comparison Procedures

  2013cited by this paper
• Evaluation of Toxicological Studies Using a Nonparametric Shirley-Type Trend Test for Comparing Several Dose Levels with a Control Group

  2012influential reference
• Nonparametric Evaluation of Quantitative Traits in Population-Based Association Studies when the Genetic Model is Unknown

  2012influential reference
• A SAS® macro implementation of a multiple comparison post hoc test for a Kruskal-Wallis analysis

  2011cited by this paper
• A Robust Procedure for Comparing Multiple Means under Heteroscedasticity in Unbalanced Designs

  2010cited by this paper
• Multiple Comparisons Using R

  2010cited by this paper
• Simultaneous confidence intervals using ordinal effect measures for ordered categorical outcomes

  2009influential reference
• Computation of Multivariate Normal and t Probabilities

  2009cited by this paper
• Simultaneous Inference in General Parametric Models

  2008cited by this paper
• Nonparametric multiple comparison procedures for unbalanced one-way factorial designs

  2008cited by this paper
• Nonparametric multiple comparison procedures for unbalanced two-way layouts

  2008cited by this paper
• Modeling and inference for an ordinal effect size measure

  2008cited by this paper
• Multiple Contrast Tests in the Presence of Heteroscedasticity

  2008cited by this paper
• Wilcoxon–Mann–Whitney test for stratified samples and Efron's paradox dice

  2007cited by this paper
• An Introduction to the Bootstrap

  2007cited by this paper
• Simultaneous confidence intervals by iteratively adjusted alpha for relative effects in the one-way layout

  2006cited by this paper
• A Unified Nonparametric Approach for Unbalanced Factorial Designs

  2005influential reference
• Multiple Comparison Procedures

  2005influential reference
• Nonparametric Analysis of Longitudinal Data in Factorial Experiments

  2003cited by this paper
• The multivariate nonparametric Behrens–Fisher problem

  2002cited by this paper
• Kruskal–Wallis, Multiple Comparisons and Efron Dice

  2002cited by this paper
• A Unified Approach to Simultaneous Rank Test Procedures in the Unbalanced One-way Layout

  2001cited by this paper
• Nonparametric methods in factorial designs

  2001influential reference
• On the Numerical Availability of Multiple Comparison Procedures

  2001influential reference
• The Nonparametric Behrens‐Fisher Problem: Asymptotic Theory and a Small‐Sample Approximation

  2000cited by this paper
• Linear rank score statistics when ties are present

  1999cited by this paper
• Statistical Principles for Clinical Trials

  1998cited by this paper
• Nonparametric Hypotheses and Rank Statistics for Unbalanced Factorial Designs

  1997influential reference
• Box-Type Approximations in Nonparametric Factorial Designs

  1997cited by this paper
• Comparison of Several Treatments with a Control Using Multiple Contrasts

  1987cited by this paper
• A modification of the kruskal-wallis statistic for the generalized behrens-fisher problem

  1984cited by this paper
• A Class of Distribution-Free Two-Sample Tests Based on Placements

  1982cited by this paper
• Statistique non Paramétrique Asymptotique

  1980cited by this paper
• A unified approach to the asymptotic distribution theory of certain midrank statistics

  1980influential reference
• SIMULTANEOUS TEST PROCEDURES-SOME THEORY OF MULTIPLE COMPARISONS'

  1969influential reference
• Multiple Comparisons Using Rank Sums

  1964cited by this paper
• A Rank Sum Test for Comparing All Pairs of Treatments

  1960cited by this paper
• A multiple comparison rank sum test: Treatments versus control.

  1959cited by this paper
• Some Theorems on Quadratic Forms Applied in the Study of Analysis of Variance Problems, I. Effect of Inequality of Variance in the One-Way Classification

  1954cited by this paper

• Beyond Bonferroni: new multiple contrast tests for time-to-event data under non-proportional hazards

  2026cites this paper
• Design and analysis of randomized clinical trials for onchocerciasis, loiasis and mansonellosis: A systematic review.

  2026cites this paper
• Spatial and temporal variation in the diversity, abundance, and energetic returns of underground storage organs on the coastal lowlands of the Cape Floristic Region, South Africa

  2026cites this paper
• The Behrens-Fisher problem revisited

  2025cites this paper
• Preclinical Ischemic Stroke Multicenter (PRISM) Trials Collective Statement: Opportunities, Challenges, and Recommendations for a New Era.

  2025cites this paper
• Résultats et survie des tiges non cimentées à collerette dans l’arthroplastie totale de la hanche pour le traitement de l’arthrose primaire : une analyse du registre Suisse

  2025cites this paper
• The relevance of climate change and sustainability in nursing education: a cross-sectional study of students’ perspectives

  2025cites this paper
• Multivariate and multiple contrast testing in general covariate-adjusted factorial designs

  2025cites this paper
• Effects of Different Adjuvants on the Protective Efficacy of a Subcellular Vaccine Against Chlamydia abortus Infection in Sheep

  2025cites this paper
• Outcomes of Collared Uncemented Stems in Total Hip Arthroplasty For Primary Osteoarthritis: A Swiss National joint registry analysis.

  2025cites this paper
• Simultaneous aligned rank transform tests in analysis of covariance based on pairwise ranking

  2025cites this paper
• Multiple Contrast Tests in the Presence of Partial Heteroskedasticity

  2025cites this paper
• Analysis of Covariance in General Factorial Designs Through Multiple Contrast Tests Under Variance Heteroscedasticity

  2025cites this paper
• Fynbos live fuel moisture displays limited variation among seasons and fire weather conditions

  2025cites this paper
• Multiple Contrast Tests for Count Data: Small Sample Approximations and Their Limitations

  2025cites this paper
• Dosing‐time, feeding, and sex‐dependent variations of everolimus pharmacokinetics in mice

  2024influential citation
• Quadratic Form based Multiple Contrast Tests for Comparison of Group Means

  2024cites this paper
• A General Framework for the Multiple Nonparametric Behrens–Fisher Problem With Dependent Replicates

  2024cites this paper
• Single CASANOVA? Not in multiple comparisons

  2024cites this paper
• Unlocking Insights: Enhanced Analysis of Covariance in General Factorial Designs through Multiple Contrast Tests under Variance Heteroscedasticity

  2024cites this paper
• Sample size planning for rank‐based multiple contrast tests

  2024influential citation
• Carbon sequestration potential of plantation forests in New Zealand - no single tree species is universally best

  2024cites this paper
• Biocontrol effects of three antagonistic bacteria strains against Codonopsis pilosula wilt disease caused by Fusarium oxysporum

  2024cites this paper
• Web APIs: Features, Issues, and Expectations – A Large-Scale Empirical Study of Web APIs From Two Publicly Accessible Registries Using Stack Overflow and a User Survey

  2023cites this paper
• Combining stochastic tendency and distribution overlap towards improved nonparametric effect measures and inference

  2023influential citation
• rankFD: An R Software Package for Nonparametric Analysis of General Factorial Designs

  2023influential citation
• Patient satisfaction after EDOF intraocular lens implantation in vitrectomized eyes

  2023cites this paper
• Application of Statistical Methods for Central Statistical Monitoring and Implementations on the German Multiple Sclerosis Registry

  2023influential citation
• Simultaneous confidence intervals from randomization tests with application in testing bioequivalence with multiple endpoints

  2023cites this paper
• The Kruskal Wallis test can not be recommended

  2023cites this paper
• Sample size planning for multiple contrast tests

  2023cites this paper
• Evaluation of histological findings with severity grade, to analyze toxicology in-vivo studies

  2022cites this paper
• A Multiple kernel testing procedure for non-proportional hazards in factorial designs

  2022cites this paper
• Influence of height on ED95 of intrathecal 2-chloroprocaine for knee arthroscopy

  2022cites this paper
• Multiple comparison procedures renewed. An overview of the Dunnett’s test and its application: using CRAN R packages

  2022cites this paper
• A prospective observational study of post-COVID-19 chronic fatigue syndrome following the first pandemic wave in Germany and biomarkers associated with symptom severity

  2022cites this paper
• Seven recommendations for alternatives to the common analysis of variance (ANOVA) with application in the life sciences -- using R

  2022cites this paper
• Self-Admitted Technical Debt and comments’ polarity: an empirical study

  2022cites this paper
• Simultaneous confidence intervals for the interpretation of primary and secondary effects in factorial designs without a pre-test on interaction

  2022cites this paper
• Redundancy, Context, and Preference: An Empirical Study of Duplicate Pull Requests in OSS Projects

  2022cites this paper
• Estimation and Testing of Wilcoxon-Mann-Whitney Effects in Factorial Clustered Data Designs

  2022cites this paper
• Statistical inference for diagnostic test accuracy studies with multiple comparisons

  2021cites this paper
• Simultaneous inference for partial areas under receiver operating curves—With a view towards efficiency

  2021cites this paper
• Efficient multivariate inference in general factorial diagnostic studies

  2021cites this paper
• Distribution-free inference about partial areas under receiver operating characteristic curves in factorial designs

  2021cites this paper
• Simultaneous inference for Kendall's tau

  2021cites this paper
• Grup Karşılaştırmalarında Alternatif Bir Yaklaşım Olarak ANOM Testi

  2021cites this paper
• Does the Design of Stablecoins Impact Their Volatility?

  2021influential citation
• Semi-industrial production of a minimally processed infant formula powder using membrane filtration.

  2021cites this paper
• Experimental investigation of Bambara nut shell ash in the production of concrete and mortar

  2021cites this paper
• A statistical method for estimating the no-observed-adverse-event-level

  2021cites this paper
• Multivariate Multiple Test Procedures

  2021cites this paper
• Ranks and Pseudo‐ranks—Surprising Results of Certain Rank Tests in Unbalanced Designs

  2020cites this paper
• Robust, multiple change-point detection for covariance matrices using data depth

  2020cites this paper
• Nonparametric MANOVA in meaningful effects

  2020cites this paper
• Statistical evaluation of in-vivo bioassays in regulatory toxicology considering males and females

  2020cites this paper
• Are Faba Bean and Pea Proteins Potential Whey Protein Substitutes in Infant Formulas? An In Vitro Dynamic Digestion Approach

  2020cites this paper
• User Experiences with Two New Wheelchair Securement Systems in Large Accessible Transit Vehicles

  2020cites this paper
• In vitro static digestion reveals how plant proteins modulate model infant formula digestibility.

  2020cites this paper
• ConfigCrusher: towards white-box performance analysis for configurable systems

  2019cites this paper
• Self-reported difficulty and preferences of wheeled mobility device users for simulated low-floor bus boarding, interior circulation and disembarking

  2019cites this paper
• Factorial analyses of treatment effects under independent right-censoring

  2019cites this paper
• Wild bootstrapping rank-based procedures: Multiple testing in nonparametric factorial repeated measures designs

  2019cites this paper
• Soil compaction and stress propagation after different wheeling intensities on a silt soil in South-East Norway

  2019cites this paper
• In-stream microhabitat mosaic depicts the success of mitigation measures and controls the Ecological Potential of benthic communities in heavily modified rivers.

  2019cites this paper
• Testing in nonparametric ANCOVA model based on ridit reliability functional

  2019cites this paper
• Nonparametric multiple contrast tests for general multivariate factorial designs

  2019cites this paper
• Multilevel Petri net‐based ticket assignment and IT management for improved IT organization support

  2019cites this paper
• An Assessment Scheme for Soil Degradation Caused by Forestry Machinery on Skid Trails in Germany

  2019cites this paper
• Robust multiple comparisons against a control group with application in toxicology

  2019influential citation
• ConfigCrusher: White-Box Performance Analysis for Configurable Systems

  2019cites this paper
• Nonparametric multiple comparisons

  2019cites this paper
• The nonparametric Behrens‐Fisher problem with dependent replicates

  2019cites this paper
• Biomarkers of Hand Osteoarthritis Are Detectable after Mechanical Exercise

  2019cites this paper
• Companies’ Participation in OSS Development–An Empirical Study of OpenStack

  2019cites this paper
• Challenges of end-user programmers : Reflections from two groups of end-users

  2019cites this paper
• Phosphorus attenuation in streams by water-column geochemistry and benthic sediment reactive iron

  2019cites this paper
• Nonparametric MANOVA in meaningful effects

  2019cites this paper
• Ranks and Pseudo-Ranks - Paradoxical Results of Rank Tests -

  2018cites this paper
• Testing in nonparametric ANCOVA model based on ridit reliability functional

  2018influential citation
• Empowering OCL research: a large-scale corpus of open-source data from GitHub

  2018cites this paper
• Power Analysis of Collapsed Ordered Categories with Application to Cancer Data

  2018cites this paper
• Application of a MALDI-TOF analysis platform (ClinProTools) for rapid and preliminary report of MRSA sequence types in Taiwan

  2018cites this paper
• Interface protocol inference to aid understanding legacy software components

  2018cites this paper
• The DSL/Model co-evolution problem in industrial MDE ecosystems

  2018cites this paper
• In vivo osteoconductivity of surface modified Ti-29Nb-13Ta-4.6Zr alloy with low dissolution of toxic trace elements

  2018cites this paper
• How Swift Developers Handle Errors

  2018cites this paper
• Scientific Program Committee

  2018cites this paper
• Cross-Project Issue Classification Based on Ensemble Modeling in a Social Coding World

  2018cites this paper
• Simultaneous inference for factorial multireader diagnostic trials

  2018cites this paper
• Fitness consequences of plasticity in an extended phenotype

  2018cites this paper
• Continuous Ropivacaine Subfascial Wound Infusion Compared With Intrathecal Morphine for Postcesarean Analgesia: A Prospective, Randomized Controlled, Double-Blind Study

  2017cites this paper
• Effects of transit bus interior configuration on performance of wheeled mobility users during simulated boarding and disembarking.

  2017cites this paper
• How do Scratch Programmers Name Variables and Procedures?

  2017cites this paper
• On negative results when using sentiment analysis tools for software engineering research

  2017influential citation
• Nonparametric MANOVA in Mann-Whitney effects

  2017cites this paper
• An Empirical Study on the Maturity of the Eclipse Modeling Ecosystem

  2017cites this paper
• Reviewing Career Paths of the OpenStack Developers

  2017cites this paper
• Mining Developer Behavior Across GitHub and StackOverflow

  2017cites this paper
• Bootstrap- and permutation-based inference for the Mann–Whitney effect for right-censored and tied data

  2017cites this paper