Performance criteria-based effect size (PCES) measurement of single-case experimental designs: A real-world data study.

Visual analysis and nonoverlap-based effect sizes are predominantly used in analyzing single case experimental designs (SCEDs). Although they are popular analytical methods for SCEDs, they have certain limitations. In this study, a new effect size calculation model for SCEDs, named performance criteria-based effect size (PCES), is proposed considering the limitations of 4 nonoverlap-based effect size measures, widely accepted in the literature and that blend well with visual analysis. In the field test of PCES, actual data from published studies were utilized, and the relations between PCES, visual analysis, and the 4 nonoverlap-based methods were examined. In determining the data to be used in the field test, 1,052 tiers (AB phases) were identified from 6 journals. The results revealed a weak or moderate relation between PCES and nonoverlap-based methods due to its focus on performance criteria. Although PCES has some weaknesses, it promises to eliminate the causes that may create issues in nonoverlap-based methods, using quantitative data to determine socially important changes in behavior and to complement visual analysis.

• Publication year

  2021

• Venue

  Journal of Applied Behavior Analysis

• Publication date

  2021-08-12

• Fields of study

  Medicine, Education

• Identifiers
  DOI 10.31234/osf.io/dcnjzPMID 35593661
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• A Proof of Concept Analysis of Decision-Making with Time-Series Data

  2021cited by this paper
• Further Analysis of Advanced Quantitative Methods and Supplemental Interpretative Aids with Single-Case Experimental Designs

  2021cited by this paper
• Accurate models vs. accurate estimates: A simulation study of Bayesian single-case experimental designs

  2021cited by this paper
• The Power to Explain Variability in Intervention Effectiveness in Single-Case Research Using Hierarchical Linear Modeling

  2021cited by this paper
• Meta-analyses and effect sizes in applied behavior analysis: A review and discussion.

  2021cited by this paper
• Quantitative Techniques and Graphical Representations for Interpreting Results from Alternating Treatment Design

  2021cited by this paper
• Mastery Criteria and Maintenance: a Descriptive Analysis of Applied Research Procedures

  2020cited by this paper
• Indexing effects in single-case experimental designs by estimating the percent of goal obtained

  2020cited by this paper
• A primer for using multilevel models to meta-analyze single case design data with AB phases.

  2020cited by this paper
• Single-case experimental designs: Characteristics, changes, and challenges.

  2020cited by this paper
• Studies Comparing Augmentative and Alternative Communication Systems (AAC) Applications for Individuals with Autism Spectrum Disorder: A Systematic Review and Meta-Analysis

  2020cited by this paper
• A Bayesian Rate Ratio Effect Size to Quantify Intervention Effects for Count Data in Single Case Experimental Research

  2020cited by this paper
• From Boulder to Stockholm in 70 Years: Single Case Experimental Designs in Clinical Research

  2020cited by this paper
• How Many Tiers Do We Need? Type I Errors and Power in Multiple Baseline Designs

  2020cited by this paper
• Comparing Video Modeling to Teacher-Led Modeling for Algebra Instruction with Students with Learning Disabilities

  2020cited by this paper
• A Priori Justification for Effect Measures in Single-Case Experimental Designs

  2020cited by this paper
• Assessing Consistency in Single-Case A-B-A-B Phase Designs

  2020cited by this paper
• Validity and Reliability Analysis of the PlotDigitizer Software Program for Data Extraction from Single-Case Graphs

  2020cited by this paper
• Correspondence between Fail-Safe k and Dual-Criteria Methods: Analysis of Data Series Stability

  2020cited by this paper
• Treatment Burst Data Points and Single Case Design Studies: A Bayesian N-of-1 Analysis for Estimating Treatment Effect Size

  2020cited by this paper
• Applying mixed-effects modeling to single-subject designs: An introduction.

  2019cited by this paper
• The effects of different mastery criteria on the skill maintenance of children with developmental disabilities.

  2019cited by this paper
• Randomized Single-Case Experimental Designs in Healthcare Research: What, Why, and How?

  2019cited by this paper
• Awareness training reduces college students' speech disfluencies in public speaking.

  2019cited by this paper
• Accounting for baseline trends in intervention studies: Methods, effect sizes, and software

  2019cited by this paper
• A Visual Aid and Objective Rule Encompassing the Data Features of Visual Analysis

  2019cited by this paper
• A randomization test wrapper for synthesizing single-case experiments using multilevel models: A Monte Carlo simulation study

  2019cited by this paper
• Development of an in-vivo metric to aid visual inspection of single-case design data: Do we need to run more sessions?

  2018cited by this paper
• Single-Case Experimental Designs for Clinical Research and Neurorehabilitation Settings

  2018cited by this paper
• Improving the assertive conversational skills of adolescents with autism spectrum disorder in a natural context

  2018cited by this paper
• An evaluation of the agreement between the conservative dual-criterion method and expert visual analysis.

  2018cited by this paper
• An Evaluation of Behavioral Skills Training for Teaching Caregivers How to Support Social Skill Development in Their Child with Autism Spectrum Disorder

  2018cited by this paper
• A Gradual Effects Model for Single-Case Designs

  2018cited by this paper
• Predictive validity and efficiency of ongoing visual-inspection criteria for interpreting functional analyses.

  2018cited by this paper
• A Preliminary Analysis of Mastery Criterion Level: Effects on Response Maintenance

  2018cited by this paper
• Linear Trend in Single-Case Visual and Quantitative Analyses

  2018cited by this paper
• Computer-based training to detect antecedents and consequences of problem behavior.

  2018cited by this paper
• Development of a Distance-Based Effect Size Metric for Single-Case Research: Ratio of Distances.

  2018cited by this paper
• Interpreting Kendall’s Tau and Tau-U for single-case experimental designs

  2018cited by this paper
• Script fading for children with autism: Generalization of social initiation skills from school to home.

  2018cited by this paper
• Improving the interview skills of college students using behavioral skills training.

  2017cited by this paper
• Introduction to the Special Issue on Single-Case Systematic Reviews and Meta-Analyses

  2017cited by this paper
• Recommendations for Choosing Single-Case Data Analytical Techniques.

  2017cited by this paper
• Methodological quality of meta-analyses of single-case experimental studies.

  2017cited by this paper
• Improving Functional Language and Social Motivation with a Parent-Mediated Intervention for Toddlers with Autism Spectrum Disorder

  2017cited by this paper
• Effects of mastery criterion on the emergence of derived equivalence relations.

  2017cited by this paper
• An Improved Rank Correlation Effect Size Statistic for Single-Case Designs: Baseline Corrected Tau

  2017cited by this paper
• Using response ratios for meta-analyzing single-case designs with behavioral outcomes.

  2017cited by this paper
• A survey of publication practices of single-case design researchers when treatments have small or large effects.

  2016cited by this paper
• Using Video Models to Teach Students With Disabilities to Play the Wii

  2016cited by this paper
• Within-subject analysis of a prevention strategy for problem behavior.

  2016cited by this paper
• Assessing generative braille responding following training in a matching-to-sample format.

  2016cited by this paper
• Using a 3D Immersive Virtual Environment System to Enhance Social Understanding and Social Skills for Children With Autism Spectrum Disorders

  2015cited by this paper
• Examination of the relation between an assessment of skills and performance on auditory-visual conditional discriminations for children with autism spectrum disorder.

  2015cited by this paper
• Adults with autism spectrum disorder as behavior technicians for young children with autism: Outcomes of a behavioral skills training program.

  2015cited by this paper
• Common pitfalls in statistical analysis: Clinical versus statistical significance

  2015cited by this paper
• Comparing Visual and Statistical Analysis in Single-Case Studies Using Published Studies

  2015cited by this paper
• The Role of Between-Case Effect Size in Conducting, Interpreting, and Summarizing Single-Case Research. NCER 2015-002.

  2015cited by this paper
• Interrater Agreement Between Visual Analysts of Single-Case Data

  2015cited by this paper
• Critical Assumptions and Distribution Features Pertaining to Contemporary Single-Case Effect Sizes

  2015cited by this paper
• Evaluating Intervention Effects in Single‐Case Research Designs

  2015cited by this paper
• Evaluating single-case research data for systematic review: a commentary for the special issue.

  2014cited by this paper
• Design-Comparable Effect Sizes in Multiple Baseline Designs

  2014cited by this paper
• Analysis and meta-analysis of single-case designs with a standardized mean difference statistic: a primer and applications.

  2014cited by this paper
• Single-case experimental designs: Reflections on conduct and analysis

  2014cited by this paper
• Incorporating nonoverlap indices with visual analysis for quantifying intervention effectiveness in single-case experimental designs

  2014cited by this paper
• From a single-level analysis to a multilevel analysis of single-case experimental designs.

  2014cited by this paper
• Visual analysis in single case experimental design studies: Brief review and guidelines

  2014cited by this paper
• Statistical Analyses of Single-Case Designs

  2014cited by this paper
• Publication bias in studies of an applied behavior-analytic intervention: an initial analysis.

  2014cited by this paper
• The Three-Level Synthesis of Standardized Single-Subject Experimental Data: A Monte Carlo Simulation Study

  2013cited by this paper
• A standardized mean difference effect size for single case designs

  2012cited by this paper
• Single-case experimental designs: a systematic review of published research and current standards.

  2012cited by this paper
• Combining nonoverlap and trend for single-case research: Tau-U.

  2011cited by this paper
• Characteristics of single-case designs used to assess intervention effects in 2008

  2011cited by this paper
• Randomization Tests

  2011cited by this paper
• A Quantitative Synthesis of Methodology in the Meta-Analysis of Single-Subject Research for Students with Disabilities: 1985–2009

  2011cited by this paper
• Effect Size in Single-Case Research: A Review of Nine Nonoverlap Techniques

  2011cited by this paper
• Lag-one autocorrelation in short series: Estimation and hypotheses testing

  2010cited by this paper
• Comparison of Overlap Methods for Quantitatively Synthesizing Single-Subject Data

  2010cited by this paper
• An improved effect size for single-case research: nonoverlap of all pairs.

  2009cited by this paper
• Single Subject Designs in Biomedicine

  2009cited by this paper
• The Improvement Rate Difference for Single-Case Research

  2009cited by this paper
• Mastery learning and assessment: Implications for students and teachers in an era of high‐stakes testing

  2008cited by this paper
• Hierarchical linear models for the quantitative integration of effect sizes in single-case research

  2003cited by this paper
• Combining single-case experimental data using hierarchical linear models.

  2003cited by this paper
• Design Specification Issues in Time-Series Intervention Models

  2000cited by this paper
• Randomization tests for restricted alternating treatments designs.

  1994cited by this paper
• The statistical versus clinical significance debate.

  1993cited by this paper
• Calculating effect sizes for meta-analysis: the case of the single case.

  1993cited by this paper
• QUANTITATIVE METHODS IN PSYCHOLOGY A Power Primer

  1992cited by this paper
• The Quantitative Synthesis of Single-Subject Research

  1987cited by this paper
• Three methods for analyzing limited time-series (N of 1) data.

  1983cited by this paper
• Social validity: the case for subjective measurement or how applied behavior analysis is finding its heart.

  1978cited by this paper
• Some current dimensions of applied behavior analysis.

  1968cited by this paper

• The Effectiveness of Video Visual Scene Display-Assisted Behavioral Skills Training in the Instruction of Skills to Prevent Online Sexual Abuse Among Adults with Autism Spectrum Disorder

  2025cites this paper
• Augmented Reality in Auditory-visual Conditional Discrimination Training for Children with Autism

  2025cites this paper
• A Meta-Analysis of Applied Behavior Analysis-Based Interventions for Individuals with Autism Spectrum Disorders (ASD) in Turkey

  2024cites this paper
• A Description of Missing Data in Single-Case Experimental Designs Studies and an Evaluation of Single Imputation Methods

  2024influential citation
• Effect Size of General English Modules Development through MPI + R Model on the Lecturer Professionalism

  2024cites this paper
• A Meta-Analysis of Self-Management Interventions in Teaching Daily Living Skills to Autistic Individuals

  2024influential citation
• Analyzing data in single-case experimental designs: Objectives and available software options

  2024cites this paper
• A Meta-analysis of Science Education Studies for Students with Intellectual and Developmental Disabilities (IDD)

  2023influential citation
• Does the choice of a linear trend-assessment technique matter in the context of single-case data?

  2023cites this paper
• Rise of single-case experimental designs: A historical overview of the necessity of single-case methodology

  2023influential citation
• Using self‐monitoring to increase on‐task behaviors of students with attention deficit hyperactivity disorders (ADHD) in inclusive classrooms in Turkey (Türkiye)

  2023cites this paper
• Characteristics of Missing Data in Single-Case Experimental Designs: An Investigation of Published Data

  2023cites this paper
• Defining and assessing immediacy in single‐case experimental designs

  2022cites this paper