To Check or Not to Check? A Comment on the Contemporary Psychometrics (ConPsy) Checklist for the Analysis of Questionnaire Items

In a recent paper, the first version of the contemporary psychometrics (ConPsy) checklist for assessing measurement tool quality has been published. This checklist aims to provide guidelines and references to researchers to assess measurement properties for newly developed measurement instruments. The ConPsy checklist recommends appropriate statistical methods for measurement instrument evaluation to guide researchers in instrument development and to support peer review. In this opinion article, I critically review some aspects of the checklist and question the usefulness of certain psychometric analyses in research practice.

• Publication year

  2023

• Venue

  European Journal of Investigation in Health, Psychology and Education

• Publication date

  2023-10-01

• Fields of study

  Medicine, Psychology

• Identifiers
  DOI 10.3390/ejihpe13100151PMID 37887152PMCID 10606083
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• Direct Discrepancy Dynamic Fit Index Cutoffs for Arbitrary Covariance Structure Models

  2024cited by this paper
• MIsgivings about measurement invariance

  2024cited by this paper
• Linking Error in the 2PL Model

  2023cited by this paper
• Modeling Model Misspecification in Structural Equation Models

  2023cited by this paper
• Regularized Generalized Logistic Item Response Model

  2023cited by this paper
• Applied Psychometrics: Estimator Considerations in Commonly Encountered Conditions in CFA, SEM, and EFA Practice

  2023cited by this paper
• Why Full, Partial, or Approximate Measurement Invariance Are Not a Prerequisite for Meaningful and Valid Group Comparisons

  2023cited by this paper
• Modeling Misspecification as a Parameter in Bayesian Structural Equation Models

  2023cited by this paper
• Model-Robust Estimation of Multiple-Group Structural Equation Models

  2023cited by this paper
• Development and psychometric evaluation of the Man-Made Disaster-Related Distress Scale (MMDS)

  2023cited by this paper
• Essays on Contemporary Psychometrics

  2023cited by this paper
• Dynamic fit index cutoffs for categorical factor analysis with Likert-type, ordinal, or binary responses.

  2023cited by this paper
• Why we need to abandon fixed cutoffs for goodness-of-fit indices: An extensive simulation and possible solutions

  2023cited by this paper
• Estimation Methods of the Multiple-Group One-Dimensional Factor Model: Implied Identification Constraints in the Violation of Measurement Invariance

  2022cited by this paper
• Comparing the Robustness of the Structural after Measurement (SAM) Approach to Structural Equation Modeling (SEM) against Local Model Misspecifications with Alternative Estimation Approaches

  2022cited by this paper
• Factor analyzing ordinal items requires substantive knowledge of response marginals.

  2022cited by this paper
• On the Bias in Confirmatory Factor Analysis When Treating Discrete Variables as Ordinal Instead of Continuous

  2022cited by this paper
• Some thoughts on analytical choices in the scaling model for test scores in international large-scale assessment studies

  2022cited by this paper
• A Test Can Have Multiple Reliabilities

  2021cited by this paper
• Power Analysis for the Wald, LR, Score, and Gradient Tests in a Marginal Maximum Likelihood Framework: Applications in IRT

  2021cited by this paper
• Robust and Nonrobust Linking of Two Groups for the Rasch Model with Balanced and Unbalanced Random DIF: A Comparative Simulation Study and the Simultaneous Assessment of Standard Errors and Linking Errors with Resampling Techniques

  2021cited by this paper
• Why Ordinal Variables Can (Almost) Always Be Treated as Continuous Variables: Clarifying Assumptions of Robust Continuous and Ordinal Factor Analysis Estimation Methods

  2020cited by this paper
• A Comparison of Penalized Maximum Likelihood Estimation and Markov Chain Monte Carlo Techniques for Estimating Confirmatory Factor Analysis Models With Small Sample Sizes

  2020cited by this paper
• Psychometric Theory

  2020cited by this paper
• Dynamic fit index cutoffs for confirmatory factor analysis models.

  2020cited by this paper
• The Trade-Off between Model Fit, Invariance, and Validity: The Case of PISA Science Assessments

  2020cited by this paper
• Psychometrics is not measurement: Unraveling a fundamental misconception in quantitative psychology and the complex network of its underlying fallacies.

  2020cited by this paper
• Pernicious Polychorics: The Impact and Detection of Underlying Non-normality

  2019cited by this paper
• Applied Psychometrics: Sample Size and Sample Power Considerations in Factor Analysis (EFA, CFA) and SEM in General

  2018cited by this paper
• Thanks coefficient alpha, we'll take it from here.

  2017cited by this paper
• Misconceptions of Measurement Equivalence

  2016cited by this paper
• A Psychometric Framework for the Evaluation of Instructional Sensitivity

  2016cited by this paper
• Monitoring Countries in a Changing World: A New Look at DIF in International Surveys

  2016cited by this paper
• Quantifying Adventitious Error in a Covariance Structure as a Random Effect

  2015cited by this paper
• Reliability, Dimensionality, and Internal Consistency as Defined by Cronbach: Distinct Albeit Related Concepts

  2015cited by this paper
• Sample Size Determination Within the Scope of Conditional Maximum Likelihood Estimation with Special Focus on Testing the Rasch Model

  2015cited by this paper
• Reflective measurement models, behavior domains, and common causes

  2013cited by this paper
• When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions.

  2012cited by this paper
• Impact of Measurement Invariance on Construct Correlations, Mean Differences, and Relations With External Correlates

  2011cited by this paper
• Effect size indices for analyses of measurement equivalence: understanding the practical importance of differences between groups.

  2011cited by this paper
• Masking misfit in confirmatory factor analysis by increasing unique variances: a cautionary note on the usefulness of cutoff values of fit indices.

  2011cited by this paper
• Instructional Sensitivity as a Psychometric Property of Assessments

  2010cited by this paper
• An Empirical Bayesian Approach to Misspecified Covariance Structures

  2010cited by this paper
• Measurement Error: Models, Methods, and Applications

  2010cited by this paper
• Understanding Measurement: Reliability

  2010cited by this paper
• Generalizability Theory and Classical Test Theory

  2010cited by this paper
• Reliability of Summed Item Scores Using Structural Equation Modeling: An Alternative to Coefficient Alpha

  2009cited by this paper
• Foundations of Factor Analysis

  2009cited by this paper
• Measurement invariance: Review of practice and implications

  2008cited by this paper
• MEASUREMENT ERROR MODELS

  2007cited by this paper
• Measurement error in nonlinear models: a modern perspective

  2006cited by this paper
• The α and the ω of Congeneric Test Theory: An Extension of Reliability and Internal Consistency to Heterogeneous Tests

  2005cited by this paper
• A Beginner's Guide to Structural Equation Modeling

  2005cited by this paper
• My Current Thoughts on Coefficient Alpha and Successor Procedures

  2004cited by this paper
• An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data.

  2004cited by this paper
• Behavior Domains in Theory and in Practice

  2003cited by this paper
• The Resolution of Some Paradoxes Related to Reliability and Validity

  2003cited by this paper
• Is More Ever Too Much? The Number of Indicators per Factor in Confirmatory Factor Analysis.

  1998cited by this paper
• Can Validity Rise When Reliability Declines

  1997cited by this paper
• A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTF as well as item bias/DIF

  1993cited by this paper
• Scale Development: Theory and Applications.

  1992cited by this paper
• A nonparametric approach for assessing latent trait unidimensionality

  1987cited by this paper
• A Sampling Model for Validity

  1982cited by this paper
• Generalizability in Factorable Domains: "Domain Validity and Generalizability"1

  1978cited by this paper
• Probabilistic foundations for coefficients of generalizability

  1968cited by this paper
• Coefficient alpha and the internal structure of tests

  1951cited by this paper

• A Note on Using Scale Sum Scores in Path Analysis

  2026cites this paper
• Italian Validation and Psychometric Properties of the New Work Values Scale

  2025cites this paper