Recommendations for Increasing Replicability in Psychology †

Replicability of findings is at the heart of any empirical science. The aim of this article is to move the current replicability debate in psychology towards concrete recommendations for improvement. We focus on research practices but also offer guidelines for reviewers, editors, journal management, teachers, granting institutions, and university promotion committees, highlighting some of the emerging and existing practical solutions that can facilitate implementation of these recommendations. The challenges for improving replicability in psychological science are systemic. Improvement can occur only if changes are made at many levels of practice, evaluation, and reward. Copyright © 2013 John Wiley & Sons, Ltd.

• Publication year

  2013

• Venue

  European Journal of Personality

• Publication date

  2013-03-01

• Fields of study

  Psychology

• Identifiers
  DOI 10.1002/per.1919
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Data from Paper “False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant”

  2014cited by this paper
• Positive results receive more citations, but only in some disciplines

  2013cited by this paper
• Journal of Personality and Social Psychology Interpersonal and Genetic Origins of Adult Attachment Styles : A Longitudinal Study From Infancy to Early Adulthood

  2013cited by this paper
• Rejoice! In Replication

  2013cited by this paper
• Learned Non-Rigid Object Motion is a View-Invariant Cue to Recognizing Novel Objects

  2012cited by this paper
• The Time Has Come : Bayesian Methods for Data Analysis in the Organizational Sciences

  2012cited by this paper
• Scientific Utopia … or Too Much Information? Comment on Nosek and Bar-Anan

  2012cited by this paper
• An Open, Large-Scale, Collaborative Effort to Estimate the Reproducibility of Psychological Science

  2012cited by this paper
• Behavioral Priming: It's All in the Mind, but Whose Mind?

  2012cited by this paper
• Replication Unreliability in Psychology: Elusive Phenomena or “Elusive” Statistical Power?

  2012cited by this paper
• Flawed science: The fraudulent research practices of social psychologist Diederik Stapel

  2012cited by this paper
• Replication studies: Bad copy

  2012cited by this paper
• The Time Has Come

  2012cited by this paper
• An Agenda for Purely Confirmatory Research

  2012cited by this paper
• Adult Attachment Orientations, Stress, and Romantic Relationships

  2012cited by this paper
• Research grants: Conform and be funded

  2012cited by this paper
• The Long Way From α-Error Control to Validity Proper

  2012cited by this paper
• Correlation and Causation in the Study of Personality

  2012cited by this paper
• A 21 Word Solution

  2012cited by this paper
• IN THE WAKE OF HIGH-PROFILE CONTROVERSIES, PSYCHOLOGISTS ARE FACING UP TO PROBLEMS WITH REPLICATION.

  2012cited by this paper
• Comprar Developmental Psychology | Mary Gauvain | 9780077126162 | MCGRAW HILL

  2012cited by this paper
• Correcting the past: failures to replicate ψ.

  2012cited by this paper
• Psychologists Are Open to Change, yet Wary of Rules

  2012cited by this paper
• Scientific Utopia

  2012cited by this paper
• Short, Sweet, and Problematic? The Rise of the Short Report in Psychological Science

  2012cited by this paper
• The ironic effect of significant results on the credibility of multiple-study articles.

  2012influential reference
• The Rules of the Game Called Psychological Science

  2012cited by this paper
• Replications in Psychology Research

  2012cited by this paper
• Scientific Utopia: II - Restructuring Incentives and Practices to Promote Truth Over Publishability

  2012cited by this paper
• Nobel laureate challenges psychologists to clean up their act

  2012influential reference
• Teaching Replication

  2012cited by this paper
• Treating stimuli as a random factor in social psychology: a new and comprehensive solution to a pervasive but largely ignored problem.

  2012cited by this paper
• Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling

  2012cited by this paper
• Why Science Is Not Necessarily Self-Correcting

  2012cited by this paper
• Rewarding Replications

  2012cited by this paper
• Replication in Prevention Science

  2011cited by this paper
• Journal of Personality and Social Psychology Why Psychologists Must Change the Way They Analyze Their Data : The Case of Psi : Comment on Bem ( 2011 )

  2011cited by this paper
• From perceptual rags to metaphoric riches--bodily, social, and cultural constraints on sociocognitive metaphors: comment on Landau, Meier, and Keefer (2010).

  2011cited by this paper
• Tracking Replicability as a Method of Post-Publication Open Evaluation

  2011cited by this paper
• Fearing the Future of Empirical Psychology: Bem's (2011) Evidence of Psi as a Case Study of Deficiencies in Modal Research Practice

  2011cited by this paper
• Willingness to Share Research Data Is Related to the Strength of the Evidence and the Quality of Reporting of Statistical Results

  2011cited by this paper
• Feeling the future: experimental evidence for anomalous retroactive influences on cognition and affect.

  2011cited by this paper
• Negative results are disappearing from most disciplines and countries

  2011cited by this paper
• Comprar Handbook of research methods in personality psychology | Robert F Krueger | 9781606236123 | Guilford Press

  2010cited by this paper
• “Positive” Results Increase Down the Hierarchy of the Sciences

  2010influential reference
• VU Research Portal From perceptual rags to metaphoric riches: Bodily, social, and cultural constraints on socio-cognitive metaphors

  2010cited by this paper
• Weird people, yes, but also weird experiments

  2010cited by this paper
• Mozart effect–Shmozart effect: A meta-analysis

  2010cited by this paper
• Single-Case Research Designs: Methods for Clinical and Applied Settings

  2010cited by this paper
• Evaluating correlations in studies of personality and behavior: Beyond the number of significant findings to be expected by chance

  2009cited by this paper
• Multitrait-multimethod analysis in psychotherapy research: New methodological approaches

  2009cited by this paper
• College Sophomores in the Laboratory Redux: Influences of a Narrow Data Base on Social Psychology's View of the Nature of Prejudice

  2008cited by this paper
• Things I have learned so far

  2008cited by this paper
• Sample size planning for statistical power and accuracy in parameter estimation.

  2008cited by this paper
• Writing an empirical article

  2008cited by this paper
• Structural equation modeling of multitrait-multimethod data: different models for different types of methods.

  2008cited by this paper
• How Well Do Criminologists Explain Crime? Statistical Modeling in Published Studies

  2008cited by this paper
• Distributional differences of the impact factor in the sciences versus the social sciences: An analysis of the probabilistic structure of the 2005 journal citation reports

  2008cited by this paper
• Experimental Test of qRule Theory

  2008cited by this paper
• The null hypothesis significance-testing debate and its implications for personality research.

  2007cited by this paper
• Handbook of research methods in personality psychology

  2007cited by this paper
• Dyadic Data Analysis

  2007cited by this paper
• Publication, Publication

  2006cited by this paper
• The poor availability of psychological research data for reanalysis.

  2006cited by this paper
• Automatic social behavior as motivated preparation to interact.

  2006cited by this paper
• Sample size planning for the standardized mean difference: accuracy in parameter estimation via narrow confidence intervals.

  2006influential reference
• Social tuning of automatic racial attitudes: the role of affiliative motivation.

  2005cited by this paper
• Why Most Published Research Findings Are False

  2005cited by this paper
• Inference by eye: confidence intervals and how to read pictures of data.

  2005cited by this paper
• Psychology's Status as a Scientific Discipline: Its Empirical Placement within an Implicit Hierarchy of the Sciences

  2004cited by this paper
• Social Cognition: How Individuals Construct Social Reality

  2004cited by this paper
• The persistence of underpowered studies in psychological research: causes, consequences, and remedies.

  2004cited by this paper
• The Mozart effect: tracking the evolution of a scientific legend.

  2004cited by this paper
• A farewell to Bonferroni: the problems of low statistical power and publication bias

  2004cited by this paper
• Sample size for multiple regression: obtaining regression coefficients that are accurate, not simply significant.

  2003influential reference
• Scientific creativity as constrained stochastic behavior: the integration of product, person, and process perspectives.

  2003cited by this paper
• The nonconsciousness of self-consciousness.

  2002cited by this paper
• United States Patent and Trademark Office

  2002cited by this paper
• From Ivory Tower to Urban Street: Using the Classroom as a Community Research and Development Tool

  2001cited by this paper
• That "vision thing": the state of theory in social and personality psychology at the edge of the new millennium.

  2001cited by this paper
• Introduction to Social Psychology

  2001cited by this paper
• Guide to Publishing in Psychology Journals: Writing an Empirical Article

  2000cited by this paper
• Null hypothesis significance testing: a review of an old and continuing controversy.

  2000cited by this paper
• Clinical significance not statistical significance: a simple Bayesian alternative to p values.

  1998cited by this paper
• Design sensitivity: Statistical power for applied experimental research.

  1998cited by this paper
• HARKing: Hypothesizing After the Results are Known

  1998cited by this paper
• Automaticity of social behavior: direct effects of trait construct and stereotype-activation on action.

  1996cited by this paper
• Controlling the false discovery rate: a practical and powerful approach to multiple testing

  1995cited by this paper
• Replication, Replication

  1995cited by this paper
• WaveLab and Reproducible Research

  1995cited by this paper
• An Experimental Test of “The Mozart Effect”: Does Listening to His Music Improve Spatial Ability?

  1995cited by this paper
• Oh Monsieur Pasteur, We Hardly Knew You!

  1995cited by this paper
• The earth is round (p < .05)

  1994cited by this paper
• The Social Psychology of Mental Health: Basic Mechanisms and Applications.

  1993cited by this paper
• A meta-analysis of infant habituation and recognition memory performance as predictors of later IQ.

  1993cited by this paper
• Music and spatial task performance

  1993cited by this paper
• The principles of multiple, nonadditive, and reciprocal determinism: Implications for social psychological research and levels of analysis.

  1992cited by this paper
• The Structuralist program in psychology : foundations and applications

  1992cited by this paper
• Neonatal imitation*1

  1991cited by this paper
• Speech Disfluency and the Structure of Knowledge

  1991cited by this paper

• Acute Stress Impairs Adaptation and Promotes Perseverative Choices in Social Decision-Making during the Ultimatum Game

  2026cites this paper
• Measurement-Reporting Practices in Social-and-Personality-Psychology Articles

  2026cites this paper
• Examining fifty years of replication studies in management and economics: a retrospective Scientometric-Bayesian multilevel analysis

  2026cites this paper
• The survival processing advantage in memory using virtual reality versus traditional desktop display: Does it make a difference?

  2026cites this paper
• Dyslexia Research and Replicability: Should We Be Worried?

  2025cites this paper
• Conducting replication in translation and interpreting studies

  2025cites this paper
• A Review of the Performance Metrics and Entrepreneurial Practices of Economics and Business Departments in UK Universities: A ‘Gresham’s Law’ Threat?

  2025cites this paper
• Replication, a Hallmark of Good Science: Unraveling the Factors That Predict Replication Success

  2025cites this paper
• “Clothe yourselves with humility”: Humility can promote fairness

  2025cites this paper
• Key trends, challenges, and opportunities in scientific journal management between 2013 and 2023: a systematic review

  2025cites this paper
• Testing the Information Distortion Model for Turkish Students’ Self-Concept and Interest in Science

  2025cites this paper
• Let’s talk about language—and its role for replicability

  2025cites this paper
• Validation of an online imitation-inhibition task

  2025cites this paper
• Going off script: Exploring the reporting of preregistration deviations in Industrial and Organizational Psychology and their relationship with questionable research practices

  2025cites this paper
• A Systematic Review on the Evolution of Power Analysis Practices in Psychological Research

  2025cites this paper
• Incentives and the replication crisis in social sciences: A critical review of open science practices

  2025cites this paper
• Applying n -gram-based evaluation metrics to assess human interpreting

  2025cites this paper
• Statistical power analysis in exercise science

  2025cites this paper
• Exploring the Influence of Empathy on the Relationship Between Communal Narcissism and Altruistic Behavior

  2025cites this paper
• Gender disparities in publishing: how networks, occupational self-efficacy and the university shape the gender publication gap among professors in Germany

  2025cites this paper
• Does competition from informal firms impact new product development? Systematic replication and extension of McCann and Bahl (2017)

  2025cites this paper
• One Decade Into the Replication Crisis, How Have Psychological Results Changed?

  2025cites this paper
• Sustainable and Responsible Consumption: An Investigation of Consumer Motivations for Adopting a Capsule Wardrobe

  2025cites this paper
• A scoping review of the Boxall Profile: a tool to assess and respond to students’ social, emotional and mental health needs

  2025cites this paper
• Kriterien zur Sicherstellung der Reproduzierbarkeit quantitativ-empirischer Studien: Empfehlungen und Positivbeispiele

  2025cites this paper
• How do pressure impact attention control in precision sports: attentional engagement or attentional disengagement?

  2024cites this paper
• The quantitative paradigm and the nature of the human mind. The replication crisis as an epistemological crisis of quantitative psychology in view of the ontic nature of the psyche

  2024cites this paper
• Uncertainty limits the use of power analysis.

  2024cites this paper
• Effect size benchmarks: Time for a causal renaissance

  2024cites this paper
• Not Twisted, Just Kinky: Replication and Structural Invariance of Attachment, Personality, and Well-Being Among BDSM Practitioners

  2024cites this paper
• Pinning down state authenticity: defining and validating a state authenticity measure

  2024cites this paper
• Irritability and stress: Prospective, bidirectional relationships in adolescence.

  2024cites this paper
• Chemical signatures in the preen oil of pied flycatchers: testing reproducibility and exploring ontogeny

  2024cites this paper
• Uncertainty, expertise, and persuasion: A replication and extension of

  2024cites this paper
• Replicating What Motivates Conflicting Groups to Engage in Competitive Victimhood: The Roles of Need for Power and Need for Morality

  2024cites this paper
• On the role of variation in measures, the worth of underpowered studies, and the need for tolerance among researchers: Some more reflections on Leising et al. from a methodological, statistical, and social-psychological perspective

  2024cites this paper
• Understanding, fast and shallow: Individual differences in memory performance associated with cognitive load predict the illusion of explanatory depth

  2024cites this paper
• Do Health Workers Believe that Personal Recovery is Possible? An Evaluation of the Recovery Attitudes Questionnaire

  2024cites this paper
• The Limits of Empiricism: A Critique of Data-Driven Theory Development

  2024cites this paper
• Optimal processing of surface facial EMG to identify emotional expressions: A data-driven approach

  2024cites this paper
• Validation of the Brief Index of Sexual Functioning for women and men (BISF-W and BISF-M) in an Italian sample

  2024cites this paper
• The roles of sensory hyperreactivity and hyporeactivity in understanding infant fearfulness and emerging autistic traits.

  2024cites this paper
• Toward more robust and integrative research on learning and individual differences: Systematic literature reviews with and without meta-analysis

  2024cites this paper
• Provide proactive reproducible analysis transparency with every publication

  2024cites this paper
• Profit motives, environmental motives, and perceived corporate greenwashing revisited: A replication and extension of de Vries et al. (2015)

  2024cites this paper
• Show me How You Use Your Mouse and I Tell You How You Feel? Sensing Affect With the Computer Mouse

  2024cites this paper
• Subjective logic as a complementary tool to meta-analysis to explicitly address second-order uncertainty in research findings: A case from infant studies.

  2024cites this paper
• Is replication possible in qualitative research? A response to Makel et al. (2022)

  2024cites this paper
• Articulatory rehearsal and phonological storage in working memory: A replication of Longoni et al. (1993)

  2024cites this paper
• The effects of coaches' pride and shame expressions on field hockey players' emotions and performance.

  2024cites this paper
• Cross-cultural variation in experiences of acceptance, camouflaging and mental health difficulties in autism: A registered report

  2024cites this paper
• A tentative roadmap for consensus building processes

  2024cites this paper
• Effects of Preferred versus Nonpreferred Music on Bench Press Performance

  2024influential citation
• A discipline-wide investigation of the replicability of Psychology papers over the past two decades

  2023cites this paper
• The institutional impact of research challenges and constraints on psychology and other social and behavioral sciences

  2023cites this paper
• Questioning Psychological Constructs: Current Issues and Proposed Changes

  2023cites this paper
• Evaluating the Pedagogical Effectiveness of Study Preregistration in the Undergraduate Dissertation

  2023cites this paper
• Combining support for hypotheses over heterogeneous studies with Bayesian Evidence Synthesis: A simulation study

  2023cites this paper
• Growth From Uncertainty: Understanding the Replication ‘Crisis’ in Infant Cognition

  2023cites this paper
• Current reproducibility practices in management: What they are versus what they could be

  2023influential citation
• Publication bias, statistical power and reporting practices in the Journal of Sports Sciences: potential barriers to replicability

  2023cites this paper
• Assessing the Evidential Value of Mental Fatigue and Exercise Research

  2023cites this paper
• How the Complexity of Psychological Processes Reframes the Issue of Reproducibility in Psychological Science

  2023cites this paper
• Sixty Years Later: A Replication Study of McGuire's First Inoculation Experiment

  2023cites this paper
• THREATS TO OBJECTIVITY IN THE SOCIAL SCIENCE RESEARCH

  2023cites this paper
• Towards reproducible and respectful autism research: Combining open and participatory autism research practices

  2023cites this paper
• Letter labels and illusory correlation: infrequent letters bias reactions to the group

  2023cites this paper
• A systematic review of null hypothesis significance testing, sample sizes, and statistical power in research using the Implicit Relational Assessment Procedure

  2023cites this paper
• We are all less risky and more skillful than our fellow drivers: Successful replication and extension of Svenson (1981)‎

  2023cites this paper
• Doctored photographs create false memories of spectacular childhood events. a replication of Wade et al. (2002) with a Scandinavian twist

  2023cites this paper
• Differences between physical therapist ratings, self-ratings, and posturographic measures when assessing static balance exercise intensity

  2023cites this paper
• To err is human- to understand error-processing is divine: Contributions of working memory and anxiety to error-related brain and pupil responses.

  2023cites this paper
• The trustworthiness of the cumulative knowledge in industrial/organizational psychology: The current state of affairs and a path forward.

  2023cites this paper
• Multiple intelligences and job performance: the moderating role of learning goal oriented training

  2023cites this paper
• Unfortunately, Journals in Industrial, Work, and Organizational Psychology Still Fail to Support Open Science Practices

  2023cites this paper
• In models we trust: preregistration, large samples, and replication may not suffice

  2023cites this paper
• State Space Mixture Modeling: Finding People with Similar Patterns of Change

  2023cites this paper
• “Good Night, Sleep Tight”: Do we have an “Evolutionary Preference” for Placing Beds in Sleeping Rooms? A Replication and Extension of Spörrle and Stich (2010)

  2023cites this paper
• Managerial stereotypes over time: the rise of feminine leadership

  2023cites this paper
• How replication studies can improve doctoral student education

  2023cites this paper
• Questionable Metascience Practices

  2023cites this paper
• Low Research-Data Availability in Educational-Psychology Journals: No Indication of Effective Research-Data Policies

  2023cites this paper
• Dehumanization of women and men in elective abortion: A preregistered replication in China

  2023cites this paper
• How well are open science practices implemented in industrial and organizational psychology and management?

  2023cites this paper
• The inside out model of emotion recognition: how the shape of one’s internal emotional landscape influences the recognition of others’ emotions

  2023cites this paper
• Relating utterance fluency to perceived fluency of interpreting

  2022cites this paper
• Replication concerns in sports and exercise science: a narrative review of selected methodological issues in the field

  2022influential citation
• Evaluating replicability of ten influential research on sensory marketing

  2022cites this paper
• Strong-Form Frequentist Testing In Communication Science: Principles, Opportunities, And Challenges

  2022cites this paper
• spectre: an R package to estimate spatially‐explicit community composition using sparse data

  2022cites this paper
• How does social complexity facilitate coping flexibility? The mediating role of dialectical thinking

  2022cites this paper
• The Reproducibility Movement in Psychology: Does Researcher Gender Affect How People Perceive Scientists With a Failed Replication?

  2022cites this paper
• Functional Connectivity of the Chemosenses: A Review

  2022cites this paper
• Accelerating Psychological Science With Metastudies: A Demonstration Using the Risky-Choice Framing Effect

  2022cites this paper
• International differences in math and science tilts: The stability, geography, and predictive power of tilt for economic criteria

  2022cites this paper
• Theoretical false positive psychology

  2022cites this paper
• Personality Heterogeneity in Adolescents with Disruptive Behavior Disorders: A Conceptual Replication Study

  2022cites this paper
• Income moderates changes in big-five personality traits across eighteen years

  2022cites this paper
• How self-awareness is connected to less experience of action crises in personal goal pursuit

  2022cites this paper
• Computational modeling of threat learning reveals links with anxiety and neuroanatomy in humans

  2022cites this paper