SCIENCE@home

The Reproducibility of Economics Research: A Case Study

Sylvérie Herbert,Hautahi Kingi,Flavio Stanchi,L. Vilhuber

Published 2023 in Social Science Research Network

ABSTRACT

Given the importance of reproducibility for the scientific ethos, more and more journals have pushed for transparency of research through data availability policies. If the introduction and implementation of such data policies improve the availability of researchers' code and data, what is the impact on reproducibility? We describe and present the results of a large reproduction exercise in which we assess the reproducibility of research articles published in the American Economic Journal: Applied Economics , which has implemented a data availability policy since 2005. Our replication success rate is relatively moderate, with 37.78% of replication attempts successful. 68 of 162 eligible replication attempts successfully replicated the article's analysis (41.98%) conditional on non-confidential data. A further 69 (42.59%) were at least partially successful. A total of 98 out of 303 (32.34%) relied on confidential or proprietary data, and were thus not reproducible by this project. We also conduct several bibliometric analyses of reproducible vs. non-reproducible articles and show that replicable papers do not provide citation bonuses for authors. reach Our

PUBLICATION RECORD

  • Publication year

    2023

  • Venue

    Social Science Research Network

  • Publication date

    Unknown publication date

  • Fields of study

    Not labeled

  • Identifiers
    DOI 10.2139/ssrn.4325149
  • External record

    Open on Semantic Scholar

  • Source metadata

    Semantic Scholar

CITATION MAP

EXTRACTION MAP

CLAIMS

  • No claims are published for this paper.

CONCEPTS

  • No concepts are published for this paper.

REFERENCES

Showing 1-35 of 35 references · Page 1 of 1

CITED BY

Showing 1-9 of 9 citing papers · Page 1 of 1