Peer-reviewed publications are the primary mechanism for sharing scientific results. The current peer-review process is, however, fraught with many problems that undermine the pace, validity, and credibility of science. We highlight five salient problems: (1) reviewers are expected to have comprehensive expertise; (2) reviewers do not have sufficient access to methods and materials to evaluate a study; (3) reviewers are neither identified nor acknowledged; (4) there is no measure of the quality of a review; and (5) reviews take a lot of time, and once submitted cannot evolve. We propose that these problems can be resolved by making the following changes to the review process. Distributing reviews to many reviewers would allow each reviewer to focus on portions of the article that reflect the reviewer's specialty or area of interest and place less of a burden on any one reviewer. Providing reviewers materials and methods to perform comprehensive evaluation would facilitate transparency, greater scrutiny, and replication of results. Acknowledging reviewers makes it possible to quantitatively assess reviewer contributions, which could be used to establish the impact of the reviewer in the scientific community. Quantifying review quality could help establish the importance of individual reviews and reviewers as well as the submitted article. Finally, we recommend expediting post-publication reviews and allowing for the dialog to continue and flourish in a dynamic and interactive manner. We argue that these solutions can be implemented by adapting existing features from open-source software management and social networking technologies. We propose a model of an open, interactive review system that quantifies the significance of articles, the quality of reviews, and the reputation of reviewers.
Learning from open source software projects to improve scientific review
Sulagna Dia Ghosh,A. Klein,B. Avants,K. Millman
Published 2012 in Frontiers in Computational Neuroscience
ABSTRACT
PUBLICATION RECORD
- Publication year
2012
- Venue
Frontiers in Computational Neuroscience
- Publication date
2012-04-01
- Fields of study
Medicine, Computer Science
- Identifiers
- External record
- Source metadata
Semantic Scholar, PubMed
CITATION MAP
EXTRACTION MAP
CLAIMS
CONCEPTS
- article significance
The importance or impact score assigned to a scientific article within the proposed review framework.
Aliases: significance of articles
- distributed reviews
A review arrangement in which multiple reviewers each assess only part of an article rather than the whole manuscript.
Aliases: distributed reviewing
- methods and materials access
Access for reviewers to the experimental methods and supporting materials needed to evaluate a study thoroughly.
Aliases: methods and materials
- open interactive review system
A proposed review framework that supports ongoing, interactive evaluation and updating of reviews after publication.
Aliases: open review system
- open-source software management
The collaborative software-development model used here as a source of design ideas for reviewing and contribution tracking.
Aliases: OSS management
- peer-review process
The conventional journal review workflow used to evaluate manuscripts before publication.
Aliases: peer review
- post-publication review
Review activity that continues after an article has been published.
Aliases: post publication review
- reviewer acknowledgment
Explicit recognition or identification of reviewers for their contributions to manuscript evaluation.
Aliases: reviewer identification
- reviewer reputation
The standing of a reviewer in the scientific community based on acknowledged reviewing contributions.
Aliases: reputation of reviewers
- review quality quantification
A scheme for assigning a measurable score or metric to the quality of a review.
Aliases: quantifying review quality
- social networking technologies
Online technologies for connecting people and tracking interactions, used here as a source of design ideas for review interaction and reputation.
Aliases: social networks
REFERENCES
Showing 1-12 of 12 references · Page 1 of 1
CITED BY
Showing 1-39 of 39 citing papers · Page 1 of 1