Peer review—evaluation by colleagues with similar knowledge and experience—is an essential component of research and the self-regulation of professions. The average person does not have the knowledge and experience needed to assess the quality and importance of research. Peers do. Therefore many important decisions about research depend on advice from peers, including:
The quality of the decisions made in each case depends heavily on the quality of peer review.
Peer review can make or break professional careers and directly influence public policy. The fate of entire research programs, health initiatives, or environmental and safety regulations can rest on peer assessment of proposed or completed research projects. For peer review to work, it must be:
Researchers who serve as peer reviewers should be mindful of the public as well as the professional consequences of their evaluations and exercise special care when making these evaluations.
The effort researchers put into peer review is for the most part not compensated. Researchers may receive reimbursement for travel and per diem when they attend special grant-review sessions and occasionally are paid a basic daily stipend, but this seldom covers the true cost of reviewing a manuscript or a stack of grant applications. As uncompensated effort, the time researchers devote to peer review can easily take second place to other obligations. Running a crucial experiment or submitting a grant application on time understandably is more important than reviewing someone else’s work.
However pressed you are for time, if you agree to do a review, you should find the time to meet your obligation in a timely manner. Research is competitive. Researchers are rewarded for discoveries. They should not lose their priority for a discovery due to the tardiness of a reviewer sending comments on a manuscript. Research is also useful. The announcement of discoveries that can benefit the public should not be delayed because someone who agreed to review a manuscript does not have the time to do the review.
Peers who are asked to make judgments about the quality of a proposed or completed project must do their best to determine whether the work they have been asked to review is internally consistent and conforms to the practices of their field of research. This certainly includes:
At the very least, peer reviewers should be expected to assess whether the manuscript or proposal under review makes sense and conforms to accepted practices, based on the information presented.
Research that conforms to accepted practices can still have problems. Research quality can be compromised by:
However, how much peer reviewers can or should do to detect these and other deceptive or sloppy practices remains subject to debate.
In addition to quality, peer reviewers are also asked to make judgments about the importance of proposed or published research. They are asked to answer questions such as:
Along with quality, judgments about importance essentially determine which research is funded or published and which researchers are hired and relied upon for advice.
Peer reviewers do not always make judgments about importance with an open mind. Studies have shown that they can be swayed by:
One way to lessen the impact of bias is to write transparent reviews. By “transparent” is meant laying out clearly for anyone reading the review how it was prepared, the literature that was used, and the reviewer’s own possible biases. If reviewers fully and carefully explain how their judgments about importance were made, others can assess whether they want to accept those judgments.
Peer reviewers have an obligation to preserve confidentiality during the review process if they have been asked to do so. While this obligation might seem obvious, it can be compromised in some seemingly harmless and other more harmful ways. For example, although researchers sometimes do, it is not acceptable to do any of the following without getting permission:
There may be times when some added advice during a review may be helpful, but reviewers should not seek this advice without getting permission. It may also be tempting to use information in a grant application or manuscript to speed up your own research, but until it has been made public, confidential information is not available for use, even to reviewers. If you are not comfortable protecting confidential information, then do not agree to be a peer reviewer.
National Institutes of Health. NIH Guide – Objectivity in Research, Bethesda, MD: NIH, 1995. (Link )
University of Michigan Medical School. Guidelines for the Responsible Conduct of Research: Right and Responsibilities of Peer Review, Ann Arbor, MI: UM, 1999.
International Committee of Medical Journal Editors. Uniform Requirements for Manuscripts Submitted to Biomedical Journals, 2001. (Link )
International Congress on Peer Review and Biomedical Publication. Home Page, 2003. (Link )
Office of Extramural Research. National Institutes of Health. OER: Peer Review Policy and Issues, 2003.(Link )
The Graduate School wishes to thank the Office of Research Integrity for the above materials found in the textbook, ORI Introduction to the Responsible Conduct of Research, Nicholas H. Steneck, PhD; Illustrations by David Zinn
HTML Version, September 2006, updated from Revised Printed Edition, June 2004