advocate could insist on publication; a note could be included with the published paper so that
reviewers are, in a sense, willing to stake their reputations on the paper.3 Through this note, the
readers would receive information about the nature of the acceptance. All referees could be given
the opportunity to write peer commentary on the paper. This procedure would greatly increase
the likelihood that important papers would be published. The increased effort given to reviewing
might also improve quality control.
Conclusions. Controversial empirical papers are expected to receive harsh treatment in
peer review, but our survey indicates that such works occasionally get published, sometimes
without much peer agreement. More can be done to encourage publication, however. We suggest
ways to accomplish this, in particular, the use of an advocacy procedure that explicitly
recognizes the need to promote this type of research.
References
Armstrong, J. S. (1982a), “Barriers to scientific contributions: The author’s formula,” Behavioral
and Brain Sciences, 5, 197-199.
Armstrong, J. S. (1982b), “Is review by peers as fair as it appears?” Interfaces, 12, 62-74.
Armstrong, J. S. (1982c), “Research on scientific journals: Implications for editors and authors,”
Journal of Forecasting, 1, 83-104.
Fisher, A. (1989), “Seeing atoms,” Popular Science, 102-07.
Fiske, D. W. and L. Fogg (1990), “But the reviewers are making different criticisms of my
paper! Diversity and uniqueness in reviewer comments,” American Psychologist, 45,
591-598.
Horrobin, D. F. (1990), “The philosophical basis of peer review and the suppression of
innovation,” Journal of the American Medical Association, 263, 1438-1441.
Mahoney, M. J. (1977), “Publication prejudices: An experimental study of confirmatory bias in
the peer review system,” Cognitive Therapy Research hh, 1, 161- 175.
McNutt, R. A., A. T. Evans, R. H. Fletcher, and S. W. Fletcher (1990), “The effects of blinding
on the quality of peer review,” Journal of the American Medical Association, 263, 137-
176.
3 McNutt et al. (1990) found no differences in the quality of reviews based on whether or not
they were signed by the reviewer. Also, those who signed the reviews were more likely to
recommend acceptance.