Peer Review Unfairly Favors Famous Names, Study Finds

Submitting articles for peer review with a Nobel Prize laureate's name attached increases acceptance rates.

1
1224

Researchers in Austria found that when a scientific paper submitted for peer review had the name of a Nobel Prize laureate attached, reviewers were at least 20% more likely to recommend acceptance compared to when the name was not attached.

The authors conducted an experiment asking reviewers to assess a paper written by laureate V. L. Smith—who won the Nobel Memorial Prize in Economic Sciences in 2002—and a relatively unknown colleague. In total, they received 534 reviews.

The study’s authors varied whether Smith’s name was mentioned, whether the relatively unknown colleague was mentioned, or whether the paper was given anonymously. The results suggest significant publication bias toward prestigious researchers such as Smith.

“Peer review has been the key method for research validation since the first scientific journals appeared some 300 years ago. For researchers—and especially for young scientists, who must excel for scientific advancement—it is crucial that this process be fair and impartial. Merton, however, argued that ‘eminent scientists get disproportionately great credit for their contribution to science while relatively unknown scientists tend to get disproportionately little credit for comparable contributions.’
Alluding to the Gospel according to Matthew 25:29, Merton termed this pattern of the misallocation of credit for scientific contributions ‘the Matthew effect in science,’ while others called it ‘status bias,’” write a team of authors led by Jürgen Huber, Head of the Department of Banking and Finance at Innsbruck University in Austria.

tired professor walking down hallwayIn order to test the possible effect of “status bias” on acceptance of research publications, authors of the current study invited a number of professionals in the field to review an unpublished paper written by the previously mentioned Nobel Prize laureate V.L. Smith (who has over 50,000 Google Scholar citations) along with a relatively unknown colleague (with 42 Google Scholar citations). This paper was first submitted to the Journal of Behavioral and Experimental Finance, after which more than 3,300 potential reviewers were invited to provide peer review.

Acceptance rates varied between the three conditions: low-prominence author mentioned, no author mentioned, and the high-performance author mentioned.

Respectively, 28.52% of potential reviewers accepted the invitation for the low-prominence author, 30.74% for the anonymous author, and 38.54% for the high-performance author. This already shows a bias toward being willing to peer review work from those of higher social status.

Furthermore, the authors hypothesized that when the high-prominence name was attached, there would be a significant increase in acceptance of the article compared to when it was anonymized. Likewise, they hypothesized that the inclusion of the low-prominence author’s name would also decrease acceptance compared to the anonymized version.

As the authors of the current study noted, the most important findings were related to the choices given to those invited for peer review: reject, major revision, minor revision, or accept. These options are standard across multiple scientific disciplines and journals requiring peer review.

Reporting their statistically significant findings, they stated:

“While 65.4% of the reviewers recommended ‘reject’ when shown the less prominent author, this number was 48.2% in the anonymized version of the manuscript and 22.6% when the prominent author was shown.”

Or, framed in the positive:

“Aggregating the two most positive categories, ‘accept’ and ‘minor revision,’ of the recommendation spectrum to a single category showed that only 9.9% of the reviewers recommended a minor revision or an outright accept when shown the less prominent author, while 23.6% gave one of these recommendations in the anonymized version, and 58.8% did so when shown the prominent author.”

The authors also included six standardized questions which are common to the peer review process for journals published by Elsevier:

  • Was the subject worthy of investigation
  • Was the information new
  • Were the conclusions supported by the data
  • Was the manuscript suitable for the particular journal
  • Was the organization of the manuscript appropriate
  • Were the tables and figures appropriate

A Likert scale from 1-5 was employed here, one being “strongly disagree” and five being “strongly agree.”

As with previous findings, the authors found an “upward shift” in the assessment of the paper’s quality with these criteria in mind when the high-prominence author’s name was attached, compared to both the anonymized version and the version with the low-prominence author.

The authors noted some limitations to the study. One, when consent for peer review was attained, the authors made it clear as part of their ethical standards that this was not just an ordinary peer review but was also part of a study on the peer review process itself. This may have had unintended effects.

A second limitation was reported in the context of potential racism. The low-prominence author’s last name was Inoua, and the authors state that he has dark skin, while Smith (a more conventionally Caucasian name) has white skin. The authors clearly state that:

“Skin tone (i.e., colorism) has long been identified as among the most salient markers for discrimination” and cite several studies to support this finding.

Thirdly, the authors are transparent that this paper, and the journal it was submitted to, may not represent the entirety of all scientific disciplines. For example, it is possible that the findings would have been different in a discipline other than business/economics.

While stating that they are “confident that similar effects likely are also present in other fields,” the authors recommend performing replication studies in other scientific disciplines to test these hypotheses further.

Previous research into this area has found similar findings: for example, that the reputation of scholars has a biasing effect on the peer review process even when the reviewers themselves do not have access to the names of authors (as in double-anonymous peer review) because these decisions ultimately rest with journal editors, who can also be prone to bias. Likewise, journal editors who have worked with authors previously are more likely to favorably assess and accept their work in the future.

If these issues are not resolved through some alteration of the peer review process, then it is likely, as the authors suggest, that academics from marginalized groups, as well as “lower-prominence academics” in general, will remain at a significant disadvantage in terms of publication acceptance.

Concluding, the authors do make some suggestions for alternative peer review mechanisms:

“…a fully transparent review process, possibly even allowing open discussions between authors, reviewers, and editors. Another option are forms of ‘structured peer review’ that prompt reviewers to help improve a manuscript but do not ask for an accept/revise/reject recommendation. Exploring the impact of such innovative review processes is a promising avenue for future research.”

****

Huber J., Inoua S., Kerschbamer R., König-Kersting C., Palan S., & Smith V. L. (2022). Nobel and novice: author prominence affects peer review. Proceedings of the National Academy of Sciences. https://doi.org/10.1073/pnas.2205779119 (Link)

1 COMMENT

  1. No kidding. I think that the same goes when a famous person says they like a certain drug, or make out it’s part of their “ambiance” or what makes them so wonderful etc…..

    You could probably do the same study, and have different people recommend certain drugs (regardless of their efficacy) compared to just a “nobody,” and see the same difference, or how when a cheaply made commodity from a sweat shop can sell for next to nothing at any of the stores with dollar attached (family dollar, dollar tree, is there a dollar bazaar store?), but when Kim Kardashian or Suzanne Summers has their name on it then how much more can it be sold for because of the added celebrity varnish-image-aura?

LEAVE A REPLY