In a new article published in Plos Biology, Alexandre Scanff and his colleagues examine publication bias in biomedical journals.
The authors use the Percentage of Papers by the Most Prolific author (PPMP) alongside the Gini index (a measure of inequality in the distribution of authorship) to identify possible editorial bias. While the current research has found that most journals have distributed publications across a large number of authors, 5% of journals had a PMPP of 10.6% or more. A random sample from the most offending journals revealed that in 61% of cases, the most prolific authors were members of the editorial board. Scanff and his coauthors explain:
“In this comprehensive survey of 5,468 biomedical journals, we describe several features of editor-author relationships among which were the following: (i) an article output was sometimes dominated by the prolific contribution of one author or a group of authors; (ii) with time lags to publication that were in some instances shorter for these prolific authors (when this information was available), and (iii) more than half of those prolific authors were typically members of the journal’s editorial board.”
Bias in the publication of academic journals is fairly common. One way that journals bias their publication is by favoring articles that report statistically significant results in clinical trials. In other words, articles finding that a specific intervention (like a medication, for example) works well are more likely to be published than research showing that medication does not work. This practice biases the literature by making treatments seem more efficacious than they are.
Other research has found that psychiatric research into schizophrenia and bipolar disorder treatments commonly overestimate treatment effects and expose patients to ineffective treatment. Authors have also pointed to problems in psychology research, noting strong evidence of publication bias that “threatens the credibility of the entire field of psychology.”
Research has found bias in the publication of trials for antipsychotics that likely overestimates their usefulness. Similarly, authors have found that in research around antidepressants, negative results are commonly eliminated from the literature or made difficult to find. Antidepressants have also been used in the treatment of autism with similarly biased literature. When corrected for publication bias, there appears to be no benefit of using antidepressants for autism.
The effectiveness of psychotherapy is likely similarly overestimated in the academic literature due to publication bias. The problem of bias in academic publications is so significant that some authors have said medical journals “have devolved into information laundering operations for the pharmaceutical industry.”
The current research begins by examining what they call “self-promotion journals.” These are journals, like New Microbes and New Infections (NMNI), in which a large percentage of the papers published are authored (or coauthored) by editors of the journal. In the case of NMNI, 32% of the 728 articles published as of June 2020 were coauthored by Didier Raoult. The editor in chief, as well as six other editors of NMNI, report directly to Raoult. For Scanff and his colleagues, these publishing practices are problematic as the relationships between authors and editors tend to promote low-quality research.
In addition to examining the PPMP as in the above example, the current research also uses the Gini index to measure the contributions of groups of authors rather than a single author. While the PPMP is a valuable measure for small and medium-size journals, the authors argue that the Gini index, a measure of the degree of unequal distribution of authorship, is more useful to examine publication bias in large journals. With large journals publishing vast numbers of papers, it becomes impossible for a single author to publish more than 10.6% (the cutoff in the current work). Therefore the Gini index may be a better indicator of this kind of bias.
The authors examined 5,468 biomedical journals published between 2015 and 2019. The PPMP median was 2.88%, meaning that, on average, the most prolific authors in each journal authored about 3% of the publications. The journals with the most extensive publication bias in terms of PPMP, the 95th percentile, have nearly 11% of their articles written by their most prolific author.
In terms of the Gini index, which ranges from 0 to 1, with smaller values indicating a more equal distribution of authorship, the median was .183. The journals with the largest likelihood of publication bias in terms of the Gini index, the 95th percentile, had a Gini score of .355. The PPMP and Gini indices show a correlational coefficient of .35, indicating a moderate positive correlation.
These findings were consistent across different fields, and date ranges and when considering all published articles in a journal or only the research articles.
The authors identify three features common to the 95th percentile scoring journals: authorship is sometimes dominated by an author or group of authors, this author or group sometimes enjoys shorter publications lags (indicating a possible lack of serious peer review), and more than half of the prolific authors were members of the journal’s editorial board. These most offending journals also engaged in questionable self citing practices, artificially increasing their journal impact score.
The authors note that not all prolific authors are involved in the questionable publication practices they are describing. Some legitimate reasons an author may be prolific within a journal include. Some authors are highly productive; some are disproportionately represented due to playing a pivotal role in one aspect of the research (like statistical analysis, for example). They also note that shorter publication lags are likely in some cases due to the ease with which journals can find reviewers willing to work on eminent authors’ research. While not all prolific authors are necessarily involved in questionable self-promoting practices, the authors sum up their paper as follows:
“Our results underscore possible problematic relationships between authors who sit on editorial boards and decision-making editors … Although our findings are based solely on a subsample of journals, they provide crucial evidence that editorial decisions were not only unusually, but also selectively, fast for the favored subset of prolific authors.”
Scanff, A., Naudet, F., Cristea, I. A., Moher, D., Bishop, D. V. M., & Locher, C. (2021). A survey of biomedical journals to detect editorial bias and nepotistic behavior. PLOS Biology, 19(11), e3001133. https://doi.org/10.1371/journal.pbio.3001133 (Link)