Editor’s Note: This article has been updated to include the statement from the authors of the retracted article.
A study from February purporting to find differences in response rates to different antidepressants—presented as evidence for potentially “personalized” depression treatment—has been retracted. The original piece suggested that some subgroups might exist for whom antidepressants were more effective than placebo. However, according to the Editor of JAMA Psychiatry, Dost Öngür, and the Editor in Chief of JAMA, Howard Bauchner, “the original findings are no longer valid.”
The two Editors collaboratively wrote a retraction notice for the piece:
“In the Original Investigation, ‘Individual Differences in Response to Antidepressants: A Meta-analysis of Placebo-Controlled Randomized Clinical Trials,’ published on February 19, 2020, and in the June 2020 issue of JAMA Psychiatry, an incorrect analysis (coefficient of variation ratios) was conducted. When a proper analysis is used (random-slope mixed-effects model), the original findings are no longer valid. Thus, the article has been retracted.”
The notice makes it clear that the original authors used an inappropriate statistical technique to arrive at their positive results. When “a proper analysis is used,” their results are “no longer valid.”
In fact, two previous papers had used the correct analysis, and both found, similarly, null results, which were written about here on Mad in America.
The Editors were alerted to the error after researchers wrote a letter documenting the misapplication of the statistical methods. Researchers on that letter included Michael P. Hengartner, Martin Plöderl, and Constantin and Alexander Volkmann. Their letter is available here.
According to that letter, the issue is that the original researchers did not correct for the fact that different depression scales have different means (average scores) and standard deviation (variance of scores). This bias in the data thus created the appearance of more variability.
The authors of the retracted article wrote a statement, posted on Retraction Watch on June 19, 2020, about the retraction. They clarified that “two different groups sent letters to the Editor pointing out potential flaws.” After reviewing their analysis, the authors reported to the Editor that their findings were incorrect; the Editor then determined that the study would be retracted.
Benoit Mulsant, the corresponding author for the retracted study, stated on Retraction Watch that “Honest errors happen in science; it is to the benefit of the scientific community that these errors are identified and corrected.”
It is unclear how the use of an “incorrect” statistical analysis was passed by the peer reviewers of the original article, who are meant to ensure the correctness of methods used by study authors.
The original study, led by Marta Maslej, also included prolific antidepressant researchers Andrea Cipriani and Toshiaki Furukawa as authors. In their paper, they describe their statistical analysis as “the use of validated methods.”
Maslej and the other authors noted that “previous efforts to identify factors associated with response to specific ADs have generally been unsuccessful.” Hence, it was clear that their results contradicted previous findings and should have been subjected to scrutiny.
In an accompanying editorial that appeared online in February alongside the piece by Maslej et al., Gerard Sanacora praised the incorrect finding as one of the first to make the dream of personalized treatment for depression an attainable reality, writing that the study “provides empirical evidence to back the long-held belief that differences exist between individuals diagnosed with MDD that may allow for more personalized pharmacological treatment approaches.”
He wrote that “it is nice to see actual evidence presented in this format. Hopefully, it will help inspire the continued search for meaningful genotypes, endophenotypes, and clinical subtypes that can one day lead to a more rational and personalized approach to the treatment of MDD.” Currently, this editorial remains available without amendment or indication that the study on which it is based has now been retracted.
The retraction serves to undercut such hopes for improving the effectiveness of antidepressants through personalization. It may also serve as a warning for how the zeal for new treatments can diminish the quality of scientific evidence if scrutiny is not maintained.
****
Öngür, D., & Bauchner, H. (2020). Notice of retraction: Maslej et al. Individual differences in response to antidepressants: A meta-analysis of placebo-controlled randomized clinical trials. JAMA Psychiatry, 2020;77(6):607-617. JAMA Psychiatry. Published online June 10, 2020. DOI:10.1001/jamapsychiatry.2020.2026 (Link)
