JAMA Psychiatry Retracts Antidepressant Study

Once an appropriate statistical method was used, the study findings were “no longer valid,” according to the editors of JAMA and JAMA Psychiatry.

Peter Simons
12
4017

Editor’s Note: This article has been updated to include the statement from the authors of the retracted article.

A study from February purporting to find differences in response rates to different antidepressants—presented as evidence for potentially “personalized” depression treatment—has been retracted. The original piece suggested that some subgroups might exist for whom antidepressants were more effective than placebo. However, according to the Editor of JAMA Psychiatry, Dost Öngür, and the Editor in Chief of JAMA, Howard Bauchner, “the original findings are no longer valid.”

The two Editors collaboratively wrote a retraction notice for the piece:

“In the Original Investigation, ‘Individual Differences in Response to Antidepressants: A Meta-analysis of Placebo-Controlled Randomized Clinical Trials,’ published on February 19, 2020, and in the June 2020 issue of JAMA Psychiatry, an incorrect analysis (coefficient of variation ratios) was conducted. When a proper analysis is used (random-slope mixed-effects model), the original findings are no longer valid. Thus, the article has been retracted.”

The notice makes it clear that the original authors used an inappropriate statistical technique to arrive at their positive results. When “a proper analysis is used,” their results are “no longer valid.”

In fact, two previous papers had used the correct analysis, and both found, similarly, null results, which were written about here on Mad in America.

The Editors were alerted to the error after researchers wrote a letter documenting the misapplication of the statistical methods. Researchers on that letter included Michael P. Hengartner, Martin Plöderl, and Constantin and Alexander Volkmann. Their letter is available here.

According to that letter, the issue is that the original researchers did not correct for the fact that different depression scales have different means (average scores) and standard deviation (variance of scores). This bias in the data thus created the appearance of more variability.

The authors of the retracted article wrote a statement, posted on Retraction Watch on June 19, 2020, about the retraction. They clarified that “two different groups sent letters to the Editor pointing out potential flaws.” After reviewing their analysis, the authors reported to the Editor that their findings were incorrect; the Editor then determined that the study would be retracted.

Benoit Mulsant, the corresponding author for the retracted study, stated on Retraction Watch that “Honest errors happen in science; it is to the benefit of the scientific community that these errors are identified and corrected.”

It is unclear how the use of an “incorrect” statistical analysis was passed by the peer reviewers of the original article, who are meant to ensure the correctness of methods used by study authors.

The original study, led by Marta Maslej, also included prolific antidepressant researchers Andrea Cipriani and Toshiaki Furukawa as authors. In their paper, they describe their statistical analysis as “the use of validated methods.”

Maslej and the other authors noted that “previous efforts to identify factors associated with response to specific ADs have generally been unsuccessful.” Hence, it was clear that their results contradicted previous findings and should have been subjected to scrutiny.

In an accompanying editorial that appeared online in February alongside the piece by Maslej et al., Gerard Sanacora praised the incorrect finding as one of the first to make the dream of personalized treatment for depression an attainable reality, writing that the study “provides empirical evidence to back the long-held belief that differences exist between individuals diagnosed with MDD that may allow for more personalized pharmacological treatment approaches.”

He wrote that “it is nice to see actual evidence presented in this format. Hopefully, it will help inspire the continued search for meaningful genotypes, endophenotypes, and clinical subtypes that can one day lead to a more rational and personalized approach to the treatment of MDD.” Currently, this editorial remains available without amendment or indication that the study on which it is based has now been retracted.

The retraction serves to undercut such hopes for improving the effectiveness of antidepressants through personalization. It may also serve as a warning for how the zeal for new treatments can diminish the quality of scientific evidence if scrutiny is not maintained.

 

****

Öngür, D., & Bauchner, H. (2020). Notice of retraction: Maslej et al. Individual differences in response to antidepressants: A meta-analysis of placebo-controlled randomized clinical trials. JAMA Psychiatry, 2020;77(6):607-617. JAMA Psychiatry. Published online June 10, 2020. DOI:10.1001/jamapsychiatry.2020.2026 (Link)

12 COMMENTS

  1. Peter thank you.
    Perhaps they need to retract more than the studies. A retraction of a study makes them look honest to some people.
    I had to smile at “personalized” “treatments”. I know they must too, And personalized really means creating a pill for each person. And what tests will be done to make the “treatments” “personalized”.
    I guess the same questions? Or will the “examinations” be personalized?
    Speaking of personalized treatment, why is it they can’t help people with sleeping problems? Why indeed? If they “know” so much about the brain, it seems they could create something better than a pill that causes bad effects. It seems the “sleep center” would be figured out by now.

  2. I love this – when we use proper mathematical analysis, the findings are “no longer valid.” Meaning they were NEVER VALID in the first place! Isn’t the real conclusion, “The findings were fraudulent, as the researchers used inappropriate methods to draw the conclusion they wanted to find?” Why not call a spade a spade?

    • @Steve McCrea:

      “I love this – when we use proper mathematical analysis, the findings are “no longer valid.” Meaning they were NEVER VALID in the first place! Isn’t the real conclusion, “The findings were fraudulent, as the researchers used inappropriate methods to draw the conclusion they wanted to find?” Why not call a spade a spade?”

      Robert Whittaker used the term “cherry picking” in “Psychiatry Under the Influence.” And it WAS, but in a larger framework, it was FRAUD. Softening the blow for those of ill-intent…well, they are not going to notice or care about his careful language. And the general public needs to hear it shouted from the rooftops: THIS IS FRAUD.

      If you can get someone to take the time to read the book at all!

      I once thought of this dystopia (for writers & publishers) that would also be a UTOPIA, sort of (for lovers of books).

      In a post-publication world, people would be paid to *read* books (NOT TO WRITE REVIEWS, JUST READ THEM!) They would get paid per book & have to take comprehension tests. But they would just get tiny amounts to read them & the writers would not get any $ at all & the publishers just a little bit more.

      This would be to get the publishers’ works in the collective unconscious so they could just cover cost for printing or online marketing & distribution…working the algorithms in cyberspace, etc.

      It would be *sort of* a utopia for the book readers, ‘cept it would be more like War Craft or gaming where the players have to play 20 hours a day just to make enough $ to survive, keep the lights on & the fridge, sort of, stocked..

      .I have even heard of people taking constipating drugs (indie film underground factoid here) so they do not have to take the time to #2!!!!!!!!!!!!!

      I hate to say this aloud, cause every time I come up with a new dystopia, IT HAPPENS!! It’s Old Skool MK Ultra, actually…

      But you could do a serious comedy skit about the taking of constipating drugs to get through the critiques of Big Pharma & the Guilds……….

      Hardy har har har………..I am not a fan of scatological humor in general,

      but I will make an exception for Big Pharm & Psychiatry!

      Ah, I started the next comic riff, INDEED!

LEAVE A REPLY