Positive Antidepressant Study “Misleading” and “Erroneous”

An analysis of last year’s positive finding in The Lancet about antidepressant efficacy shows errors, obfuscations, and misrepresentations.

Peter Simons
3
1940

New research by the Nordic Cochrane Center has challenged last year’s positive study about antidepressant efficacy. The authors write:

“The evidence does not support definitive conclusions regarding the benefits of antidepressants for depression in adults. It is unclear whether antidepressants are more efficacious than placebo.”

Photo Credit: Flickr

Many analyses of the research on antidepressant drugs have found that they are no better than placebo for the treatment of depression. These results have been especially reliable for mild to moderate depression, and especially when unpublished trials of the drug are included. At best, the drugs show an incremental improvement over placebo, which is not clinically detectable,

However, a study in The Lancet last year, led by Andrea Cipriani and conducted by authors with strong ties to numerous pharmaceutical companies, suggested that previous research was mistaken and that the drugs were effective. Nonetheless, researchers have suggested that the Cipriani study was flawed in multiple ways.

Now, a new study using the rigorous procedures of the Cochrane Collaboration has challenged that controversial positive conclusion put forth by Cipriani and his colleagues.

The research was conducted by three members of the Nordic Cochrane Center in Denmark, led by Dr. Klaus Munkholm. It was published open-access in the journal of the British Medical Association, BMJ Open.

Munkholm and the other researchers found several errors and misrepresented data in the Cipriani analysis. They write that “Several methodological limitations in the evidence base of antidepressants were either unrecognized or underestimated in the systematic review.”

For instance, certain types of antidepressant research were far more likely to find a positive result:

  1. Research using the controversial “placebo-run in” design. This type of study eliminates people who respond to the placebo effect–which removes it as a fair comparison. The study essentially compares people on the drug to a smaller group of people who didn’t respond to the placebo effect.
  2. Published versus unpublished clinical trials. Antidepressants were far more likely to have failed in the unpublished trials, and even when they succeeded, they were likely to have a much smaller effect size.
  3. Studies with a high risk of bias. Studies that were graded as having unclear methods, risk of unblinding, and other limitations were more likely to find a positive antidepressant effect.

Cipriani and his colleagues claimed to have followed the Cochrane handbook’s guidance on how to report the risk of bias in clinical trials. However, the researchers from the Nordic Cochrane Center found that Cipriani had misrepresented this data in several ways.

Munkholm and the other authors write, “The outcome data reported by Cipriani et al. differed from the clinical study reports in 12 (63%) of 19 trials.”

In some situations, the Nordic Cochrane authors could not determine whether Cipriani and colleagues had followed the Cochrane rules because Cipriani and his co-authors failed to describe their methods adequately. In other situations, however, Cipriani and co-authors defied those rules:

Cipriani and colleagues didn’t analyze study drop-outs, even though that is one of the regulations in the Cochrane handbook. Researchers must analyze why participants drop out in the placebo versus the active antidepressant groups. If it is due to adverse events, suicidal ideation, or ineffectiveness of the drug, that changes the risk/benefit ratio. But according to Munkholm and co-authors:

“The authors [Cipriani et al.] did not consider the reasons for dropout, although this is also recommended by the Cochrane Handbook.”

Additionally, they write that “Cipriani et al. did not use the standard Cochrane categorization of low, unclear or high risk of bias due to a lack of blinding […] While this implied the presence of a blinding issue, their categorization did not affect the overall risk of bias assessment, and it seemed that the stated-not tested domains were counted as ‘low risk of bias.’”

This means that Cipriani and colleagues:

  1. Didn’t use the Cochrane categorization, despite claiming to in their methods.
  2. Didn’t count the “not-tested” or “unclear” domains as being at risk of bias, even though unclear methods are a way of biasing results.

Further, many of the studies were at risk of bias due to selective outcome reporting. The researchers chose only to report favorable outcomes, despite the presence of primary findings that showed no effect of the antidepressant. This is one of the most egregious ways to bias research studies, and again, Cipriani and his co-authors failed to correctly follow the Cochrane Handbook in their treatment of the issue:

“A trial was only rated at high risk of bias in case both outcomes were missing. This is not in accordance with the Cochrane Handbook.”

Even discounting the bias issue, the evidence was less strong than the Cipriani analysis suggested. The average improvement over placebo (on the 52-point Hamilton Depression Rating Scale) was less than two points–a clinically insignificant difference. At least three points, and perhaps as much as seven points, is the minimum clinically noticeable change on the Hamilton scale.

All of this evidence confirms previous studies. Although Cipriani and his co-authors obfuscated this data in their summary, writing that the benefits of antidepressants outweigh the risks, the evidence itself tells a different story. The raw data in the Cipriani analysis suggests that antidepressants do not outperform placebo by a significant degree.

According to Munkholm and the other researchers, “It seems misleading to rank the antidepressants when we have very low confidence in the evidence.”

“In the light of our findings, the [Cipriani et al.] review should not inform clinical practice. Second, our reanalysis has highlighted the need for a radical change in the way antidepressant trials are being conducted, reported, and interpreted.”

 

****

Munkholm, K., Paludan-Miller, A. S., & Boesen K. (2019). Considering the methodological limitations in the evidence base of antidepressants for depression: A reanalysis of a network meta-analysis. BMJ Open, 9(e024886). http://dx.doi.org/10.1136/bmjopen-2018-024886 (Link)

Previous articleAndrew Rich: “I Didn’t Know Stuff Like This Existed”
Next articleIs Medicine Overrated?
Peter Simons
MIA Research News Team: Peter Simons comes from a background in the humanities where he studied English, philosophy, and art. Now working on his PhD in Counseling Psychology, his recent research has focused on conflicts of interest in the psychopharmaceutical research literature, the use of antipsychotic medications in the treatment of depression, and the general philosophical and sociopolitical implications of psychiatric taxonomy in diagnosis and treatment.

3 COMMENTS

  1. You can name cheap chemical narcotics for the masses “antidepressants”, but the essence does not change.
    I’m not talking about antipsychotics, medications that prove the planet is overpopulated.
    In a state of psychosis, i had the idea that with the help of such tablets the government recycles chemical waste.

  2. One of the saddest aspects of this study to me is the involvement of Ioannidis and Turner. They signed their names on this study, though I do not know their actual level of involvement. The paper fails to discuss many of the problems with antidepressant trials that Ioannidis himself raised in past papers. And Ioannidis famously asked Cochrane if meta-analysis was doing nothing more than taking biased studies and laundering them through Cochrane’s name. He wondered if it was not doing more harm than good. And then John went and signed his name to a meta-analysis of the most biased set of studies in medicine, laundering them through his name. And the paper failed to discuss many of these biases at all.

    Ioannidis spoke to the media about the paper, and was quite vocal in his support of the result, so it seems his participation was more than token. I think this paper has done great damage to the reputation of Ioannidis and Turner, who are known for their commitment to good evidence, which they seem to have cast aside here.