A new study aimed to assess the efficacy of medication, therapy, and combined treatments. The researchers analyzed data from thousands of studies across a comprehensive range of psychiatric disorders. They found that both medication and therapy had limited efficacy in treating these problems.
The researchers write, “Across disorders and treatments, the majority of effect sizes for target symptoms were small.”
The only treatment with a large effect size was psychotherapy for OCD.
The study was published in World Psychiatry. The researchers included renowned Stanford statistician and methodologist John Ioannidis, who, in his influential paper “Why Most Published Research Findings Are False,” found that “for most study designs and settings, it is more likely for a research claim to be false than true.”
According to the researchers, biases in the psychiatric research literature could lead to an overestimation of treatment efficacy. For instance, a treatment that is just marginally better than a placebo might meet the criteria for statistical significance while failing to be clinically significant. Publication bias and other problems can inflate this problem further. Positive findings are much more likely to be published. When researchers find that a treatment is ineffective, their study may never see the light of day.
For that reason, the current study focused on effect size, which measures how effective a treatment is, rather than just signifying whether it is effective or not effective.
The researchers describe the data that was included in their study: “One hundred and two meta-analyses, encompassing 3,782 RCTs and 650,514 patients, were included, covering depressive disorders, anxiety disorders, post-traumatic stress disorder, obsessive-compulsive disorder, somatoform disorders, eating disorders, attention-deficit/hyperactivity disorder, substance use disorders, insomnia, schizophrenia spectrum disorders, and bipolar disorder.”
They focused on studies that compared medication or therapy with placebo or treatment as usual (TAU). They did not include studies that used a waiting list comparator since this can artificially inflate effect sizes. They also assessed the risk of bias in the included studies, which is often ignored in other meta-analyses.
Their analysis found that, on average, psychotherapy had an effect size of 0.34, and medication had an effect size of 0.36 when compared with placebo or TAU. They note that this is considered a small effect.
When comparing therapy with medication, the researchers found that their efficacy was about the same (no statistically significant difference). Combined treatment (therapy plus medication) had an effect size of 0.31 above monotherapy.
To put these effect sizes in context: The researchers suggest that for depression, an effect size of 0.88 is needed before the improvement would be clinically noticeable. For schizophrenia, an effect size of 0.73 is required.
There were some exceptions. Cognitive-behavioral therapy (CBT) for OCD had a large effect size, reaching 1.03. However, most patients were also taking medication, so this could be considered a combined treatment.
Medication alone for OCD had a medium effect size (0.56), as did CBT for PTSD (0.54) and therapy for borderline personality disorder (0.57), although this last relied on a single meta-analysis with a high risk of bias.
Another interesting exception was that CBT was far better than medication for PTSD (effect size 0.83, considered a large effect).
One compelling negative finding was that antipsychotic drugs were not very effective at treating schizophrenia. The researchers write, “For the acute treatment of schizophrenia with pharmacotherapy, differences in response rates in comparison with placebo were small (23% vs. 14%).”
In many cases, specific drugs were found to decrease specific aspects or symptoms but did not have much of an effect on the overall diagnosis. These findings were also often based on a few studies with a high risk of bias, so they may not be reliable.
The researchers also found that the included meta-analyses had a high risk of bias. Forty-eight percent of the meta-analyses failed to account for bias, and 47% failed to register their studies. This was true for both medication studies and therapy studies.
They also note that most of the included studies were short-term and that data on long-term efficacy is lacking.
In conclusion, they write,
“After more than half a century of research, thousands of RCTs, and millions of invested funds, the effect sizes of psychotherapies and pharmacotherapies for mental disorders are limited, suggesting a ceiling effect for treatment research as presently conducted. A paradigm shift in research seems to be required to achieve further progress.”
Leichsenring, F., Steinert, C., Rabung, S., & Ioannidis, J. P. A. (2022). The efficacy of psychotherapies and pharmacotherapies for mental disorders in adults: An umbrella review and meta-analytic evaluation of recent meta-analyses. World Psychiatry, 21, 133–145 (Link)