While publication bias has been known to overestimate the efficacy of antidepressant treatments, a new study suggests that research on the use of psychotherapy in depression suffers from a similar bias.
“Our findings indicate that psychological treatment is efficacious and specific, but, as is the case for antidepressants, less than the published literature conveys,” the researchers, led by Ellen Driessen, wrote.
In an effort to account for previously unpublished results, the study, which appears in the journal PLOS One, examined all of the grants funded by the National Institutes of Health (NIH) on psychotherapy for depression between 1972 and 2008.
The researchers identified studies that were funded by the NIH but never published, they then requested the unpublished data from the original investigators. Out of 55 grants from the NIH that met this criterion, only 42 (or 76.4%) had published their results. The investigators who led 11 of the remaining 13 studies turned over their unpublished data for the meta-analysis.
When the researchers combined the data from the 42 published studies with the 11 unpublished studies, the effect size for psychotherapy in depression was reduced by 25%. However, the psychological treatments remained significantly more efficacious than the control conditions even after the unpublished data was added.
The largest difference in effect size between published and unpublished trials appeared in studies where therapy was being compared to a pill placebo. Adding in unpublished study data comparing therapy to a pill placebo caused a 45% drop in effect size.
Interestingly, the unpublished data did not change the comparison of therapy to antidepressants. Even with the new data, no significant difference appeared between the two, though antidepressants combined with therapy had a larger effect than antidepressants alone.
When asked why their studies had remained unpublished, the investigators explained that “they did not think the findings were interesting enough to warrant publication, that they got distracted by other obligations or that they had practical problems.”
“The efficacy of psychological interventions for depression has been overestimated in the published literature, just as it has been for pharmacotherapy,” Driessen and her colleagues conclude.
“Clinicians, guidelines developers, and decision makers should be aware that the published literature overestimates the effects of the predominant treatments for depression.”
*
For More Coverage of this Study:
The New York Times: “Effectiveness of Talk Therapy Is Overstated, a Study Says”
Medical Daily: “Psychotherapy Does Work For Depression Patients, But Its Effectiveness May Be Overstated In Studies”
NPR: “Studies May Overstate The Benefits of Talk Therapy For Depression”
*
Driessen E, Hollon SD, Bockting CLH, Cuijpers P, Turner EH (2015) Does Publication Bias Inflate the Apparent Efficacy of Psychological Treatment for Major Depressive Disorder? A Systematic Review and Meta-Analysis of US National Institutes of Health-Funded Trials. PLoS ONE 10(9): e0137864. doi:10.1371/journal.pone.0137864 (Full Text)
