The Impact of Regression to the Mean in Psychiatric Drug Studies

A new article by Michael P. Hengartner explores the possibility that the statistical phenomenon of regression to the mean may be responsible for the dramatic effects of placebo in psychiatric drug studies. Hengartner is a clinical psychologist at the Zurich University of Applied Sciences, and the paper was published online ahead of print in BMJ Evidence-Based Medicine.

Regression to the mean is a statistical term. Under this theory, scores that are outliers (at the extremes) are likely to move toward a more average value spontaneously. An example illustrates this phenomenon.

People with moderate depression, experiencing a variety of environmental stressors, are likely to improve slightly when a stressor is removed, or when their life gets better. This slight improvement may even be enough to move them toward not meeting criteria for depression. People with severe depression, though, are likely experiencing intense environmental stressors. If one of these problems changes for the better, it can make for a dramatic shift in someone’s life.

Thus, the people on the extremes of these experiences are statistically more likely to undergo a more significant shift toward the average experience. These changes happen spontaneously, through the shifting of all of the complexities of a person’s life.

The usual metaphor for regression to the mean is that of a baseball player. After a great season (let’s say they hit a lot of home runs), fans expect that player to do just as well in the next season. But, often, that doesn’t happen. In sports, this is called the sophomore slump. This occurs because the player was not that much better than the other professional players—they just got lucky that season. Chances are, across a whole team, one player would get a bit more fortunate than the rest. But luck doesn’t hold forever. The next season, that player is back to average, and a different player gets the lucky hits.

Most psychiatric drug effectiveness studies enroll people with pretty severe symptoms—meaning that their entire participant pool may consist of these outliers.

Hengartner provides the real example of a clinical trial of repetitive transcranial magnetic stimulation (rTMS) for veterans with depression. The participants selected for this trial had undergone several trials of medication and did not improve, so they were considered “treatment resistant.” Many of the veterans also had diagnoses of PTSD, substance abuse, and were deemed to be suicidal.

Since these veterans had already experienced several trials of medication without improvement, they had already experienced the potential for the placebo effect (receiving treatment intended to remedy their symptoms)—and it had not worked for any of them. Thus, it could be expected that the placebo response rate in this study would be very low, close to zero.

Instead, after less than two weeks, an astonishing 37% remitted after fake rTMS treatment. A slightly higher 41% remitted after the real treatment (note that this difference was not statistically significant, meaning that the actual treatment was no better than placebo). That is, 37% had remission of symptoms, not just improvement, meaning they improved so much they no longer had the diagnosed disorder.

Now, this is an incredibly high number. According to this study, 37% of people could be cured of depression, PTSD, suicidality, and substance abuse, after having tried drugs with no improvement, all by using a fake treatment, in less than two weeks. The researchers of that study, according to Hengartner, pin that improvement on the placebo effect—attention from medical staff and the belief that this treatment would work.

But they’d already tried two treatments that were supposed to work, without experiencing that improvement. Why would they believe this experimental, controversial procedure would work better than the established treatments?

Hengartner provides an alternative possibility. Both the placebo effect and the actual rTMS effect were actually due to regression to the mean. The participants selected for this study were undoubtedly scoring at the extreme ends of the measures used to test them, with severe, suicidal depression and PTSD symptoms, as well as substance abuse. So the statistical trend that researchers should have expected, based on years of research into regression to the mean, is that many of these participants could have gotten better spontaneously, without any treatment at all.

Hengartner provides evidence from other research to back up this claim. A study from 2013 found that, without treatment, 53% of people could expect to experience full remission from depression within a year. Even in the short-term (3 months or less), 23% of people had remission from depression without treatment.

Another analysis cited by Hengartner found that the number of people who experienced full remission from depression was the same whether they received placebo treatment or no treatment at all—a clear indicator of regression to the mean.

Finally, he compares these numbers to the STAR*D trial—the largest and most comprehensive real-world trial of antidepressant medications. In the short-term results from STAR*D, 28% of the participants experienced remission while taking an antidepressant drug.

This number—28% after taking a drug—may be statistically different from the 23% who experienced remission after no treatment, but it is unlikely to be clinically significant. And it should be noted that in the long-term, the STAR*D trial demonstrated the failure of the drugs—only about 3% of the participants that were still enrolled in the experiment had remission of depression at the one-year follow-up. Meanwhile, no treatment at all results in more and more people improving over time, up to 53% by one year.

Hengartner’s analysis suggests that regression to the mean is responsible for a large part of psychiatric drug effectiveness and that long-term data demonstrate better results from no treatment at all, throwing into question much of the research base of psychiatric interventions.

 

****

Hengartner, Michael P. (2019). Is there a genuine placebo effect in acute depression treatments? A reassessment of regression to the mean and spontaneous remission. BMJ Evidence-Based Medicine. Published Online First: 11 April 2019. doi: 10.1136/bmjebm-2019-111161 (Link)