Survival analysis is a statistical technique often used to test whether psychiatric drugs prevent relapse. But, in a new article in BMJ Evidence-Based Medicine, researchers write that it is inappropriate to use it for that purpose.
The researchers were Joanna Moncrieff at University College London, Janus Christian Jakobsen at the University of Southern Denmark, and Max Bachmann at the University of East Anglia.
They note that while this method may be appropriate for some situations when it comes to psychiatric drug studies, it is prone to conflating withdrawal with relapse and can lead to misleading conclusions. Thus, they suggest it should be avoided for these studies. The researchers write:
“We suggest that survival analysis should not be routinely employed in trials of interventions aimed at relapse prevention in long-term psychiatric conditions.”
In survival analysis, researchers generally compare one, or both, of the following: whether there are more relapses off the drug and whether those relapses happen sooner.
The first question is a vital one: do people taking the psychiatric drug being studied suffer fewer relapses? But the problem is that discontinuing the drug leads to withdrawal effects, which are often counted as relapse, especially in drug trials. So, because of withdrawal, people who discontinue the drug appear to relapse more—even though it is actually drug withdrawal.
The second question is more troubling: what if there are the same number of relapses over time, but the relapses happen sooner for those who discontinue the drug? And again, many of those “relapses” in the discontinuation group may actually be drug withdrawal effects.
Moncrieff, Jakobsen, and Bachmann suggest that in this situation, although the drug looks better statistically, it is not necessarily worth the adverse effects of the drug. That is, if the same number of people ultimately relapse whether they are taking the drug or not, then the drug may not be worth it—even though it looks better statistically.
The researchers note that there has been no research into whether a slightly delayed relapse is a clinically relevant outcome. So they ask whether patients would opt to continue the drug indefinitely if they knew their chances of relapse were the same if they stopped it.
“The clinical relevance of a temporary delay in relapse in a long-term psychiatric condition that may last for decades has not been established, and statistically significant results are a questionable basis for implementing an intervention that may be of limited importance to patients,” they write.
To clarify this, the researchers use two examples. The first is a study of whether antipsychotic drugs prevented relapse after the first episode of psychosis. At the 18-month follow-up, it seemed that the risk of relapse was greater for those who had discontinued the drug. But at the 7-year follow-up, the risk of relapse had flipped—it was now greater for those who remained on the drug.
The second example involves using esketamine for relapse prevention in treatment-resistant depression. In the first eight weeks, people who did not take the drug had higher relapse rates. But by about the 9-month mark, there was no difference between the groups in terms of relapse risk. The researchers write:
“Evidence of a withdrawal effect has a bearing on the cost-benefit analysis of starting treatment, and is particularly important in view of the fact that acute trials of esketamine have not demonstrated a clinically relevant effect.”
Ultimately, they argue that survival analysis is misleading and should be avoided in trials of psychiatric relapse prevention. Additionally, they point to the importance of long-term follow-up since withdrawal effects bias the short-term results.
Moncrieff, J., Jakobsen, J. C., & Bachmann, M. (2021). Later is not necessarily better: limitations of survival analysis in studies of long-term drug treatment of psychiatric conditions. BMJ Evidence-Based Medicine. (Link)