In a new study, researchers found that four of the five original studies on the effectiveness of alprazolam (Xanax) found it to be no better than placebo. Two of the negative studies remain unpublished, while two more were spun to appear positive in publication despite the drug’s failure to beat the placebo. Only one of the five studies considered by the FDA in approving the drug actually showed a positive result.
The conclusion? “Publication bias substantially inflates the apparent efficacy of alprazolam XR,” the researchers write.
Xanax is the most widely-prescribed benzodiazepine in the United States, according to the researchers. It is also the most commonly abused benzodiazepine.
The article was written by Rose Ahn-Horst at Harvard Medical School, Massachusetts General Hospital, and McLean Hospital, and former FDA reviewer Erick Turner at Oregon Health & Science University and the Veterans Affairs Portland Health Care System. It was published in Psychological Medicine.
Turner has used FDA data to illuminate the influence of publication bias and lack of efficacy of other drugs as well, including antidepressants and esketamine.
Because the FDA requires that pharmaceutical companies submit data on all trials that will be used for regulatory approval, it is possible to examine the data from all the pivotal studies conducted rather than just the ones that end up published.
In this case, the researchers found that the FDA received data on five Phase 2 or Phase 3 trials of Xanax extended release—alprazolam XR—for panic disorder. The FDA considered four of the studies to be negative—showing that the drug did not beat placebo on all primary endpoints. They approved the drug on the strength of the fifth study, which was considered positive.
Study 1, conducted between 1986 and 1989, was presented as a positive result in its published version. However, it was considered negative by the FDA statistician because it only achieved statistical significance on five of the seven primary endpoints. The published version omitted the negative results, spinning the study as positive.
Study 2, conducted between 1988 and 1990, was published as a positive study, and the FDA agreed—all seven endpoints were statistically significant. However, data from one of the three study sites was destroyed before the FDA analysis by the study investigator, and the FDA reported that the “validity of the data reported could not be verified.” This was the positive study used by the FDA to approve the drug. Notably, the FDA had initially required the usual two positive studies, but according to the researchers, it relaxed this requirement in this case.
Study 3, conducted between 1990 and 1991, was a complete failure, finding the drug to be no better than placebo on any of its primary outcome measures. All five outcomes were nonsignificant. Yet this study was published as a positive trial after the researchers omitted the primary endpoint and presented only “preliminary results” using a different statistical method than they had used in the FDA’s records.
Study 4, conducted between 1990 and 1991, was nonsignificant on all five primary outcome measures. The study was never published.
Study 5, conducted between 1994 and 1995, was considered a negative trial by the FDA, and they did not include more information about their statistics in this case. The study was never published.
When analyzing the published literature, the researchers found an effect size of 0.47 in favor of Xanax (alprazolam). However, when they added the unpublished data, they found that the effect size shrank to 0.33.
According to the researchers:
“These findings arguably alter the risk-benefit ratio for the prescribing of this benzodiazepine, especially in the light of recent attention to their contribution to the opioid crisis and the availability of safer alternatives.”
Ahn-Horst, R. Y., & Turner, E. H. (2023). Unpublished trials of alprazolam XR and their influence on its apparent efficacy for panic disorder. Psychological Medicine, 1-8. https://doi.org/10.1017/S0033291723002830 (Link)