This post was written by Ken Spriggs and comes from his DIYEHR (do it yourself electronic health record) site, and is the third in a series of ‘RxISK Stories’ taken from the RxISK.org website. It brings out nicely how some people are only aware of the problems their drug causes when they stop. This is particularly true for people taking statins, but can be found taking anything from aspirin to azathioprine. Poison is an emotive word but all drugs are poisons which taken chronically will poison. Think arsenic; when taken in the short-term it clears skin and eyes but long-term not so good. Ken writes:
I found myself browsing the second or third page of azathioprine related google results and an abstract about withdrawing from azathioprine when Crohn’s is in remission. It’s personal because I’m someone who stopped taking azathioprine against the advice of my physician. I ended the drug about 15 months ago and I’ve not only remained in remission but I no longer have to deal with the side effects. There’s a lot I’d like to discuss about the article because I disagree with the conclusion and would like to question their process as well.
Title: A Randomized, Double-Blind, Controlled Withdrawal Trial in Crohn’s Disease Patients in Long-term Remission on Azathioprine Source: Gastroenterology – June 2005 (Vol. 128, Issue 7, Pages 1812-1818, DOI: 10.1053/j.gastro.2005.03.031)
Here’s a link to the full article and below is the abstract in its entirety. See what you think:
Background & Aims: An open study reported that patients with Crohn’s disease in remission who have taken azathioprine for longer than 3.5 years are at low risk of relapse when azathioprine is discontinued. To confirm this observation, we performed a multicenter, double-blind, non-inferiority withdrawal study. Methods: Patients who were in clinical remission on azathioprine for ≥42 months were randomized to continue azathioprine or to receive an equivalent placebo for 18 months. The primary end point was clinical relapse at 18 months. Results: Forty patients were randomly assigned to receive azathioprine and 43 to receive placebo. Characteristics of patients at entry were similar in the 2 study groups. At 18 months, 3 patients had a relapse in the azathioprine group, and 9 had a relapse in the placebo group. Kaplan-Meier estimates of the relapse rate at 18 months were 8% ± 4% and 21% ± 6%, respectively. The hypothesis that placebo was inferior to azathioprine was not rejected (P = .195). Among the baseline variables, C-reactive protein level >20 mg/L, time without steroids <50 months, and hemoglobin level <12 g/dL were found to be predictive of relapse in the multivariate analysis. Conclusions: This study shows that azathioprine withdrawal is not equivalent to continued therapy with azathioprine for maintenance of remission in patients with Crohn’s disease who have been in remission on azathioprine for ≥3.5 years. Thus, azathioprine maintenance therapy should be continued beyond 3.5 years.
Ok so I had to read that last sentence twice because it doesn’t seem to go with the rest of the abstract. To me it looks like in the placebo group there were 34 out of 43 which didn’t relapse. That’s phenomenal! That means 79% don’t need azathioprine! Yeah! No more achy joints, nausea, fatigue, diarrhea, trips to the pharmacy, money spent, blood tests, not to mention azathioprine is listed as a carcinogen.
Let’s look at the azathioprine group, they experienced 3 relapses out of 40. If you apply that knowledge to the placebo group it implies that not 9 people relapsed because of azathioprine withdrawal but only 6 relapsed. Three, we expect, would have relapsed anyway. This means we should expect 37 out of 43 to remain in remission. That changes the 79% to 86%! Even better!
Based on the evidence here’s the conclusion I would think is more responsible to give a Crohn’s patient: There was as trial done which included a fairly small number of patients and it showed that if you’ve been in remission (a Crohn’s Disease Activity Index score of less than 150) for at least 42 months and are still taking azathioprine then you can consider stopping. You stand an absolute chance of 9 out of 43 (21%) of coming out of remission but only a 6 out of 43 chance (14%) of coming out of remission because you stopped azathioprine.
Contrast this with their conclusion: “Thus, azathioprine maintenance therapy should be continued beyond 3.5 years.”
If you didn’t get my point think about it this way. Suppose you had a treatment which worked for 79% of patients and it had no side effects to speak of. Would this not be a wonder drug? Of course it would! Call the marketing department, this one’s gonna be an easy sell! In this case the “wonder drug” is NOT taking the azathioprine and, unfortunately, there is no marketing department to call for that.
Further consider you’re one of the 34 patients in the placebo group who remained in remission for 18 months and then your doctor tells you that you were in the placebo group but to start taking your azathioprine again. What? Would this seem like reasonable advice? There’s no possible way I’d start the medication again. Why would anyone in remission want the side effects? It’s a senseless conclusion for the vast majority of the placebo group.
To turn their conclusion on its head again, we can say that staying on azathioprine only helps about 14% of patients stay in remission.
This brings me to questions about methodology and resourcefulness. The paper acknowledges that azathioprine has side effects but it makes no attempt to quantify them. What they could have done is to ask the study participants at the end of the trial if they thought they were in the placebo group or not. It’s crazy to me the researchers overlook this. Participants could also be asked if their quality of life improved during the trial. That’s actually the most important thing of all! Suppose 79% of participants in the placebo group said their quality of life improved while no one in the azathioprine group did. (It’s also possible, though unlikely, for someone who did relapse to report that their quality of life improved. This would happen if the side effects were worse than the relapse.) But alas, this information is MIA. But why? Quantifying side effects should mean a lot to a study about a drug. It could always turn out that side effects are worse than treatment. These researchers completely ignore this possibility. Bad science.
To shore up this massive design flaw in the study participants should be asked questions about the side effects of azathioprine before the study begins, during the study, and also after.
I’m reminded of a quote by David Healy from Pharmageddon: There is no evidence based approach to determining whether treatments have injured a patient or what to do when it happens. Why ever not?
(RxISK.org is a patient oriented organization dedicated to researching and reporting side effects.)
There’s another important issue I’d like to touch on. When I went through my simple analysis I didn’t once mention statistical significance. In order to disagree with the conclusion there was no need to think about statistical significance, we just had to count.
Now I’d like to question that perhaps the entire design of the experiment was amiss. Bare with me cause this is gonna get simple. The first sentence of the abstract says: “…patients with Crohn’s disease in remission who have taken azathioprine for longer than 3.5 years are at low risk of relapse when azathioprine is discontinued.” Why do you need statistical significance to answer this? You don’t. Instead what you do is gather a bunch of people who meet the criteria and tell them to stop taking their azathioprine. Then count the number of relapses over the next 18 months. That’s it. Really. You can then report that X% of patients relapse when they withdrew from azathioprine. If you wanted to say how many relapsed because of azathioprine withdrawal then you’d have to have one group stay on the medication and count the number of relapses. My take on the second step is to say it might be interesting to try and predict who will relapse based on bloodwork results or genetics or something related to lifestyle and diet. I’d guess the sample size of participants needed probably renders the idea prohibitive. And if a study like that isn’t plausible then the added knowledge is academic since it’s not possible to have any idea who will relapse. I think the study wasted half the participants, the entire control group isn’t necessary.
Ok so now you’re wondering why these authors with their PhD’s and MD’s did what they did. Well, they followed ”This method universally used in non-inferiority trials…”. When I read something like that I’m reminded of the Sokal affair and Fashionable Nonsense. As these ideas apply to medicine Healy sums it up with this quote from Pharmageddon:
Explaining why Fisher’s ideas have such traction within medicine is not easy. Regulators have followed this line because the definition of statistical significance offers them a rule of thumb, an almost mechanical procedure that takes the place of a judgment call. For pharmaceutical companies, the issue is simple; Fisher’s ideas mean that positive effects in a minority of clinical trials can transform a weak and inessential drug into one apparently certified by science, while at the same time airbrushing its hazards out of existence.
But why do doctors follow this line? A number of medical academics have attempted to grapple with this, pointing out that current dependence on statistical significance testing has created a “junk epidemiology” in the domain of therapeutics. So Louis Lasagna, the first professor of clinical pharmacology in the United States, and later dean of medicine at Tufts University, who introduced randomized controlled trials into drug development, described the approach outlined above as “p-value madness”. For Sandor Greenland, professor of epidemiology and statistics at UCLA, “statical thinking [of this type] has produced a chronic psychosis” – by which he means that researchers relying on Fisher’s idea have lost touch with reality. Ezra Hauer, a professor of civil engineering from the University of Toronto and authority on analyzing road traffic accidents, explains that “in this manner good data are drained of real content. The direction of the empirical conclusions is reversed and ordinary human and scientific reasoning is turned on its head.” For Charlie Poole, professor of statistics and epidemiology at the University of North Carolina, “Statistical significance should be abandoned immediately and universally.” Ironically when faced with this issue in 2011, with investors’ dollars at stake, the US Supreme Court argued that statistical significance cannot be the arbiter of what an investor might deem as significant risk. But patients it seems do not have the same rights as investors.
The differences between much of clinical practice and other branches of science were starkly framed by Kenneth Rothman, professor of epidemiology at Harvard and editor of the journal Epidemiology, in a note about submissions to the journal:
“When writing for Epidemiology, you can . . . enhance your prospects if you omit tests of statistical significance… We would like to see the interpretation of a study based not on statistical significance, or lack of it, for one or more study variables, but rather a careful quantitative consideration of the data in the light of competing explanations… Misleading signals occur when a trivial effect is found to be ‘significant,’ as often happens in large studies, or when a strong relation is found ‘non-significant,’ as often happens in small studies.”
At the beginning of this post I mentioned I stopped taking azathioprine. It was about 3 and a half years after surgery and I’d been in remission the entire time. It’s been about 15 months since I stopped talking it and I haven’t felt this good since I was 17 and Crohn’s was years away from wrecking me. So here’s what makes me upset: Azathioprine has serious side effects. When I was on it I suffered diarrhea, chronic fatigue and my knees often ached as if I had arthritis. After I stopped taking the drug these symptoms cleared up in about 3 weeks with the exception of the diarrhea. I was like a new person. I went and took tennis lessons, joined a league, and started getting out of bed much easier and an hour earlier. The researchers never even tried to quantitate the effects of the drug. It was passively acknowledged at most. Side effects weren’t even considered in the conclusion. There’s probably physicians who have read this paper and they go on prescribing azathioprine even though someone’s been in remission for years. It’s sad and it’s irresponsible. Patients should demand an exit strategy from their physicians but that’s a different issue.
Lastly a few more questions to ask: What was the financial role of GlaxoSmithKline? The company was mentioned as providing the azathioprine in the form of their brand name Imuran but that’s it. Was the paper ghostwritten? Do the researchers have financial ties to GSK?