Author’s note: I was asked by the MHRA (Britain’s FDA) to give a 20-minute presentation on Spontaneous Reporting Systems to a group looking at a birth defect related issue. This is close to what I said.
Are We Anecdotes?
On illustration (1) you see a recent article from Vice, which along with other new media outlets often offers better reporting on a range of issues from the Islamic Caliphate to healthcare than traditional media. This article on Pfizer’s varenicline – Champix in the UK and Chantix in the US – runs to 10 pages with great illustrations.
The piece outlines the case of Chris Kunkel who went on Champix to stop smoking. He found that it killed the craving for nicotine very effectively. But after being on the drug for a few weeks an urge to kill himself came over him just like the urge to go and get a sandwich from the fridge. He took a huge number of pills but survived because his wife came home early and found him. He had no prior history of mental illness, no reason to think he was going to commit suicide. His case. as outlined, was convincing.
The article makes it clear that there have been thousands of other spontaneous reports like this that appear to make a strong case. But the article also makes clear there are a large number of Randomized Controlled Trials (RCTs) that have been meta-analysed by Pfizer which show no signal of an increased rate of suicidal or homicidal events on Champix. When these studies are analysed by experts independent of Pfizer, they too have found no evidence of a linkage.
This leaves independent experts saying that they as scientists have no option but to go with what the scientific literature shows. That the Chris Kunkel and other cases are simply anecdotes. It’s just as though someone gave an MMR vaccine to one of their children who later develops autism and claimed that the vaccine must have caused the problem. This is anecdotal.
“These are anecdotes” is not a reasonable characterization of many of the reports of what happened on Champix, and it is not the case with drug-induced injuries in general.
The strongest evidence we have as to whether a drug causes a problem does not come from RCTs or any other controlled study but rather from good clinical accounts which include the elements of challenge, dechallenge and rechallenge.
Illustration (2) is the ‘Christmas tree light bulb’ test. When Christmas trees had real light bulbs rather than diodes, invariably when they were taken down after being stored for a year the lights didn’t work. The trick was to unscrew each bulb in turn until you unscrewed one and the bulbs came on. You could screw that back in again and the bulbs would go off again. So you removed it and threw it away and the lights worked fine. This is absolutely conclusive evidence that this was the dud bulb. Similar evidence is the strongest we have that a drug can cause a problem. It doesn’t tell us how often the drug causes the problem but nothing gives us this information.
The trouble is challenge – dechallenge – rechallenge cannot be undertaken easily in the cases of drugs given during pregnancy. It is against that background that companies claim that RCTs offer the only gold standard. That all other kinds of observational studies are flawed.
RCTs: The Gold Standard Way to Hide Adverse Events
Some of you may be aware of the Restoring Study 329 article (illustration 3) that appeared in the British Medical Journal three months ago. It is now among the articles with the highest impact of any article in the last 25,000 that the BMJ has published.
Illustration (4) is taken from the article shows you at the bottom under the heading Keller et al what the original publication reported in terms of the suicidal events that happened in this trial of paroxetine given to adolescents.
In the box above it labelled SKB, SKB some years later conceded that there were more suicidal events than had appeared in the published article. In the box above that labelled FDA you see FDA stating that in fact they were of the opinion that there were more such events than GSK had now reported to them.
Finally in the box at the top labelled RIAT you see that in our publication we were of the view that there were yet more events than the ones FDA had found and in fact we think there are even more than what we’ve reported here.
This may hint at one reason why when experts analyse the Champix clinical trials they don’t find an increased risk. They are analysing the published literature for the most part and the published literature doesn’t necessarily show all that happened. The Study Reports companies increasingly post on websites do not offer the full data either.
On illustration (5) – labeled Box 2 – you see some of the many ways we found that data can be hidden in controlled trials.
- The company may use an idiosyncratic coding system. In the case of Study 329 they coded the suicidal events as emotional lability.
- There can be a failure to transcribe the raw data into the forms then used to analyse what happened – 15% of the events went missing in this way.
- There are various different ways to group the events that may hide the problem from an outsider who is unaware of the issues and not especially motivated to look more closely.
- In the trials of many drugs from statins to antidepressants, it is common that concomitant drugs will be given to both the active treatment and the placebo groups and this dilutes any apparent signal from the active treatment.
- There is usually no effort to analyse the events that happen during the taper phase of a trial – often the most dangerous phase.
Even if Done by Angels
But even if the trials were done by angels, so there was no hiding, no miscoding, nothing untoward, RCTs can still hide adverse events. In the case of the Champix studies it may be that plunging people into nicotine withdrawal is a dangerous thing to do and that compared with that Champix treatment comes up as being somewhat better even if Champix itself can cause people to become suicidal.
Is this bizarre possibility possible? Yes it is. You see here the proceedings of a symposium held in Cambridge in 1959 a year after imipramine the first antidepressant was launched. On the next two slides you see delegates to the meeting talking about the fact that imipramine which treated the kinds of patients that would otherwise have been referred for ECT, severely depressed patients, could in the opinion of these clinicians also cause people to become agitated and suicidal and that the problem cleared up once the drug was removed. I can show you more quotes than the ones you see here but there was no argument from anyone at the meeting against the notion that this wonderfully helpful drug could also cause people to become suicidal (Illustrations 6, 7, & 8).
That sets up the scenario you see in this next illustration (9), which is that if imipramine is an effective treatment for severe depression in the way that the SSRIs for example aren’t, then it will reduce the rates of suicidal events that happen in this trial compared to placebo even though it’s a drug that can cause people to have a suicidal event. If you were to put imipramine into the kind of trials that the SSRIs were put into, that is trials of patients with mild depression, you get the same result as you get with the SSRIs – namely that on the active treatment there is an increased rate of suicidal events compared to placebo (Illustration 10).
This kind of problem happens every time an illness and a drug can cause a superficially similar thing. It means that controlled trials are not a good way to discover whether a drug is causing a problem or not (Illustration 11).
I could show you a range of other tricks that companies can do with controlled trials. There are several ways to use the problem a drug causes to hide the problem that the drug might be causing.
* * *
The basic message (illustration 12) is that if you don’t know what the problem is to begin with, RCTs won’t help you. RCTs are principally useful when you know what you are doing in the first instance and they are used to demonstrate that you know what you are doing.
Even then they may come up with the wrong answer, but as in the case of the HRT studies where they suggested that it might not be a good idea for everybody in the entire universe to be taking HRT, even if the answer is wrong it puts the onus back where it should be which is on large and powerful corporations who have a lot of resources to pinpoint the populations where the benefit is likely to exceed the risk if they want to continue to make money out of vulnerable people.
So if controlled trials don’t sort the issue out for us as to whether a drug causes a problem, what are the other options? There are a variety of case controlled studies, cohort studies and nested cohort studies.
In the case of SSRIs and autistic spectrum disorder (ASD) there are 5 such studies as you see here (Illustration 13). When you analyse these you find that there is an almost two fold increase in the risk of a mother having a baby with ASD when she has been on an SSRI compared to when she has not been on an SSRI.
The standard company response to data like you see here is that in a few of the studies the confidence intervals touch the 1.0 line and their view is that because they do that there’s no increase in risk in these studies and because we have a mixture of studies some of which show an increase in risk and others which show no increase in risk, therefore in fact there is no problem.
In the next illustration (14) you see 17 studies which have looked at what could be termed developmental delay. I’ll just quickly draw your attention to the Malm et al study 7 lines down.
Broadly speaking when you sum all of these studies, you get a result which shows that there is a close to a doubling of the rate of developmental delay in children born to women taking SSRIs during pregnancy.
On the next illustration (15) you see a very provocative image of holocaust denialism. Texts like this ask you to note that General Eisenhower wrote a 600 page book about events from June 1944 through to the end of the War and didn’t once mention concentration camps. Do you think that if what people claim was really happening that he wouldn’t have mentioned it?
The issue here is not to impute motives of Holocaust Denialism to companies but to show you how a problem can be structured so that that is the de facto outcome you get.
Faced with a problem like ASD companies convene panels of experts within the company – the epidemiology group, the animal testing group, the pharmacological group, the pharmacovigilance group, the risk management group and others – all of whom will be given a portion of the problem to look at from their point of view. All groups are asked is there incontrovertible evidence from your point of view of a problem. All will answer no there isn’t. This is a system where it’s almost impossible for a problem to register.
The person coordinating this will often be a person who knows nothing about any of these areas. Their job is to coordinate. When faced with for instance evidence of the developmental delay studies this coordinator if asked whether it would be appropriate to include studies of developmental delay with the ASD studies would say “Well, we didn’t do that”.
Why not? “My brief was to only look at studies that had ASD specified in the article.
Do you not think that developmental delay is much the same thing? “I don’t know. I know nothing about these things, we simply stuck to articles that had ASD in their body.”
This is standard company approach and may explain what researchers find when they go into company archives and seem to be faced with overwhelming evidence that the company knew there was a problem but denied all knowledge.
A structure of this kind is put in place in the name of Objectivity. In practice it makes it impossible for the company to see the bigger picture.
In public, when a company is faced with the data from the previous slides for ASD or any problem, they say (Illustration 16):
- The ASD rate in these studies doesn’t exceed the background rate in the wider population
- There is no evidence that ASD (leaving out neuro-developmental delay (NDD) or that NDD is leaving out ASD) is caused by SSRIs.
- There is no evidence that ASD is caused by Prozac, or by Paxil – Seroxat or by Zoloft and if not caused by any individual SSRI cannot be caused by SSRIs.
- The risks are not statistically significant and therefore do not exist.
- They will claim that depression causes ASD, just like it causes birth defects and suicide and dependence and ingrown toenails.
- They will claim only RCTs give conclusive answers. This unthinking mantra allows companies to get away with a lot.
Registries & Objectivity
In terms of adverse events, there is one more option – pregnancy registries. These are a specific birth defect option and arguably the best option. FDA and some companies have put limited registries in place (Illustration 17). These have been piecemeal and voluntary. To really look at what is going on we need to exploit the possibilities of Big Data. Every woman who gets antenatal care has the details of the pregnancy and every drug she is taking recorded as is, including over the counter medication. If such material were made available in e-form, including data on the children followed out to five or ten years, we would be better able to pinpoint issues than we are now.
Who should analyse the data? A lot depends on your vision of what is needed.
Regulators and companies have been the traditional people to interrogate post marketing data but their brief has never been exploratory (Illustration 18). The SSRIs have now been on the market 25 years and the data linking them to Developmental Delay is just emerging – when we could have had warnings of this possibility 20 years ago.
For instance, take the Malm et al study that I mentioned earlier. This was a large epidemiological study by a good group of researchers who as a footnote at the end of their article noted a 10 fold increase in Foetal Alcohol Spectrum Disorder (FASD) in children born to women taking SSRIs during pregnancy. They clearly had no idea what to make of this (Illustration 19).
This incidental finding is exactly the kind of thing that needs someone other than an epidemiologist to spot possible connections. SSRIs can cause compulsive alcoholism, so the Malm finding is not surprising to some of us – including to pharmaceutical companies who are running trials of S2 and S3 blockers as a treatment for alcoholism because they are persuaded the link is solid.
It clearly was a surprise to Malm and would be to most epidemiologists or regulators. Spotting connections like this needs a convergence of different skill sets – exactly the opposite model to the one companies and regulators have in place. For any event that is new, almost by definition, motivation counts for more than expertise as expertise is expertise in what has happened that will often block perceptions of something new that breaks the mold. There is no more motivated group of people in this universe than mothers or mothers to be.
At the heart of the proposal that women should be the main interrogators of pregnancy registry data lies the question of objectivity (Illustration 20). Objectivity doesn’t come from mechanical exercises like RCTs or case controlled studies. It comes from having the best data you can get and having people with a range of biases look at that data to see if a consensus emerges. The range of “biases” can help a group see why what may appear to be a link really isn’t one and how things that might seem to make no sense initially actually do make sense. It’s when people with different starting points agree that we get objectivity. It comes from just the opposite approach to the one Companies and Regulators take – in the name of objectivity.
It is nineteenth-century paternalism to think that women will be scared by some oddity tumbling out of such a database and that they will act rashly as a result. Women as a group spend more time risk managing than men and will look for consensus and operate on the basis of that consensus. Telling women that they shouldn’t have access to this data is rather like telling them that they shouldn’t have control of their own bodies, that men or the state are going to decide for them. This might be the case in the Caliphate, but in London?
* * * * *
This blog is adapted from DavidHealy.org