Critical Psychiatry Textbook, Chapter 6: Psychiatric Drug Trials Are Not Reliable

7
1905

Editor’s Note: Over the next several months, Mad in America is publishing a serialized version of Peter Gøtzsche’s book, Critical Psychiatry Textbook. In this blog, he discusses the ways in which drug trials are biased, including breaking of the double-blind and industry manipulation. Each Monday, a new section of the book is published, and all chapters are archived here.

There is probably no other area of medicine in which the academic literature is so at odds with the raw data.
—David Healy, professor of psychiatry122

I have spent most of my professional life evaluating the quality of clinical research, and I believe it is especially poor in psychiatry. The industry-sponsored studies … are selectively published, tend to be short-term, designed to favor the drug, and show benefits so small that they are unlikely to outweigh the long-term harms.
—Marcia Angell, former editor, New England Journal of Medicine123

Before discussing the effect of psychiatric drugs, we need to realise that most placebo-controlled trials and head-to-head comparisons of two active drugs are heavily biased.7,8 As an example, significantly more patients improved on fluoxetine when fluoxetine was the drug of interest than in trials where fluoxetine was the comparator drug.124

There are eight major reasons why placebo-controlled trials of psychiatric drugs are flawed, and most of them apply also to head-to-head trials.

Illustration of a magnifying glass and a pill bottle on a pink backgroundBy far, most drug trials run for only a few weeks, even though most patients are treated for many years. This is not evidence-based medicine, and drug effects can change over time due to development of tolerance.

Events after the trial stopped are ignored. The main effect of this is that the harms are underestimated (as I will discuss in Chapters 7 and 8).

By far, most trials are conducted by the drug industry, which often manipulates the design, analysis, and reporting of results.6,7

About half of the deaths and half of the suicides occurring in trials of psychiatric drugs have been left out of the published trial reports.125

The four additional reasons are the use of rating scales, lack of effective blinding, withdrawal effects in the placebo group, and manipulated data analyses and selective reporting.

Rating scales

Rating scales are used for measuring the reduction of symptoms. They do not say anything about whether the patients have been cured or can live a reasonably normal life.

The scores on such scales may easily improve even if the patients have not been helped, e.g. when someone is knocked down by a major tranquilliser (a psychosis drug) and expresses abnormal ideas less frequently.4

The effect of depression pills is measured on rating scales, e.g. the Hamilton Depression Rating Scale,126 which contains items that are not specific to depression, including sleeping difficulties, anxiety, agitation, and somatic complaints. These symptoms are likely to respond to the non-specific sedative effects that occur not only with many depression pills but also with other substances, e.g. alcohol, opioids, psychosis pills, and benzodiazepines, but we do not prescribe alcohol, opioids, or benzodiazepines for people with depression or call them depression pills.

Using the Hamilton scale, even stimulants like cocaine, ecstasy, amphetamines, and other ADHD drugs could be considered depression pills. Almost everything could. Indeed, many drugs that are not considered to be depression pills show comparable effects to them, e.g. benzodiazepines, opiates, buspirone, stimulants, reserpine, and other psychosis pills.24

Lack of effective blinding

Because of the conspicuous adverse effects of psychiatric drugs, placebo-controlled trials labeled double-blind are not double-blind. It takes very little unblinding before the small differences recorded can be explained by bias in the outcome assessment on a subjective rating scale.7:51

The blinding is broken for many patients in these trials, in some cases for all of them, as in a trial of alprazolam versus placebo.127 Researchers had reviewed the blinding problems and ended their paper by saying that, “The time has come to give up the illusion that most previous research dealing with the efficacy of psychotropic drugs has been adequately shielded against bias.”127 This was in 1993, but the psychiatrists and the drug industry ignore this fundamental flaw in their research because it is useful for them to pretend the problem doesn’t exist and that what they measure are true and beneficial drug effects.

The unblinding is a major reason why it is so much easier to invent new diseases than to invent new drugs.128,129 When my research group reviewed the type of diagnoses investigated in placebo-controlled trials of depression pills, we counted 214 unique diagnoses, in addition to depression and anxiety.130 The trials were driven by commercial interests, focusing on prevalent diseases and everyday problems to such an extent that no one can live a full life without experiencing several of the problems for which these drugs were tested. We concluded that depression pills are the modern version of Aldous Huxley’s soma pill intended to keep everyone happy in Brave New World.

In 2001, Lundbeck’s American partner Forest had performed a trial of citalopram for compulsive shopping disorder.131 Another Lundbeck drug, escitalopram, reduced the daily incidence of hot flashes in menopausal women from 10 to 9.132 This also looked like a joke but the trial was published in a US flagship journal, JAMA.

Helped by the lack of blinding, drug companies can show that their drugs “work” for virtually everything. Just think about the variety of drugs that are claimed to work for depression and schizophrenia.

Healthy volunteers who pretend they are ill can help, too. Some patients participate in depression trials without being depressed just to cash the money, as a healthy person told a doctor on a train ride:133 “I’m not depressed … the trials are advertised … For a 20 day trial that’s £2000 … it’s nice to see your regular friends.” When the fake volunteers notice they are on active drug because of its adverse effects, they can fake some improvement.

Withdrawal effects in the placebo group

The patients recruited for placebo-controlled trials are virtually always already on a drug like the one being tested. This is because such patients are much easier to find than patients who are not on drugs.

After a short wash-out period without this drug, the patients are randomised to the new drug or placebo. The patients are likely to be those who have not reacted too negatively on getting such a drug before,24 which means that the trials will underestimate the harms of the tested drug.

The patients might also react more negatively to placebo, e.g. because they miss the sedation or euphoria these drugs may cause (see, for example, the package insert for olanzapine134).

Some patients get withdrawal symptoms that are misinterpreted as a relapse of the disease because the symptoms can be the same as those that define a relapse. Introducing longer wash-out periods does not remove this cold turkey problem. If people have been permanently brain-damaged before entering the trials, wash-out periods cannot compensate, and the symptoms that have been masked by ongoing treatment (e.g. tardive dyskinesia) might reappear. Even if that is not the case, the patients might suffer from withdrawal symptoms for months or years.11,135,136

Manipulated data analyses and selective reporting

When Joshua Carp criticised the brain imaging studies (see Chapter 3), noting that when analytic flexibility is high, investigators may elect to use methods that yield favourable outcomes and discard methods that yield null results,73 he cited a study carried out by my research group.

We compared protocols for randomised trials we had obtained from ethics review committees with the trial publications.137 Two-thirds of the trials had at least one primary outcome that was changed, introduced, or omitted, while 86% of the trialists denied the existence of unreported outcomes (they did not know, of course, that we had access to their protocols when we asked). These serious manipulations were not described in any of the 51 publications.

This was the first time this phenomenon had been shown to be common, in a consecutive cohort of trials. Other attempts at getting access to trial protocols had failed, but I succeeded to get access in Denmark by guaranteeing that we would not describe the individual trials in our publication.

What we uncovered is known as the Texas sharpshooter trick. You fire a gun towards a target but miss it. Next, you erase the target and draw a new one around your bullet hole and pre-sent this to the public. You hit the bull’s eye by committing fraud.

Based on our data, we did another study that is also relevant to know about when judging the trustworthiness of published trial reports.138 In those 44 trials that were industry-initiated, the sponsor had access to accumulating data during 16 trials through interim analyses and participation in data and safety monitoring committees, but such access was disclosed in only one corresponding trial article. An additional 16 protocols noted that the sponsor had the right to stop the trial at any time, for any reason, which was not noted in any of the publications. The sponsor therefore had potential control over a trial in progress in 32 (73%) of these studies.

When the sponsor can peep repeatedly at the data as they accumulate, there is a risk that the trial will be stopped when it is favourable. Trials reported as having stopped early for benefit exaggerated the effect by 39% on average compared to trials of the same intervention that had not stopped early.139

We also found out that constraints on the publication rights were described in half of the protocols, which noted that the sponsor either owned the data, needed to approve the manuscript, or both. None of the constraints were stated in any of the publications.

Ghost authorship is also an important issue. It is the failure to name, as an author, an individual who has made a substantial contribution to an article. We found that none of the 44 protocols stated that clinical investigators were to be involved with data analysis.140 There was evidence of ghost authorship for 33 trials (75%), which increased to 91% when we included cases where a person qualifying for authorship was acknowledged rather than appearing as an author. In 31 trials, the ghost authors we identified were statisticians. We likely overlooked some ghost authors, as we had very limited information to identify the possible omission of other people who would have qualified as authors.

A study by David Healy of papers on sertraline (Zoloft, Pfizer) showed that in a three-year period, 55 papers had been written by a medical writing agency whereas only 41 papers had been written by other people.141 Only two of the 55 papers acknowledged writing support from people not listed as authors, and all results were favourable for Pfizer. Healy has described how frank some companies are towards doctors: “We have had our ghostwriter produce a first draft based on your published work. I attach it here.” When Healy was unhappy with the glowing review of a drug and suggested changes, the company replied that he had missed some “commercially important” points and published the paper in another academic’s name.142

What we uncovered based on the protocols was new and it shocked the international research community. There were many comments about our results in scientific journals and the media.

I have been a kind of research detective all my life, and I once participated in a team with Richard Smith, Editor-in-Chief of the BMJ, that investigated a case of fraud in a trial committed by a researcher in Asia.

Fraud is much more common than people realise.27 In 2021, Smith wrote about research fraud in the article, Time to assume that health research is fraudulent until proven otherwise?143 He mentioned that a colleague had informed anaesthesiologist Ian Roberts that none of the trials he had included in a systematic review showing that mannitol halved deaths from head injury existed. All of them had a lead author who purported to come from an institution that didn’t exist. The trials were published in prestigious neurosurgery journals and had multiple co-authors, some of whom didn’t know they were authors until after the fake trials were published. When Roberts contacted one of the journals, the editor responded that, “I wouldn’t trust the data.” Roberts wondered why he then published the trial. None of the trials have been retracted.

Also in 2021, an analysis of individual patient data in 153 randomised trials submitted to Anaesthesia showed that 44% had untrustworthy data and 26% were fatally flawed, i.e. 70% were garbage.144 When individual patient data were not available, it was more difficult to detect scientific misconduct, and now “only” 22% were garbage.

It is clear that we cannot take science on trust but must investigate every time if it is reliable. Roberts, an editor in a Cochrane group, has stated that it is a huge mistake that the motto for Cochrane reviews of trials is “Trusted evidence.”145 This motto was introduced by Cochrane’s new CEO, journalist Mark Wilson, shortly after he took office in 2012. It sounds like self-praise from a drug company, which reflects that Wilson was marketing-oriented and did not understand what science is about. He ruined the Cochrane Collaboration and suddenly left in a middle of April 2021 after its major funder had declared that it would cut its funding substantially.146

By far most Cochrane reviews of psychiatric drugs are unreliable because most of the included trials are unreliable, and the Cochrane authors and editors are not sufficiently critical towards the source material.

***

To see the list of all references cited, click here.

***

Mad in America hosts blogs by a diverse group of writers. These posts are designed to serve as a public forum for a discussion—broadly speaking—of psychiatry and its treatments. The opinions expressed are the writers’ own.

***

Mad in America has made some changes to the commenting process. You no longer need to login or create an account on our site to comment. The only information needed is your name, email and comment text. Comments made with an account prior to this change will remain visible on the site.

7 COMMENTS

  1. Peter,

    As you stridently and poetically write about Vast Pharma being like a machine, I send you this song because…

    “We must Rock this Machine until it Stops”.

    https://youtu.be/HZtP8NaGYV0

    Rock The Machine.

    I’m losing will, love
    My hands are soft as cotton gloves
    Machine has eaten up my job
    My meaning, my cause

    Machine with the strength of a hundred men
    Can’t feed and clothe my children
    Can’t greet a sailor coming in
    Or know of desperation

    I shovel coal down Spencer’s dock
    I took your dear into wedlock
    And you did bear us a beautiful flock
    River rock the machine down on the docks
    River rock the machine until it stops

    Oh low Liffey, show me gold
    Like a heron I will go
    Oh roaring Liffey when you’re high
    Like a cormorant I will dive

    In the Galway glow at night we rise
    We’re gold enough to win back time
    We’re gold enough to win back the pride
    River rock the machine until it dies

    Today, dear, I’m unemployed
    Before I spoke, you read my eyes
    I feared of nothing or no one
    But I feel old and I’m so young

    I miss the graft, I miss the boys
    I plead for purpose in the void
    Time is inward, ticking tighter
    I wish my load was a mountain lighter

    I shovel coal down Spencer’s dock
    I took your dear into wedlock
    And you did bear us a beautiful flock
    River rock the machine down on the docks
    River rock the machine until it stops

    Oh low Liffey, show me gold
    Like a heron I will go
    Oh roaring Liffey when you’re high
    Like a cormorant I will dive

    In the Galway glow at night we rise
    We’re gold enough to win back time
    We’re gold enough to win back the pride
    River rock the machine until it dies

    River rock the machine down on the docks
    River rock the machine until it stops

    Songwriter: Lisa O’neill

    Report comment

  2. https://youtu.be/wQGUYgNhQa4

    …just to add…having quit antipsychotics abruptly some years ago, and having found that experience gruelling and nail biting, I said I’d give you a note of how I might find quitting antipsychotics for a second time most recently, after not going off abruptly but rather easing off with a long slow liquid tapering. I can now say that this second withdrawal has been superior in its effortlessness. The gradual tapering has been painless. It is the difference between night and day.

    Report comment

    • Glad to hear you’ve been slowly weaned off the antipsychotics, Daiphanous Weeping, and that the “gradual tapering has been painless.” But – as one who was slowly weaned off the psych drugs, including the so called “antipsychotics,” twice – I imagine you know, as a regular reader of this website, that a drug withdrawal induced manic psychosis can happen … even years later. So the trick is to stay off those drugs.

      And I think the trick is to tell any psychiatrist you might run into in the future that you are allergic to the anticholinergic drugs. This informs the psychiatrist that you are knowledgable about anticholinergic toxidrome, and that they should prescribe neither the antidepressants nor antipsychotics.

      At least this “trick” – slice of knowledge – helped me fend off, and embarrass, future greed only inspired psychiatrists.

      Thank you, as always, for your honesty, Dr. Peter.

      Report comment

      • Hello Someone Else,
        Thank you for responding. I read it. You seem to be “preaching to the converted”, as the old saying goes, where it comes to having a dread of antipsychotics. Something I arrived on MIA already with. What many may not understand is that what I call this experience of schizophrenia is even MORE abysmal to endure, and this is why so many unfortunate sufferers of tormenting hallucinations feel like finishing themselves off, which is often a lethal prerequisite for imbing any noxious pie in the sky “cure all”, be it strong alcohol, heroin, antipsychotics, and any other method of potentially life saving distraction. I feel sympathy for people with hellish hallucinations, that make them no longer wish to be living, if they freely choose for themselves a “fix” out of despair that I find incorrigable, if it stops them suiciding on a monday or a wednesday or a friday. Yes the “fix” probably will bite them back via addiction or difficulty coming off again. There are obviously better ways to mend. I am with you on your desire to end the prescribing of psychiatric medications. That said, I am not everybody. I cannot speak for everybody.
        The tip about telling a psychiatrist of being allergic is a good one. Although to be honest it was my PSYCHIATRIST who came up with the idea of giving me a liquid form of the antipsychotic SO THAT I COULD COME OFF ANTIPSYCHOTICS. And it was my PSYCHIATRIC NURSE who went and got me tiny syringes so that I could decrease the dose down to atoms. But here marks a division between those in MIA who are okay to see psychiatry evolve into something more humane and those who just want psychiatry scrapped. I am not convinced that society can do without some form of “help” for people driven so crazy by hallucinations that they want to go shoot up a kindergarten or jump off a bridge or snuff out their newborn ten hours after giving birth. The nature of that help obviously needs to change. The medications are poisons as far as my own experience guides me to feel. I ended up almost dead of neuroleptic malignant syndrome. But EVEN after that near fatal catastrophe, some years later, and suffering nightmarish hallucinations, I wished it had sent me to my eternal peace, and that desperation was what prompted me to “try anything”, which inevitably had me re-trying antipsychotics again, even after knowing they could destroy me. If that does not tell people anything about how grim it is to live with severe unending hallucinations 24/7, that one prefers to risk life itself just to stop them, then I don’t know what could.
        You have your valid opinion on the possibility that any psychiatric medication can cause mania. I agree that the brain is very delicate to whatever seeps into it. But my own schizophrenia began long long long before I ever took any medication. Maybe I am a rarity. We should hold out for the possibility that some people have bodies and brains that are so. I do not believe in easy answers for all, as if we are all the saaaaaame. That view seems similar to the blanket presumptions of old psychiatry. There needs to be an openness to “difference”. That means we need to keep open to the possibility that YES there is an “anticholinergic toxidrome” for some people….and YES there is not this for other people. Any replacement to psychiatry ought to honour this extent of “individual uniqueness” or NOTHING has been replaced at all.

        Laughably, I hope that you do not feel here that “I, myself” have been trying to “vandalize” your “dearly held opinion”. I want a world where YOU continue to beat your OWN drum LOUDLY and LOUDER yet.

        Even though it is not quite what I say, I like your FREEDOM to say what YOU say.

        The music I put is my way to celebrate coming off antipsychotics, even if I die from having done so, because for me, I was dying hourly by being back on them.

        Heresy is not when one person hotly disagrees with everything a metaphorical campaign is doing. A heretic is unwelcomed precisely because they really DO agree with MOST of what the campaign stands for but just not quite ALL of it. It is this “close proximity” to the brotherly bonded that gets them deemed “disloyal” by virtue of a minor “difference” of “held opinion”. An accusation of betrayal is not felt towards the glaringly separate, but towards the “almost intimate”. What often matters though, even at times mattering moreso than brotherly bonding, is that we each stay “intimate moreso” to “OUR OWN” feeling, and our OWN opinion. Life is too short to be out of touch with OUR OWN PERSONAL vision of what may make an improvement to our complicated shared world. There is nothing amiss about having a dearly held vision, so long as we realize there are possibly eight billion “different” ones.

        THAT realization is probably the widest vision of all.

        Report comment

  3. I like the idea of this Critical Psychiatry Textbook. Unfortunately, at least where I live (India, but I assume it’s true for most other places), psychiatrists will just hand-wave all this away to the detriment of the suffering people who unfortunately end up in their offices due to a lack of choice. They’ll dismiss it as “internet nonsense”, “no credibility” even if it comes from a highly credentialed person. It’s an incredibly rare professional who will heed the advice given here.

    It’s a miserable situation for the suffering. But it’s the truth. This will largely be stuck to the niche diaspora on Mad in America who largely already know these issues. I hope it’s not the case, but frustratingly, it’s true.

    Report comment

LEAVE A REPLY