Jonah Lehrer was also Wrong About Antipsychotics

We spend a lot of time writing about knowledge dissemination in mental health, and over time, have increasingly recognized the important role of science journalists in our society. Thus, we have watched the recent rise and fall of Jonah Lehrer with great interest. Mr. Lehrer wrote a piece for the New Yorker last year that addressed antipsychotics. However, his well-written and entertaining piece didn’t seem to reflect the data on how pharmaceutical companies have promoted antipsychotics – instead, it seemed like he had jammed two narratives together that didn’t quite fit, at best not taking all the available into account, at worst, discarding the data that didn’t fit his story. At the time, we were a bit puzzled, and wrote a blog post about it (see below). As it turns out, this turns out not to be an anomaly but more a pattern of behavior, as he just resigned his position at the New Yorker for inventing quotes in a recent book.

We don’t mean to “pile-on”, as this is a tragic circumstance, but we noticed how many people read Lehrer’s piece who do not read, for instance, Mad in America.com, and feel it was a missed opportunity to inform the public of a very interesting story – the overestimation of the efficacy and safety of the newer antipsychotics due to enthusiastic and overwhelming promotion by pharmaceutical companies and their associates. Unfortunately, that story apparently didn’t fit the narrative Lehrer wanted to write.

We badly need good science writers, especially those who do rigorous work and consider all the available data in their stories.

Originally posted on Mad in America on December 13, 2011:

In a recent article in the New Yorker, titled, The Truth Wears Off, science writer Jonah Lehrer discusses an intriguing problem in science. The problem is that scientific results which are confirmed at one point are sometimes overturned after further testing – today’s “facts” are tomorrow’s “fallacies.” The reason for this, as he sees it, are subtle biases at work that taint the scientific method, so that well-done experiments designed by well-meaning scientists are eventually shown to be problematic. To support this line of reasoning he provides an excellent example of an experiment by John Crabbe.  Crabbe’s group attempted to do the exact same experiment in three different labs, each in different parts of the country. One would expect that if all the experiments were well-controlled at all three research labs, each would reach similar results. His group did their absolute best to replicate all the variables in each of the labs as much as possible, and in spite this, the results varied somewhat for each lab. The natural conclusion is that eliminating all bias in experimentation is probably impossible and that replicability is more difficult and complex than commonly considered.

In addition to Crabbe’s study, Lehrer also cites the clinical trials of antipsychotics to support his view. According to Lehrer, by 2007, scientists were scratching their heads in exasperation because several large studies began to show that the drugs were not as efficacious as was presumed in the 1990s, when the drugs were first introduced. In his words: “But the data presented at the Brussels meeting made it clear that something strange was happening: the therapeutic power of the drugs appeared to be steadily waning.” Having followed the atypicals for the last decade, this piqued our interest. We believe that the atypical antipsychotics are NOT an example of the type of subtle research bias that Crabbe is writing about otherwise, but rather an example of scientists getting closer and closer to the true efficacy of the drugs, as various types of overt bias (and even outright fraud) are observed, noted, and integrated into the literature. While Crabbe’s research was designed to explore the nuances of the scientific method, the story of clinical trials of antipsychotics does not belong in the same category. We are concerned that this mis-categorization may be confusing, and explain our different interpretation of the same story, below. We do not mean to single out Jonah Lehrer for criticism- he is undoubtedly one of our best science writers, with the challenging task of understanding and explaining multiple scientific sub-fields to the layperson. However, we do think that it will be useful to understand how the atypical antipsychotic story differs from the subtle biases that complicate other kinds of research.

The Clinical Trial Process

Lehrer writes, “Before the effectiveness of a drug can be confirmed, it must be tested and tested again, different scientists in different labs need to repeat the protocols and publish their results. The test of replicability, as its known, is the foundation of modern research.”  This portrays the clinical trial process in the best light possible; one might call it a scientific ideal that is seldom realized in psychiatric research. Under the current regulatory process, for a company to get their drug approved, they must submit two positive studies to the FDA. This would seem to support the importance of replicability, however, there is a major problem with this.  Namely, that although a company has to submit two studies to the FDA, the company can do as many studies as they want. As Paul Leber of the FDA CNS division has said, “How do we interpret two positive results in the context of several more studies that fail to demonstrate that effect? ….in a sense the sponsor could just do the studies until the cows come home until he gets two of them that are statistically significant by chance alone, walks them out and says he has met the criteria.” To get their two positive studies, they might have to do five.

It is a mistake, therefore, to think of psychiatric research on patented, for-profit drugs as an unbiased scientific endeavor. This completely misses the primary purpose of clinical trials, which are fundamentally designed as the centerpiece of a company marketing program. When the company eventually submits their data to the FDA, then they must submit all their data, even the negative data. But it is up to the company to publish whatever data it wants, and in most cases, the companies have published the positive trials and withheld the negative trial data. In other words, the published literature, especially the early literature, is not a true representation of all the available data on a drug.  And in the case of psychiatric medications, when one examines all of the data that was collected, the drugs look less effective, and more harmful, than originally portrayed. This does not mean the drugs have slowly lost their efficacy- it means that when all the data are examined, or unbiased experiments finally take place, the scientific community is able to examine a less biased database regarding the true efficacy of the drug.

In psychiatric drug research, this process appears to take 10-20 years, as increasingly contradictory data slowly trickles in.

But, Back to Fundamentals

But, perhaps a more important point is the premise that atypical antipsychotics were shown to be highly effective in the 1990s. We understand that the drugs were marketed as highly superior to first-generation antipsychotics- one of us worked in a psychiatric hospital in the late 1990s, and we watched as clinicians told patients that the new antipsychotics were miracle drugs. So, it is surely easy to find research psychiatrists who endorse their use – some with conflicts-of-interest with the makers of atypicals, others with no such conflicts but who are eager to find alternatives to first-generation drugs such as thorazine and haldol. However, it is just as easy to find evidence that these drugs never should have been considered to be a major improvement over these older drugs in the 1990s. For instance:

In 1992, the FDA wrote Johnson and Johnson regarding their atypical antipsychotic, Risperdal: “We would consider any advertisement or promotion labeling for RISPERDAL false, misleading or lacking fair balance under section 502 of the Act if there is a presentation of data that conveys the impression that Risperidone is superior to haloperidol or any other marketed antipsychotic drug product with regard to safety or effectiveness.” It is hard to imagine a more clear statement from the FDA- they would not allow advertisements claiming that Risperdal was superior to the older antipsychotics. (Although they couldn’t run advertisements claiming this fact, they did use the peer-reviewed scientific literature to convince clinicians of this claim).

Ten years later, in 2002, psychiatric historian David Healy wrote the authoritative history of the antipsychotic drugs, “The Creation of Psychopharmacology.” Healy dedicated only a few pages to the atypicals such as Risperdal, writing, “…they were not obviously more effective than haloperidol [haldol], except for their marginal benefits on negative symptoms.”

Professor David Cohen published an article in 2002 analyzing the methodology used in clinical trials of atypical antipsychotics, concluding that biased study designs were likely responsible for many of the purported “benefits” of the newer medications. The list of deliberate confounds is long. The studies he analyzed were almost exclusively funded by the makers of atypical antipsychotics.

Many more examples could be given, but the point should be clear- an examination of the available evidence from the 1990s forward surely calls in question the degree to which the atypicals were a major step forward; this issue is well-covered in the critical literature.

More recently, there has been clear evidence that scientific evidence of adverse effects was deliberately withheld by the makers of atypical antipsychotics. Almost every manufacturer of atypicals has been fined millions of dollars for illegal marketing. As time goes by, a flow of documents generated through legal discovery increases what is known about these drugs. Again, this just means that we uncover more (previously discovered but hidden) data about these drugs, not that their efficacy is now waning.

Conclusion

The issue of replicability in research is an interesting one. However, it would be a mistake to buy into the premise that the efficacy of atypicals has been dropping dramatically since their introduction in the 1990s. Instead, what has been waning is the influence of bias and marketing on the perception of the atypical antipsychotics.

A more important scientific question, perhaps, is: How do we reduce the amount of time (nearly 20 years?) that it takes to reduce the impact of this bias and marketing, and learn the true utility of a new psychiatric medication?

***

Mad in America hosts blogs by a diverse group of writers. These posts are designed to serve as a public forum for a discussion—broadly speaking—of psychiatry and its treatments. The opinions expressed are the writers’ own.

***

Mad in America has made some changes to the commenting process. You no longer need to login or create an account on our site to comment. The only information needed is your name, email and comment text. Comments made with an account prior to this change will remain visible on the site.

18 COMMENTS

  1. I cannot believe I am hearing this:

    “We do not mean to single out Jonah Lehrer for criticism- he is undoubtedly one of our best science writers”

    You’ve got to be kidding me, this disgraced fabulist and liar, is someone the public will be better off without:

    The New York Times Book Review, criticized [Lehrer’s book] Imagine for its “many elementary errors” and “formulaic” approach, as well as for “Lehrer’s failure to grasp some fundamental principles of scientific thinking.”

    “More troubling is Lehrer’s failure to grasp some fundamental principles of scientific thinking. He uncritically accepts studies whose results support his argument, rarely bothering to discuss whether or how often they have been replicated. On the basis of one experiment, for instance, he claims that “being surrounded by blue walls makes us more creative.” Maybe, or maybe not. The researchers actually displayed questions on a computer screen with a blue background, and research on such “color priming” effects is hardly settled science. Perhaps Lehrer thinks the findings must be valid because they appeared in a prestigious journal.”

    “Nature called portions of the [his other] book into question based on current understanding of neuroscience”

    Scientific American has just said today

    “While his actions were largely interpreted as being an affront to journalism, they amounted to much more: a disrespect for and a betrayal of the fundamentally pure enterprise that is science.”

    Lehrer brought his profession into disrepute, jeopardized public trust in the New Yorker’s integrity, and brought shame to himself. I am glad he was caught out.

    “A more important scientific question, perhaps, is: How do we reduce the amount of time (nearly 20 years?) that it takes to reduce the impact of this bias and marketing, and learn the true utility of a new psychiatric medication?”

    I can reduce it down from 20 years to 5 seconds. Apply the following method: psychiatry can’t prove your brain is diseased, so why drug it at all?

    Done.

    Lehrer was a two bit neuromythology salesman and I’m glad he got caught. He certainly wouldn’t have admitted it out of guilt. He was remorseless until caught.

    One quote to finish, from Jonah Lehrer himself:

    “If someone is thinking about a math problem . . . then her prefrontal neurons are thinking about that problem.”

    I supposed he asked her prefrontal neurons what they were thinking?!

    But continue, Jonathan Leo, to believe he was “one of the best” science writers as you have said here in your article.

    Report comment

    • “Apply the following method: psychiatry can’t prove your brain is diseased, so why drug it at all?”

      Some people find that an intoxicated state brought on by drugs to be preferable? I agree that the brain isn’t “diseased” and there is no “imbalance” If someone is truly given “informed consent” the pro’s and con’s of any drug (let’s assume legal drug at this point), do they not have a right to choose it? And isn’t it likely that a percentage will find their drug-induced state preferable to non-drug?

      Report comment

  2. Good discussion here, but hey,”Anonymous,” I am wondering who you are. I suppose there are many “anonymouses” (ummm, is that a word?). I very much like your comments.

    Especially your response to the original authors’ question, which basically asks, but in more euphemistic words, how can we shorten the time it takes to find out the fraud committed by drug companies?

    I think there is another important answer as well. And that is there needs to be a powerful movement of people who have been damaged by this fraud, their lives ruined by the “miracle drugs.”

    I have been more and more disappointed by this website, with its emphasis on Harvard “experts,” and its apparent belief that by intellectualizing out there in Cloudcuckooland, somehow the world’s problems. or at least the problem of abusive psychiatry, will be solved.

    (Not being connected with Harvard myself, I am able to notice that the more pretensions to expertise the writers express, the more intellectually third-rate their writing is.)

    No movement of oppressed people has ever been successful until those people themselves have taken control of their own destiny.

    I for one am very troubled by the belief of many of the people writing on MIA that they are in charge of helping the poor helpless mental cases. Being one of the poor helpless mental cases myself, I want to tell you that I am greatly unimpressed by the five-thousand-word articles written by these “experts” that have virtually no intellectual content. These people, our condescending saviors, have no respect for us and aren’t allies at all. They are just indulging their self-important fantasies at our expense.

    I think our movement, the psychiatric survivors movement, needs something better than this.

    Report comment

    • Hi Ted. I’m sorry, this is a really hard thing, but I have chosen at this point to remain anonymous on here. I’m flattered you liked my comment, and if you are THE Ted Chabasinski, you’re a hero of our movement, and I have read a lot about your great work.

      This site, is a big tent, and because it is so slick, and large, and one can easily see the potential of it, (bearing in mind it has been going less than a year), I too fell into for a time when I first discovered it, critiquing various elements of it harshly*. I have since grown to love it, it is a vibrant community and it was started in good faith by Bob Whitaker, someone who I am sure we both respect a great deal.

      I read your comment above as being a broad comment? and not specifically about the author/s of this article right here? I think Jonathan Leo has shown some courage in his work, he’s the guy who raised the ire of an elite medical journal http://blogs.wsj.com/health/2009/03/13/jama-editor-calls-critic-a-nobody-and-a-nothing/

      I had disagreed with Leo’s praise for the fabulist Jonah Lehrer’s science writing, that’s all.

      Anyway… if your comment was broad, and about some/many things you have read on this site, I can only offer that what the psychiatric survivor opinion of the “stock” of bloggers that are on this site is, needs to be tempered by the fact that while many of these writers might not be 100% behind our positions (or even our rights!), they tend to be closer to the mark than in many other publications, online or offline.

      *There is one psychiatrist blogger here, he’s not even a blogger here, his articles are syndicated here, republished here after originally appearing on his own site. His name is David Healy. If the topic of discussion is your critique/a critique of the state of this website and your disappointments with it, I must add, his work being syndicated here is my only great disappointment with this website. But I don’t call the shots.

      I will say I’m touched that you liked my comments, and I’m glad you’re here, but I do agree if what you’re getting at is, you can see this large community of critical of psychiatry people, and you can imagine the potential for us to build something bigger and better one day for our movement. But I respect this website, and I respect the good faith efforts of all the bloggers (Healy excepted), and it is still early days so please have faith.

      Soon I am told, there will be an internet forum available within this site for various discussions, general discussions, perhaps like the one we are having here.

      You have done great work in your life for the movement, and I thank you for that. I have also enjoyed the articles you wrote for this site and encourage you to write more.

      Report comment

      • Wow, thank you for your kind words. And yes, my comments were not directed at the authors of that particular article. I have seen their work before, and it is almost always right on target. I hope if they read it, they didn’t take it as directed at them, because it wasn’t. But I do feel frustrated by the site. I think there is too much emphasis on mental “health” professionals, and that most of them rarely have anything worthwhile to say, and take too long to say it as well. In my opinion, if you really have a good point to make, you should be able to be clear and concise.

        As far as a discussion group, I think that would be great, and I hope the site adds it soon. I would certainly participate.

        And finally, yes, I have great respect for Bob personally and his work as well, but I still feel frustrated by MIA.

        Report comment

  3. Hello to both “Anonymous” and Ted Chabasinski. A quick comment. Jonathan Leo and I co-author many of the blogs together- you have to look at the byline closely because sometimes they are written by just one of us, sometimes that are both of us. In the case of the original Lehrer post, it was co-authored, and I admit to writing the line on Lehrer. At the time, I had been impressed by how engaging his writing was, but critical thinking is a completely separate skillset, and it’s apparent now what was going on. Perhaps the line on Lehrer was too deferential, and I don’t mind if you give me some heat for writing it, but I also think we saw something slightly ahead of the crowd, in that he didn’t seem to even bother to investigate the topic he was writing about.

    Thanks for your comments on our blog.
    -Jeff

    Report comment

  4. When I read Lehrer’s piece in the New Yorker, my first thought was that the earlier studies were pile-on-the-bandwagon junk and the later studies were more objective, and Lehrer had completely missed this potential interpretation, which showed a certain naivete about the drug biz. Or maybe it shows how cynical I am.

    Report comment

LEAVE A REPLY