Saving Science: It’s Time to Solve Publication Bias

Jeffrey Lacasse, PhD
16
138

(Cross-Posted with the Foundation for Excellence in Mental Health Care).

Erick Turner has published an important piece in CNS Drugs entitled “Publication Bias, with a Focus on Psychiatry: Causes and Solutions.” It should be required reading for any medical student, non-medical helping professional, or practicing prescriber. Rather than being an unbiased reflection of the underlying data, the psychiatric literature is instead sort of a funhouse mirror – the published results are consistently, overwhelmingly, positive – no matter what the actual data are. The implications are enormous. Note that the methodology in such trials is often deliberately constructed to bias such trials in favor of the sponsor’s drug – even so, publication strategies are used to minimize any bad news resulting from such trials, and to maximize the impression that the drug is effective.

Professionals that read these studies in the peer-reviewed literature (or systematic reviews based on these studies) will likely believe the results, and adopt biased impressions of psychiatric medications – they will believe them to be more effective, and less risky, than the actual data show them to be. When it comes time to inform their patients about treatment options, they will do their best to accurately inform their patients … but physicians exposed to biased literature are likely to then misinform their patients. The misinformation will almost inevitably be in the direction of lower thresholds for treatment and higher efficacy/lower risk for psychiatric medications.

Turner is correct in writing that these problems are in science across the board, but the potential for real-world harm is much higher in medicine in general and psychiatry in particular. There also seem to be different levels of awareness of these problems across scientific fields. I have had many conversations with social scientists who express awareness of the problems with publication bias, and are concerned – they often see the issues as a problem that must be solved over time. However, anecdotally I also hear of these same social scientists going to their prescriber and expressing concern about the evidence base in medicine and/or psychiatry – having read articles like Turner’s, and having some understanding of the challenges of making clinical decisions based on experimental evidence. Apparently, in many cases these conversations do not go well.

Having been trained in evidence-based medicine and in the primacy of randomized controlled trial data, many prescribers are apparently made very uncomfortable by the questioning of the paradigm. And one can easily imagine a clinician reading Turner’s article and then feeling completely overwhelmed: If Turner’s analysis is valid (I believe it is), the ramifications are so enormous that they may be difficult for the average clinician to believe. Human beings are justificationary creatures and physicians are no different. One strategy for addressing the problem would be to integrate this material into medical schools as soon as possible so that emerging physicians are well-trained on these issues. Of course, this is easier said than done, and could create significant angst among medical students as they try to reconcile a curriculum focused on clinical trial evidence with the unfortunate reality of selective and biased reporting.

So, what to do, moving forward? Turner focuses on problem-solving at the level of regulatory authorities, authors, and journals. His suggestions are all thoughtful, and most are realistic. Here I would like to close by repeating some more radical proposals that I believe would help solve the problem of publication bias (and associated problems) in psychiatric research. If academic medicine is actually interested in solving the deep scientific problems in the psychiatric literature, it could:

1. Admit that medical journals “have devolved into information laundering operations for the pharmaceutical industry” (Richard Horton, Lancet editor) and adopt a new mission for medical journals in which they only publish scientific criticism of trials (Richard Smith, BMJ editor).

2. Ban ghostwriting of trials and enforce the same authorship norms on academic psychiatrists as researchers in psychology or economics.

3. Encourage dissent and transparency in journal publications. When publishing a clinical trial in an academic journal, provide patient-level data to a researcher with a dissenting point of view and allow them to publish a brief summary following the article. Deliberately subject the underlying data to a rigorous, public critique. Put any statistical tricks or changes in endpoints (which Turner explains well) on display and allow the reader to make their own decisions regarding the validity of the trial results.

4. Support efforts for data-sharing of unpublished patient-level data. Realize that sociologists and philosophers of science (and increasingly, the general public) will always be skeptical of any claims made where the underlying data are not available for re-analysis. Change scientific norms so that data unavailable for external re-analysis is simply not taken seriously.

16 COMMENTS

  1. Thank You Dr. Lacasse;

    However, I believe you’re still being too charitable with the prescribers here.

    They know full damn well they’re being bullshitted, because Every prescriber in the State of California knows that J&J’s Risperdal comes with State Police Registration.

    And there’s only one explanation for that. What they are doing is illegal and they know it. If they really Are that Stupid they have no business being prescribers.

    http://psychroaches.blogspot.com/search?q=18C95

    Prescribers playing That get out of jail free card has worn way beyond thin.

    • Agreed. These prescribers know all too well about these problems. They just don’t care. The abuses are institutionalized. They are not going to criticize them because they would be criticizing their own livelihoods. People need to understand that Big Pharma is a half a trillion dollar industry worldwide (80 billion of which are made out of psychiatric drugs alone).

      The only way to stop this is via medical malpractice and criminal lawsuits. That is, in my opinion, where the focus for change should be: in the light of this data, we should engage pro bono attorneys to sue the worst offenders. The DOJ investigations had a great effect on Big Pharma, to the point that they seem to be abandoning the field of psychiatric drugs entirely. Not because they are uncomfortable with paying the fines, but because the executives at companies like GSK or Elli Lilly have probably reached the right conclusion that under the increased scrutiny to their corrupted practices is only a matter of time before one of them ends up in jail. It is one thing to use shareholder money to pay for fines that still make the drug profitable, quite another to go to jail… That is how Big Tobacco was defeated and that is how the psychiatric scam will too be defeated.

      • Cannotsay2013:

        I agree totally but this isn’t a solution that a lot of people in our movement seem to be comfortable with. However, like every other human and civil rights movement true freedom will only be won in the courts of this country. The drug companies and the psychiatrists will not turn us loose because it’s the right thing to do. They will only turn us loose when brought to question in the courts by the law.

        I believe that you are correct that the only way all of this will be stopped is when CEO’s of these companies face time in prison. Then and only then will they begin setting us free. They’re the ones who heavily influence the psychiatrists and doctors of other specialties to push their toxic drugs, often with great incentives.

    • Jeffrey,

      I regretted not acknowledging this great post when you wrote it because I believe it is critical to keep hammering away at the corrupt pseudoscience that BIG PHARMA has bought in medicine and psychiatry in particular.

      I just wrote a post about your great review of MAD SCIENCE while questioning your seeming neutrality about the topic in that review. This reminded me of this post and I came back to reread it having been very impressed at the time.

      After rereading it, I think you are very brave and noble to keep pounding away at this junk science passed off as valid in a very professional, scientific way, which may and can change this deadly paradigm eventually or at least warn would be victims to do their own homework.

      Back to my question on your other post, does this type of seeming “anti-psychiatry” put your job at risk? How do your colleagues react to these critiques by you?

      Again, I am very glad you are posting here as an expert in many topics. What else do you teach besides the DSM?

      I will understand if you can’t respond to all of this, but I am very curious while grateful for your expertise.

  2. It is a mistake not to provide all the information found during research, but in any research relating to medicine, it should not be allow publication bias because any adverse or negative result has a direct impact on people.

    In many cases, the publishers are reluctant to publish negative results, in many others companies themselves are behind this bias due to marketing strategies, but at the end I think that researchers have the key to finish with publication bias and make complete information to get everyone. We are, from below, who must show everybody the importance of publishing all results, the good ones and secondary results (akas. negative results).

    Changes of mentalities are not achieved suddenly, they need a progressive adaptation and on this aspect, things are moving forward. Proof of this is the emergence of magazines like The All Results Journals (http://www.arjournals.com), the first Total Open Access Journals dedicated to negative results.

    Hopefully soon, everybody will all understand the importance of the publication of any results duly justified, and avoid wasting so much time and money on research.

  3. I’m not objecting a big bunch here, but I will say it’s a myopic view of research publication at this point in its general de-evolution. Perhaps “optimistic” is a better word?I suggest you widen the blinders and reach out because there is great dammage being done in other fields. Yes you “pay lip service” but the focus sounds rather ill tempered and I doubt much will come from acting as if its worse here.

    If my newness to the topic is off base, I request a correction at your convenience.