Should Physicians Read Journals? Given Current Standards, Maybe Not

Randy Paterson, PhD, RPsych
27
76

The image is so familiar it is a stereotype: The physician’s desk, piled high with copies of medical journals, where she or he reads the latest research updates between patients. Medical science, it is said, progresses so quickly that if practitioners do not keep up their knowledge base will be obsolete within five years.

But is the reading of journals useful? Can it potentially inculcate misinformation as much as progress? Is the knowledge gained worthwhile?

Much has been written about the flaws of individual studies. In this blog I recently focused on an example: The infamous Study 329 on the use of paroxetine for adolescent depression. Reading studies such as these, with their swapped outcomes, hidden side effects, and often shockingly biased interpretations of data, may produce only misplaced beliefs in readers, and may actually result in less competent practice. If the physician reads only the summary abstract – as many do, being strapped for time – there may be little relationship between the reported outcomes and what the data actually say.

Imagine that such problems did not exist, however. Imagine that we live in a never-never land of balanced and dispassionate reporting of study results, that published work is competently done, that all outcomes are clear. This is hard to do, because it is so far from current reality. Could there still be a problem?

Well, yes. Several. But let’s just focus on one: Publication bias.

Most drugs and medical procedures are evaluated in multiple trials. Indeed, a single trial is seldom (-to-never) sufficient for a drug to be approved for use. Some trials are reported in the literature; others are not.

Imagine that you are a particularly diligent physician who reads all of the journals relevant to your field. Drug Z appears in four studies, and in all four it outperforms the inert placebo administered to the comparison group. Sounds good. Good enough, perhaps, to influence your practice. Here is a reliable treatment that generally seems to work very well.

The problem is that there are 10 trials of Drug Z, not 4. You can’t read the remaining six, because they have never been published. This isn’t a problem if publishing is something of a random process. Put 10 trials in a bag, then pull out four of them and print them: you will probably have something vaguely resembling a representative sample.

It has been amply demonstrated that this is not how publishing works, however. Submit two studies: one showing that Drug Z works, and one showing no difference from placebo. The study finding a difference between groups is much more likely to see print. Journals like publishing promising findings, not failures.

This understates the problem, however, because the entities carrying out the research have a vested interest (a clear, documented, and obvious conflict of interest) in seeing supportive studies published and negative trials suppressed. So the more accurate picture is that we give our bag of 10 studies to a person (or, say, a pharmaceutical company) who wants Drug Z to look good, turn our backs while they read them all over and hide the ones they don’t like under a mat, then pick four from the rest to submit to journals.

Now switch roles and become a practicing physician trying to do your best for your patients. You read the published literature and attempt to get an impression of the overall trend in results for Drug Z. You never hear the results of the negative trials and don’t even know the studies exist. As far as you are concerned, four trials have been conducted on Drug Z and the results are unanimous. Needless to say, you start including it in your prescribing habits, happy that you have been keeping up with recent developments.

Psychology is, to a great extent, the science of the disconnect between external reality and internal representations of that reality. Even with a perfectly balanced presentation of data, biases and distortions are bound to develop in any human mind. The last three patients to whom you gave Vitamin C recovered from their colds immediately, and as a result you have developed a gut conviction of its efficacy.

Added to the problem of fallible human information processing however, is a system of medical reporting that introduces extreme distortions in research before practitioners ever set eyes on it. In such a situation, erroneous convictions about treatment efficacy are inevitable. It is difficult to see how a system such as this could lead to reliable enhancements in practice – but easy to see how a reader could be misled by organizations that deliberately strive to slant the impression the reader gets.

So should physicians be reading journal accounts of pharmaceutical trials? It is difficult to see the worth in such an exercise – at least, as the journals operate currently. It is even possible that doing so will lead to decrements rather than improvements in clinical practice.

Is there a solution? Yes, an obvious one. Organizations carrying out clinical trials should have to register their study with journals prior to collecting the data, with a clear commitment to publish the results regardless of outcome. This wouldn’t solve the problems of distortion in the write-up or the downplaying of side effects, but it would help significantly with the publication bias problem.

If this is likely to help, why haven’t the journals already pledged to do this? Well, um, they did. Years ago. And in 2007 the practice was put into law in the United States. And then…very little changed. Journals have gone on publishing trials that were never registered, and null trials have continued to go missing.

A Hopeful Development

For many years this problem has been noted amongst scientists and practitioners in medicine and other health disciplines (clinical psychology is no less prone to this type of concern). More recently, it has gone beyond the health science nerd community and has begun to seep into the public consciousness. Medicine and other health disciplines have begun to feel the pinch of appropriate skepticism and disrepute.

As is often the case, shame motivates where ethics fail. With public exposure we may see improvements in the pipeline from laboratory to clinical practice.

One of the chief proponents of change has been Ben Goldacre, British physician, science writer (the excellent Bad Pharma, among others), and medical gadfly. He has been a chief proponent of alltrials.net, a website devoted to changing publication practice in the medical field. Here is a sample of his style, in a TED talk:

And the word is gradually seeping out. This week there are articles in The Economist (click to see this lovely piece of reporting) and elsewhere on the problem. As exposure continues, we can expect that the worm of shame may begin to do its work.

Is this important?

All of this returns us to a familiar two questions:

Are the lives of people with health concerns important?
Does the field of healthcare have any pretensions to being based on evidence?

If the answer to these two questions is “No,” then poor research, poor reporting, and poor practice are no great problem – though it would probably be more honest if we stopped pretending that healthcare believed in science or in the improvement of human welfare and treated it instead as a simple revenue-generating business. That’s not why I got into it (in the allied and every bit as fallible field of psychology), but without a firm adherence to the principles of science it is difficult to argue that it is anything else to skeptics.

If the answer to either (or both) of these questions is “Yes,” however, then the current state of affairs is clearly unsatisfactory and needs considerable change – not just a pledge designed to calm the waters, but an actual commitment to responsible research and the balanced reporting of evidence.

27 COMMENTS

  1. There is also a self-publication bias in research, Journals like publishing promising findings, not failures and so do authors! The only time it is “acceptable” to publish when the null hypothesis is true or there is no effect is when you are a year 7 Ph.D. student and your funding is running out so you need to graduate.

    I am not a medical Doctor and I often wondered about drug trial research. in my field if I read a study that doesn’t pass the sniff test I should, in theory, be able to find enough information to re-run the experiment or study myself. (This is a form of peer review). How can you do the same with a drug trial? You would need the approval of the FDA and the cooperation of the drug company which you would never get.

    Publication bias is a big issue for me since most of my research is based on Meta-analysis, that is my “data” is the results from several published studies, combined and analyzed to form new information and knowledge. If only half the story is being published then I am making it 10x worse by over emphasizing one side of the issue.

    I like the idea of scientists pledging to publish ALL of their research, no matter what the outcome. Open Access On-Line journals can also help. Without a page limit they SHOULD be more accepting of research where the null hypothesis is true.

    Doctor X

    • “I like the idea of scientists pledging to publish ALL of their research, no matter what the outcome. ”

      Pledging is not enough. Pre-registering the trials should be mandatory and trials which were not registered as such should not be allowed to be published or even conducted. All the data save for the personal information which could identify the patients should be accessible to anyone for free in public repositories. Ideally the trials should not be conducted by pharma but by independent bodies with no monetary incentives tied to the outcome. At least this is the nice dream I have…

  2. I would add that despite the huge positive publication bias, when long-term studies or reviews do support the null hypothesis, or even worse, show that the treatment is destructive over the long term, they are generally explained away or dismissed. Examples include Wunderlink, Harrow, the WHO studies, the long-term arm of the MTA, the Quebec ADHD study, the Raine study, and many more. Knowing that we’re already dealing with a bias toward believing Psychiatry’s story, it should be considered all the more important and relevant when data comes out that breaks through that barrier and says we have something more to think about. And yet folks like William Pelham, who honestly assessed the long-term MTA outcomes, are condemned as biased or ill-informed and business continues as usual.

    This really does make me question whether any kind of research or factual data will ever bring about major reform in psychiatry. I’m afraid the field is simply based on false premises, and looking at the data will ultimately undermine its very foundations. And yet millions of people are making billions of dollars off this chicanery. So rather than reforming, I expect psychiatry as a profession to dig in and defend itself to the last inch of ground, yielding nothing to actual research. To allow the actual facts to surface will bring about the complete downfall of the current paradigm, and those with their snouts in the financial trough are not about to let that happen without a monumental battle!

    —- Steve

    • You are probably correct that no amount of research and factual data will bring about change in the practices of big Pharma and most psychiatrists… change, that is, away from the disease model, away from the use of invalid diagnoses, away from overprescription of medication, etc.

      There is a good TV show on CNBC called American Greed (perhaps that should be the title of Whitaker’s next book). I enjoy this show… the con artists are so ingenious at skewing information and opinions to fit their Macchiavellian needs for power and profit. And they usually convince themselves that what they are doing is justifiable and that are not criminals. They only stop their abuses when they are sent to prison, i.e. when there are consequences and people rise up against them.

      I think the same thing has to happen with drug companies and corrupt psychiatrists if any major change is to happen. A much larger proportion of the American public would have to decide to educate themselves, stop believing without evidence that these drugs are effective treatments for “mental illnesses”, stop believing in mental illnesses without biomarkers, etc.

      Currently, most emotionally distressed Americans are sheep being led to the slaughter and there is no reason from Big Pharma’s perspective not to continually profit off of them.

  3. I was surprised and informed to read this: Big pharma gets a lot of the heat that should be placed on “Big journal”: http://www.washingtonpost.com/news/morning-mix/wp/2015/03/27/fabricated-peer-reviews-prompt-scientific-journal-to-retract-43-papers-systematic-scheme-may-affect-other-journals/
    and even more so about journals’ profits: http://svpow.com/2012/01/13/the-obscene-profits-of-commercial-scholarly-publishers/ – as i commented in the blog post where these came up: – that is important and VERY relevant information, and I had absolutely NO idea!!! I doubt many others do, but it puts the purposes of the NEJM and CMA editorials into clear perspective for me. That gravy train needs to keep rolling along, and the experimercials are the coal feeding the furnace. (to mangle metaphors!)

    here’s the blog post: http://1boringoldman.com/index.php/2015/07/20/a-snails-pace/#comments

  4. I read your piece last night, and slept on the issue (odd as it may sound, ‘sleep consciousness’ is where I often work through challenging issues, supposedly within the ‘collective unconscious,’ historically with much success in real life, however – apparently a shaman type gift, according to a book I’m reading right now). Nonetheless, I awoke a bit alarmed, concluding I must agree with you, but also with an immediate solution.

    The doctors do need to stop reading the corporate produce “biased” medical journals NOW. What’s being called “evidence based medicine” today is much more of a misinformation system, than anything else. Marcia Angell was warning of this reality ten years ago, too. It’s time for the people within the medical industry to wake up. Because, as we all know, this corporate misinformation fest has resulted in deaths from prescription drugs being the third largest killer of Americans. Intentionally or unintentionally, the medical industry is, in effect, being used as a ‘weapon of mass destruction’ against the U.S. population right now, by the corporatocracy – the “evil bankers and corporations” Thomas Jefferson forwarned us of years ago.

    I don’t believe most doctors want to be killing so many patients, due to misinformation, however. But this needs to end NOW. And there is an immediate solution already available. But the medical industry needs swallow one of the reality ‘pills’ (it seems the online bloggers are calling this “the red pill”), and WAKE UP.

    The solution to this disconserting issue of massive corporate / FDA medical misinformation is the truth is already out there. But the medical industry needs to go back to the understanding medicine is an art, not a science. Patient opinions need to be listened to, and taken seriously. And just like us patients who “know more than the doctors,” the Internet is where the medical industry needs to search for the truth about the drugs, not in the medical journals.

    The paternalism, and forced druggings, need to end. But the truth is already out there. Thanks for addressing this important public health problem, us humans (including the doctors) all need to work together. By law, corporations MUST legally behave as psychopaths, thus should NEVER be trusted.

    • Someone Else,
      I agree with virtually everything you say, except that instead of saying “But the medical industry needs to …” or “…this needs to end NOW,” I say “Let’s make them stop.” We, as medical consumers, have the power if we make it our business to educate ourselves and act on it. This is already happening, as shown by the increasing interest in alternative practitioners. Allopathic (pill-pushing, industry-sponsored) medicine will not stop of its own volition (why would they?); only we can make them stop. I do not see mainstream psychiatry as totally unique, just the very worst of an increasingly bad lot. There is harm and corruption being done in general medicine, pretty much for the same reasons as in psychiatry: key opinion leaders prostitute themselves for Big Pharma, and the majority of doctors in the trenches are clueless and do not know any better. Is it a big shock that 8 out of 9 members of a panel responsible for formulating cholesterol standards had consulting gigs for statin makers? What makes psychiatry worse, of course, is the coercion and the complete and total absence of any objective metrix supporting the drugging.

      But instead of saying “this must stop,” let’s make them stop. Educate ourselves and others, including those doctors who are willing to be educated. Who else can we rely on? Not the industry and not the government. By allowing drugs to be advertised directly to the consumer, out elected representatives virtually guaranteed that no mainstream news media will expose the harm (who will say “no” to all that advertising revenue?). Good people are doing important work and making healing discoveries as we speak. As medical consumers, instead of blind trust in the doctors and their prescription pads, we need to become medical detectives and sleuths.

      • GetItRight,

        Thank you for the compliment, and constructive criticism. I do agree with you, and unfortunately had to give up trust in allopathic doctors ten years ago. I educated myself. Now I am trying to reestablish a mutually respectful relationship with mainstream medicine. I taught the head of family medicine at the “one of America’s top 5 hospitals,” my new doctor, the truth about the iatrogenic bipolar epidemic in this country, step by step, by medically explaining it to him. He tried to correct me.

        But the Internet is a better, and more accurate source of medical information today, than the medical literature – this was actually my point, Randy. My doctor tried to tell me Wellbutrin couldn’t cause increased libido. This was right after GSK lost the lawsuit for marketing Wellbutrin as the “happy, horney, skinny drug.” To this day, the medical literature does not show increased libido as even a possible side effect of Wellbutrin. At my last appointment, this doctor gave me a teaching physical, so I could help enlighten his students to the reality that some patients know more than the doctors because THE INTERNET IS A WONDERFUL SOURCE OF MEDICAL INFORMATION TODAY.

        I’ve also been sending my research into iatrogenic bipolar to the ELCA Lutheran synod headquarters since 2006, as well as to my former doctors – in an attempt to educate them as to their industries’ almost unfathomable in scope crimes against patients. And to point out the impropriety of the religious hospitals profiting off of psychiatry’s shananagins, especially against innocent children. But covering up child abuse for the paternalistic religions has historically been, and still is, one of the primary functions of the psychiatric / psychological industries.

        I’ve also given numerous pastors of various religions, Whitaker’s book, and am trying to educated teachers and child care leaders about the issues. I even had lunch with the head of the organization that drugs the majority of public school children in the area in which I currently live, gave him Whitaker’s book. And have also given him the medical evidence that the neuroleptics do, in fact, cause both the negative and positive symptoms of “schizophrenia.”

        I am trying to educate people, but it’s embarrassing for “the two original educated professions” to have their paternalistic “dirty little secret” exposed. The medical evidence is clear, however. The etiology of most “schizophrenia” today is likely psychiatrists profiting off of covering up child abuse and easily recognized iatrogenesis. Psychiatry / psychology quietly replaced the duties of the witch hunters of old, and has been intentionally harming people for profit for ages. It’s shameful. Although, I know some practitioners want to actually help people, and we’re duped and deluded.

        • Someone Else,

          thank you for your comments. Just one bit of clarification: I did not intend my comment as a criticism (constructive or not) of anything you had previously said. I have read your various comments decrying corruption at the various levels of society (shrinks, pastors, bankers) and I agree with you. I was just focusing on the fact that abusers or exploiters will not stop just because it is the right thing to do; it is up to the rest of us to make them stop. I see the momentum toward alternative therapies as a very promising trend. When people realize that pill pushing medicine is not to be trusted in general, they will, hopefully, come to the same conclusion when it comes to psychiatry. Psychiatry is a more difficult nut to crack for all sorts of reasons, including the fact that psychiatry is not there to help the suffering patient; it is there to make the rest of society feel safe and comfortable. We just have to keep trying, changing minds one at a time!

  5. Part of the problem has always been doctors actually not reading the research. Many, if not most doctors, will read the abstracts and conclusions of research and skim the rest. Most research on various treatments in psychiatry have always shown the same thing. The relatively little difference between placebos and treatments would seem to indicate that many treatments appear to work, but that they all work poorly. One always has to consider in reading research the biases introduced, the nature of the research population, the validity of the diagnoses, and the clinical significance of the findings. For example, in many psychiatric medication studies a 50% reduction on a particular scale is used to determine a positive outcome. In many of these studies, one gets an average drop from 30 on a scale to 14 for the drug and 16 on the placebo with a statistical significance between the two groups. But one has to ask whether this minor difference is clinically significance, especially if one takes into account sedation by the drug and all the possible biases.
    Research in psychiatry has always shown most treatments to be clinical failures. The only conclusions one can reach, if one actually reads the studies, are that psychiatric conditions are heterogeneous, that DSM diagnoses are not diseases, and that simplistic uni-modal treatments don’t work. This means that we always have to treat people as individuals, trying to help them with their problems and issues, and possibly consider some researched modalities have having some limited value in some people. When it comes to medication, the published research has clearly shown that we have no idea what these medications actually do, and that there are possible negative long-term consequences, but little long-term benefit.
    It is crucial in reading research to understand that in order for research to be considered valid, the results from all research done from various viewpoints, clinical experience, and patient experience should all point in the same direction, with little controversy. If the whole picture doesn’t make sense together, then there is something wrong. This is clearly the case for research on medication. It is also the case for research on psychotherapies, where research on the therapies themselves often focus on showing that a particular technique works, while research on what actually works in therapy indicates that it is not any particular technique that shows value.
    One just has to actually read, in detail, the research in psychiatry to understand how far off the rails psychiatry has come.

  6. Just a few brief comments about your other fine article regarding “Study 329” on the use of paroxetine for adolescent depression, which you linked to above.

    “Like most medication trials, the duration was limited – in this case, the treatment phase was just 8 weeks.” That’s an absurdly short amount of time to study any drug, especially one to treat an often lengthy, reoccurring, or complex problem like depression. This ridiculous time frame should have invalidated the study from the very get-go.

    Furthermore, GlaxoSmithKline stated in court, “This ‘cutting edge,’ landmark study is the first to compare efficacy of an SSRI and a TCA with placebo in the treatment of major depression in adolescents. Paxil demonstrates REMARKABLE Efficacy and Safety in the treatment of adolescent depression.”

    In my experience, Paxil “works” by making one feel indifferent, apathetic, numb, caring less about everything, disinhibited, etc. These results, along with subtle intoxicating effects of “spellbinding” Dr. Breggin writes about, may seem to be a distraction from, or an improvement of, of one’s depression, but it is hardly a long-term solution.

  7. Hi Randy,

    Thanks for the article, but I find it hard to share your optimism that publication bias is the main problem and that the reliability of the literature can be fixed by simple technical fixes. From R.W. & L.C.’s latest book and Robert’s earlier work and Peter Gotzsche’s work and David Healy’s writing, my impression of the psychiatric literature is not that it is flawed by unintentional bias. My impression is that it is tainted by gross, widespread institutional corruption. If supposed papers by academics are actually ghostwritten by drug companies on a large scale, that is an abandonment of the most basic scientific standards. I guess I am skeptical that you can get trustworthy results from people who have already shown their willingness to sign ghostwritten papers, or to fudge their data, or bury inconvenient results, no matter what technical measures you put in place. I find it hard to believe that technical measures can replace basic honesty and integrity. – Saul

  8. I have read a lot of reports on these issues about psychiatric medications from the perspective of a “consumer.” I have always been skeptical of them for countless reasons. And forced medication… don’t get me started.

    But here I am writing from the perspective of a research scientist, engineer, and entrepreneur. I am working on developing tunable LED lighting for buildings like nursing homes, prisons, schools, and psychiatric hospitals (locations all too similar, no?). These lights provide additional blue light in the morning and reduce the blue in the evening. Specific wavelengths of blue (around 480nm) are linked to intrinsically photosensitive retinal ganglion cells (ipRGCs) which regulate your circadian rhythms. In simplest terms – these receptors use blue light to tell your body what time of day it is, giving you cortisol in the morning and melatonin in the evening (among other effects). I am particularly interested in treating “conditions” like bipolar and schizophrenia but the technology provides benefits anyone trapped indoors for 24/7 (emphasis on the word trapped is why I include schools).

    There is published evidence in peer reviewed journals detailing the relationship between sleep and bipolar disorder and schizophrenia. And also articles linking light spectrum to sleep problems. One could argue there is a no-brainer in bringing these products to this populations. However, the road to bringing a technology to market in the healthcare industry requires, like pharmaceuticals, a long road of clinical trials and other expensive and timely research.

    These products are available now, though nobody really knows about them. I have looked into doing some studies and publishing the data which takes time and money… meanwhile more and more time goes by and people continue to suffer in buildings with health deprecating lighting while a solution exists waiting to be implemented. How would you recommend bringing these products to market? I am sincere in my hopes of helping people. I have distaste and distrust for pharmaceuticals and don’t want to come off as if I am one of them – I’m trying to provide a more natural solution. Yet – at the end of the day the universe will perceive me as an entrepreneur trying to sell a product. So how can I earn the trust of the community without having to take 7 years to do so.

    • Flimjannery;

      if you look at sites such as LowBlueLights.com, which sells glasses that block blue light and promote sleep if the glasses are worn during the evening hours, you will see that the concept and its benefits are becoming well-known. If your potential customers are facilities, rather than individual consumers, I think that may complicate the situation.
      Are these facilities that promote drugging? If that is the case, I do not see how your product would make that much difference in helping people. Plus, given the importance of Vitamin D in mental health, being indoors 24/7 is not optimal, just the opposite. But I believe what you are trying to do has promise and potential and wonder why you are not seeking to make your product available to consumers, to be used in homes? I believe that LowBlueLights.com already sells LED bulbs that block blue lights (to be used in the evening). I also hope that your LED lighting does not rely on transformers, to obviate the dirty electricity/EMF problems.

      • I’m particularly interested in helping populations that are trapped indoors against their will 24/7. I am not advocating that people be indoors 24/7, I want to help those that are stuck theres. Product for home-use are becoming available, but good ones are expensive. You can get bright blue lights, but ideally you have something that is tunable and shifts with the day. You don’t want to have to use separate lamps at different times of day. For an example of a good home-use product thats on the way, you can google Ario Living is coming out with a floor lamp. As for the drugging part, I am hoping that the lights will improve health and reduce the amount of drugs used on patients.

  9. “The problem is that there are 10 trials of Drug Z, not 4. You can’t read the remaining six, because they have never been published” This is only partly true.

    Jones et. al (http://www.bmj.com/content/347/bmj.f6104) found that 68% of large (generally Phase 3) trials are published within 5 years of completion. (5 years is admittedly a long time, but remember that at least 13 months, and commonly up to 20 months, passes between completion of the trial and regulatory approval). While the discussion section in the article attributes non-publication of the remaining 32% to “industry hiding negative results”, an examination of the paper’s supplementary material reveals that some 90% of the unpublished trials are of drugs that were never approved.