Critical Psychiatry Textbook, Chapter 5: Psychiatric Diagnoses Are Not Reliable (Part Two)

5
2492

Editor’s Note: Over the next several months, Mad in America is publishing a serialized version of Peter Gøtzsche’s book, Critical Psychiatry Textbook. In this blog, he discusses the poor reliability and validity of psychiatric diagnoses and the concept of overdiagnosis. Each Monday, a new section of the book is published, and all chapters are archived here.

Two concepts are essential when discussing diagnostic tests, their validity and reliability.

The validity of a diagnostic test refers to its ability to measure what it is purported to measure, which involves its ability to distinguish between people with and without a particular disease.

The two principal measures of test validity are sensitivity and specificity, which are the proportion of those who are ill that test positive, and the proportion of those who are healthy that test negative, respectively. Most people believe that the predictability of positive and negative diagnostic tests are constants, which they are not, as they depend on the prevalence of the disease that is being tested for.105 The more uncommon a disease is, the more false positives will there be. This is why screening for depression is a bad idea. The screening test for depression recommended by the WHO is so poor that for every 100 healthy people screened, 36 will get a false diagnosis of depression.7:46,106,107

A bearded white man repeated pattern with hands in an expression of confusionWhen I criticise my colleagues for using such poor tests, I am told that they are only a guideline in the diagnostic work-up and that additional testing will be performed. In an ideal world perhaps, but this is not what most doctors do. Many patients report that there was no further testing and that they got a diagnosis and a prescription in about ten minutes.108 This is expected, as about 90% of prescriptions for depression pills are written by general practitioners,7:256,108,109 and they don’t have much time.

The reliability of a diagnostic test depends on the accuracy and reproducibility of the test results. The accuracy is defined by comparing the test results with a final true diagnosis. There is no such final true diagnosis in psychiatry, and it is therefore not possible to determine the accuracy of a diagnostic test. But its reproducibility can be determined in observer variation studies where two or more psychiatrists suggest a diagnosis for the same patients.

Four of the five textbooks did not mention a single result from observer variation studies and gave the erroneous impression that psychiatric diagnoses are both valid and reliable. With rare exceptions, e.g. the admission that no questionnaires for diagnosing adult ADHD have been validated,17:615 diagnoses were not doubted. One book claimed that the reliability of the diagnoses is good and noted that, to ensure that the criteria-based diagnoses are sufficiently reliable, they were tested before usage in a big international study, and diagnoses that showed low reliability were either removed or the criteria were strengthened.16:23

It is not clear what the authors referred to, but what they wrote is wrong.7:32,110,111 As one commentator put it after the appearance of the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5): “Real sciences do not decide on the existence and nature of the phenomena they are dealing with via a show of hands with a vested interest and pharmaceutical industry sponsorship.”112

One of the books showed a figure demonstrating that the number of patients diagnosed with schizophrenia in Denmark had quadrupled from 2000 in 1971 to 7400 in 2010.19:225 This should have woken up the authors but they did not comment on this stunning finding, even though something must be wrong with the validity of the diagnosis. This diagnosis can often not be sustained, e.g. it was rejected in 64% of 1023 people.1:173

A psychiatrist wrote to me:7:360 “In my twenties, I had a massive breakdown. At the time, I instinctively resisted all psychiatric labels and medical treatments. When I look back now, I can easily see how, in the wrong hands, I could have been labelled schizophrenic, as I heard voices and had delusions and severe anxiety. Now I know my breakdown was no different to what my patients experience.”

Diagnoses stick to the patient. Once made, everything the patient does or says during a hospital admission becomes suspect, as the patient is under observation, which means that the initial, perhaps tentative diagnosis, all too easily becomes a self-fulfilling prophecy.7:30 The doctor’s intuition and experience may suggest very quickly what the problem is, and there is a considerable risk that the doctor from then on asks leading questions, which yields the required number of error points and leads to a misdiagnosis.

There is much overlap between the different diagnostic categories, often called high comorbidity, although the problem is not that the patient has several “diseases” but that the diseases are so vaguely defined, with overlapping symptoms, that many patients can get several diagnoses.

Even the book that was critical of psychiatric diagnoses failed when it came to observer variation studies. When the two authors, a psychologist and a psychiatrist, discussed the validity and reliability of the diagnoses, they mentioned kappa, Cronbach’s alpha, Hamilton’s Depression Scale and a wealth of other scales and issues over 25 pages, but not a single result from inter-observer variation studies.17:165

Kappa values measure to which extent two observers agree beyond chance. If kappa is 0.60, it means that the agreement is only 60% of the difference between chance agreement and perfect agreement, which is pretty poor. There are many problems with kappa.105 It presents statistical problems and does not tell us, for example, if the disagreement is important, which it surely is for psychiatric diagnoses because a diagnosis almost always lead to drugs, often for many years without interruption,113,114 and also often to a downhill chronic course for the patient.5:8,119:24

The claim that the new diagnostic checklist system introduced by the American Psychiatric Association for its DSM-III in 1980 is reliable has been convincingly refuted in a book.7:32,102,110 The disappointing results when two psychiatrists assess the same people have been buried in a smoke of positive rhetoric in surprisingly short articles, given the importance of the subject.

The documentation is hard to find, but two people did the work, which was a huge task.110 Even the largest study, of 592 people, was disappointing despite the fact that the investigators took great care in training the assessors.111 For bulimia nervosa, which is very easy to diagnose, the kappa values when two physicians interviewed the same people were above 0.80, but for major depression and schizophrenia, two of the most important diagnoses, the kappa values were only 0.64 and 0.65, respectively. This is frightening considering the devastating consequences of false positive diagnoses.

When researchers interviewed 463 people about 91 key symptoms for psychiatric disorders, they found that all of them experienced thoughts, beliefs, moods, and fantasies that, if isolated in a psychiatric interview, would support a diagnosis of mental illness.1:168,115

If the general population is exposed to just a few of the various diagnostic checklists that are being used, a large proportion will get one or more psychiatric diagnoses. When I lecture and try three diagnostic tests on the audience—for depression, ADHD and mania—about a quarter test positive for one or more diagnoses. Imagine if you tested people suspected of having cancer with a test that gave a quarter of them an erroneous cancer diagnosis. We wouldn’t allow such a poor test to be used.

DSM-III from 1980 was replaced by DSM-IV in 1994, which was even worse than its predecessor and listed 26% more ways to be mentally ill. Allen Frances, chairman for the DSM-IV task force, has argued that the responsibility for defining psychiatric conditions needs to be taken away from the American Psychiatric Association because new diagnoses are as dangerous as new drugs: “We have remarkably casual procedures for defining the nature of conditions, yet they can lead to tens of millions being treated with drugs they may not need, and that may harm them.”116 Frances noted that DSM-IV created three false epidemics because the diagnostic criteria were too wide: ADHD, autism, and childhood bipolar disorder.

Psychiatric diagnoses are uncritically believed not only by psychiatrists but also by the media. Even websites that are critical towards overdiagnosis of diseases and overtreatment with drugs and advocate for a new biomedical and social model, convey information like, “One in four people in the world are prone to be affected by mental disorders at some point in life. These mental disorders are the leading causes of ill-health and disability worldwide.”117

Several things are wrong with such commonly seen statements. First, many people are over-diagnosed. They do not suffer from a mental disorder but have problems in their lives. Second, they are not affected by a mental disorder. As already explained, to label people’s problems does not create a being that attacks people. Third, mental disorders are not leading causes of ill health and disability. People suffering from deprivation, poverty, unemployment, and abuse suffer ill health and disability; they are not attacked by some psychiatric monster.96

The bottom of journalism was reached when the United States established The Carter Center’s Guide for Mental Health Journalism, which is the first of its kind.8:162,118 This institution educates journalists to write flawed articles and to never question psychiatric diagnoses. Journalists should pin down exactly what a professional says is wrong with a patient and use that information to characterize a person’s mental state. There is no encouragement for journalists to consider how people so diagnosed see themselves, or whether they accept their diagnostic label, or if the professional might be wrong.

According to the Carter Center, the DSM-5 is a reliable guide for making diagnoses. There is no mention of the fact that the diagnoses are arbitrary constructs created by consensus among a small group of psychiatrists, or that they lack validity, or that psychiatrists disagree a lot when asked to examine the same patients, or that most healthy people would get one or more diagnoses if tested enough.

Reporters are told to write that behavioural health conditions are common and that research into the causes of and treatments for these conditions has led to important discoveries over the past decade. They should also inform the public that prevention and intervention efforts—meaning drugs—are effective and helpful. This is the same message that the American Psychiatric Association and leading psychiatrists all over the world have been promoting for many years.

The guide prompts reporters to echo the message from the American Psychiatric Association that psychiatric conditions are often undiagnosed and undertreated, and that psychiatric treatment is effective. The guide avoids any discussion about how ineffective and harmful the drugs are and makes people believe that “treatment” also includes psychotherapy, even though this is rarely offered.

Nothing is mentioned about overdiagnosis. Reporters are not encouraged to explore why it is that the public health burden of mental disorders has grown dramatically in the past 35 years, at the same time as the use of psychiatric drugs has exploded.5:8,119:24

The guide states that between 70% and 90% of people with a mental health condition experience a significant reduction in symptoms and improvement in quality of life after receiving treatment. The source of this false information is the National Alliance on Mental Illness (NAMI), a corrupted patient organisation.7:357 It is true that most people improve, but that would have happened without treatment. Like many of the textbook authors, the Carter Center seems to have “forgotten” why we do placebo-controlled trials, and it has never been documented that psychiatric drugs improve quality of life; in fact, they worsen it (as will be explained in Chapters 7 and 8).

Reporters are told to emphasize the positive and avoid focusing on the failures of psychiatric care. The guide does not provide any resources for obtaining the perspectives of people with lived experience, most of whom would speak critically of the conventional wisdom.

Unfortunately, the Carter Center is seen as a leader in training journalists on how to report on mental health. It encourages journalists to act as stenographers repeating conventional dogma.

It is difficult to see much hope for America. Journalists are told to convey the strongly misleading narratives created by the drug industry and US psychiatrists on industry payroll to the great harm of our patients and societies.5-7

It is very strange that there is such an institution in America. What the Carter Center does is like telling Chinese journalists that if they want to know what it is like for the Chinese people to live under a dictatorship, they should not ask the people but the Chinese leaders.

One book noted that a good rule of thumb is not to make a depression diagnosis in the first two weeks after stopping drug abuse or intake of medicine.16:258

This principle should apply to all patients. Diagnoses can make it difficult to get the education patients dream about, a job, certain pensions, to become approved for adoption, to get an insurance or child custody, or even just to keep a driver’s licence.120,121 Psychiatric diagnoses are often being abused in child custody cases when the parents get divorced.120 Even when the diagnosis is obviously wrong and the psychiatrist herself seriously doubted it when she made it, it cannot be removed.121 It sticks to the patients forever, as if they were branded cows.

Already on the next page, this book ventured in the opposite direction saying that older people are at risk for underdiagnosis of depression because relatives and sometimes the doctor accept and explain their sadness as understandable, based on the many losses of friends and perhaps the spouse and physical capacity.16:259 The truth is the opposite. Old people are overdiagnosed to an unbelievable extent and sadness is a normal feeling, not a psychiatric diagnosis.

The book about child and adolescent psychiatry mentioned that diagnoses are designations for a condition, a kind of snapshot, and not designations for people.19:36 It advises that diagnoses should be continuously assessed, re-evaluated and changed, and be considered dynamic tools with limited applicability outside of clinical and research contexts.19:36

This is brilliant, but why do psychiatrists not say the same about adults? They also change over time and a person in deep distress will not always be in deep distress. That person might be fine both before and after the visit to the doctor. Why is it then impossible to get a wrong diagnosis removed?

The authors warned that one must not indulge in uncritical use of diagnoses, e.g. they are often used as an admission ticket to social services. They claimed that if clinicians respect the limitations and scope of diagnoses and limit their use of diagnoses for administrative and official purposes, diagnoses do not in themselves imply a risk of stigmatisation.

This looks like a tautology. If diagnoses are used correctly, they do not lead to stigmatisation. If people are stigmatised, it is because diagnoses are not used correctly.

The reality is that diagnoses are not being used correctly, which leads to a lot of stigmatisation and misery.7,8 Think about other issues. If people drove correctly, there would be no traffic deaths. If people drank alcohol correctly, there would be no alcoholics. If people ate correctly, no one would be overweight. What does this tell us? Nothing.

I shall end this chapter by praising Australian psychiatrist Niall McLaren whom I have met several times. He has written a very instructive book with many patients’ stories telling us that anxiety is a key symptom in psychiatry.9 If a psychiatrist or family doctor doesn’t take a very careful history, they might miss that the current episode of distress, which they diagnose as depression, started as anxiety many years earlier when the patient was a teenager. They should therefore have dealt with the anxiety with talk therapy instead of handing out pills.

Niall has developed a standard way with which he approaches all new patients in order not to overlook anything important. It takes time, but the time invested initially pays back many times over and leads to better outcomes for his patients than the standard approach in psychiatry.

Niall explains that “the value of biological psychiatry is that it isn’t necessary to talk to a patient beyond asking a few standard questions to work out which disease he has, and that can easily be done by a nurse armed with a questionnaire. This will give a diagnosis which then dictates the drugs he should have.” Sarcastic? Yes. True? Yes.

It doesn’t seem to matter whether a diagnosis is correct or wrong. It will follow you for the rest of your life.

***

To see the list of all references cited, click here.

***

Mad in America hosts blogs by a diverse group of writers. These posts are designed to serve as a public forum for a discussion—broadly speaking—of psychiatry and its treatments. The opinions expressed are the writers’ own.

***

Mad in America has made some changes to the commenting process. You no longer need to login or create an account on our site to comment. The only information needed is your name, email and comment text. Comments made with an account prior to this change will remain visible on the site.

5 COMMENTS

  1. “The book about child and adolescent psychiatry … advises that diagnoses should be continuously assessed, re-evaluated and changed, and be considered dynamic tools with limited applicability outside of clinical and research contexts.19:36”

    I agree, the same should be true for adults, especially given the “invalidity” of all the DSM stigmatizations.

    And, as one of the few people who was (seemingly) able to get a “bipolar” misdiagnosis off my medical records. I will say to anyone who has had the common adverse or withdrawal effects of an antidepressant and/or ADHD drug misdiagnosed as “bipolar,” bring Bob Whitaker’s ‘Anatomy of an Epidemic’ to the most ethical and competent primary care physician you can find. Plus bringing a copy of this is also helpful.

    https://www.behaviorismandmentalhealth.com/2009/09/06/bipolar-disorder-is-not-an-illness/

    Specifically, point out the “Note” section of that document. If you were given more antidepressants and/or antipsychotics, and was made psychotic, you can freely admit to that. But explain the “psychoses” were created with the antidepressants and antipsychotics, via anticholinergic toxidrome poisoning.

    https://en.wikipedia.org/wiki/Toxidrome

    And “psychoses” can also be created, via a “drug withdrawal induced super sensitivity manic psychosis.”

    So then, after the doctor calls you a “one in a million” medical researcher, and when the doctor asks you if you want the “bipolar” stigmatization removed, you can say that since your “mania” and “psychosis” were created with the drugs – via antidepressant effects or withdrawal, and/or anticholinergic toxidrome and a super sensitivity manic psychosis – none of which is “bipolar.” Thus it would be defamation of character, which is illegal, NOT to take the stigmatizing and incorrect diagnosis off your medical records.

    God bless, and good luck. Thank you for all you do, Peter.

    Report comment

  2. My favorite psych test is the Hoffer/Osmond Diagnostic or HOD, which is bad mouthed by orthodox shrinks who know anything (but not enough) about it. It was first used in the early 1960’s for screening alcoholics for psychedelic therapy in Saskatchewan, and became a regular for orthomolecular practitioners (another reason for Big Time Psychiatry’s dislike) because it was a quantitative test that could be used repetitatively, like a fever thermometer. It doesn’t take a Rhodes Scholar to see why Big Time Psychiatry dislikes it so, in addition to its place of origin (Sakatchewan)

    Report comment

  3. https://youtu.be/4O3ITKAG4_8

    Peter I like most of what you say. I see a comparison in the overdiagnosing of schizophrenia to what some have thought to be the overdiagnosing of body dysphoria. But this does not imply that some people are lying or confused about their schizophrenic experiences or their dysphoria experiences. I am against the view that there can ever be a “one size fits all” prevailing opinion of whatever unique individuals experience. A veiw that says yes schizophrenia is only a symptom of trauma. Or yes schizophrenia is only a symptom of a brain condition. And I am against a blanket view that says nobody has schizophrenia or everyone has schizophrenia. It is a bit like saying nobody is trans or everybody is trans. I don’t like exteme polarized conclusions that allow no freedom for “the different”. The body and brain are so very poorly comprehended and complex and enmeshed that I do not believe anyone can fully declare themselves an “expert” on my own, either an expert medical model proponent or an expert debunker of that medical model. Expertize, whether pro medicine expertize or anti medicine expertize, is about finding out what you have got going on in you. It has its place but it is NOT YOU. It lies OUTSIDE of YOU. It is lesser than YOU.
    So I am glad you wrote that there is no encouragement for journalists to consider how people so diagnosed see THEMSELVES, or whether they accept their diagnostic label, or if the professional might be wrong. This notion of reprioritizing the importance of how the patient sees themselves is essential to the Declaration of Human Rights where it says we are each entitled to “Hold our own opinion”.
    My opinion of me is that I actually do have schizophrenia. I have had about seventeen psychiatrists who would agree with me. I hasten to say that not one of them foisted that diagnosis on me. It was a diagnosis I came to understand for myself. Having the diagnosis does not mean I want any drugs or medical treatment. I believe NO treatment exists for it and the best that can be offered is palliative care and talk therapies. I do not know what causes it in me. It may be that I have a rare brain or it may be the energy field within my brain is like a berserk aurora borealis. It may be I am stuck in a dreamstate while awake or it may be a consciousness thing has gone out of sync, consciousness that may not be rooted in the brain. Or it may be that my hallucinations and dreamy delusions are a form of phantom limb pain, given that I get tactile hallucinations all day, though possible voice hearing is a form of phantom ears or phantom hearing. Brains are majestic and mysterious and so I just do not believe anyone who claims that such and such is going or such and such is not going on. Certitude is best left to each unique individual. I believe I have schizophrenia and I prefer to feel this is my affliction but even if it was not this and supposing it were caused by some forgotten trauma that was causitive, the driven mad have every right to believe of THEMSELVES whatever they freely wish to. As you point out. You may not want me to think I have schizophrenia and you may rather I “hold your opinion” and not my own. This is okay. Sharing new ideas is beautiful. But a person may not want a Muslim to wear a veil, or a Christain to wear a cross. We can want other people to think like us and hold our own opinion but at some point there has to be a wish for the other to find happiness in any opinion even if it disappoints our own.
    But you seem like a man with a mission, to improve the world, and ALL missions that offer improvements matter. But the overarcing mission should be that NO mission is more important than just “freedom of choice”.
    I am very much with you on your campaign to erradicate vile treatments that wreck lives. And I am with you on the need to cease the absurdity and scandal of over diagnosis. And I am with you on poking a stick at the arrogant certitude of blundering science. I may not be quite with you on sort of saying that the experience of hallucinations and delusions are kinda “normal”. It is not normal to suffer so much that you want to jump out of a window. Self annihilation, like self mutilation seen in zoo animals is not healthy and normal. It is not desireable to suffer to that extent and just glibly call it a tough day, even if you feel it is a natural response to awful circumstances, the state of such suffering should not be dismissed as if it does not exist, or it is of no consequence because everyone feels that bad sometimes. The notion that I DO have “something the matter with me” fits with my intolerable level suffering, from my hallucinations and so on.

    I am late for a music class so must wrap this up clumsily. This morning I saw the video linked and thought to share it. It is two philosophers talking about the brain. They drift off topic somewhat towards discussing the follies of scientific education, but the bit about phantom limbs interested me. I like the way these two gentlement are not disguising “arguement” as “academic discission” but rather seem politely curious to “know more” about our fascinating human bodies.

    Report comment

  4. Back to add a smidgen…

    Many see the “trauma imformed care” paradigm as “caring”, as opposed to other paradigms or ideologies or preferrences that are deemed not to be “caring”. Sometimes I like to call the notion of truly caring “baby caring”, in as much as to be actually felt to be “cared for” usually occurs when our “inner child” or “inner baby” is respected for its “feelings” and “thoughts” and “opinions” and granted its dearest wish.

    So I do see “trauma informed care” as “baby caring”, as it is very much in alignment with “respect” for “feelings” and so on, and dignifies “free choice”.

    But being allowed to “hold your own opinion”, as is boldly stated as a basic human right in the Declaration of Human Rights, ALSO is “baby caring”.

    It is “baby caring” EVEN IF that person’s “feeling” or “thought” or “opinion” or “self regard” seems like the very antithesis of “self care” to someone else. What seems like a “baby caring” and “free choice” to one person may seem like recklessness or indolence or indoctrination or ideology or churchy gullibility or entrenched attitudes or political dogma or whimsy or nonchalance or even sin to another person who “holds a different opinion”.

    But if we are to make “freedom of choice” the top of the tree of fine principles for humans to aspire to have for their own sakes and for the sakes of others, then we must tolerate that what “baby caring” seems like for one person, as a path of freedom, may not seem that way to another.

    When there are two actual babies in a room, and one of them wants a teddybear you do not give the other one also a teddybear, since both babies are “different unique individuals”.

    A truly “caring” world will respect this potential to be interiorily “different” in adults and even ALL living creatures, for animals have their own version of “baby caring” needs and their needs for “free choice”.

    I have been MIA’s version of a “big baby” in routinely saying that what “feels” more “baby caring” for “me” at “this time” is not to go in for “trauma informed care” but to continue to believe I have schizophrenia. But by doing this I AM doing something “for me” that is the “same” as the “caring” within “trauma informed care”.

    A truly “caring” kind of “trauma informed care” will welcome those who snub it in favour of their own version of “baby caring” for themselves.

    All over the planet people cluster with their idea of what love is. Catholicism, Marxism, Capitalism, Hinduism, Atheism, Psychology, Tibetan singing bowls, Anarchy, CBT, Trauma Informed Care. But for love to BE love it has to be “rejectable” without punishment or ostracisation or exclusion…or it is not love, and it is not “baby caring”.

    Report comment

LEAVE A REPLY