Editor’s Note: Over the next several months, Mad in America is publishing a serialized version of Peter Gøtzsche’s book, Critical Psychiatry Textbook. In this blog, he discusses the problems with observational studies and other flaws in ADHD research. Each Monday, a new section of the book is published, and all chapters are archived here.
For affective disorders, some authors expressed less certainty than for schizophrenia. In one textbook, the authors claimed that the risk of affective disorder is increased 3-4 times if a parent is depressed,19:210 and the risk of bipolar is increased 4-6 times if a first-degree relative is bipolar,19:216 but they also admitted that it is very difficult to separate inheritance and environment and to investigate if the changes are a cause or consequence of the depressive condition.19:210
A major risk factor for becoming depressed has nothing to do with biological psychiatry but is simply living a depressing life you feel you cannot escape from. There was very little information in the textbooks about this. One book said that stress, living conditions, and trauma can play a role for affective disorders but not how much, in contrast to its claims about the role of genes, which was 50%.17:353 Another textbook mentioned trauma, especially in relation to the first manic episode,18:113 and a third emotional abuse, neglect, and physical abuse with odds ratios as high as 9 to 12.16:263 It also noted that steroids, birth control pills, and oestrogen-blocking drugs increase the risk of depression but there was no mention that psychiatric drugs, e.g. benzodiazepines, depression pills, and ADHD drugs, can also cause depression,7,8,11,34,44,45 even though this is highly relevant, given their widespread use.
This was a general problem with the textbooks. I gave another example just above of the psychiatrists protecting their guild interests by not mentioning that the drugs they use can cause the very disorders they try to treat. This is dishonest and unhelpful.
ADHD and the fallibility of observational studies
For the ADHD diagnosis, risk factors included the mother’s prenatal intake of tobacco, alcohol, or cocaine; decreased intrauterine growth; foetal exposure to insecticides, lead, or mercury; pre-eclampsia; premature birth; complicated births with hypoxia; low birth weight; postnatal infections; exposure to heavy metals; and possibly neuroinfections.17:612,18:229
It was claimed that even though environmental factors may contribute, they play a minor role.18:229
It should always be remembered that such claims about causality come from observational studies. They might therefore not be correct, but I did not notice any reservations in the text-books.
In contrast, top researchers in epidemiology have strong reservations about what their colleagues publish. Observational studies are fraught with difficulties, which is easy to realise if we look at nutritional research.46 People who eat little fruit and vegetables, or drink more than others, cannot be compared to vegetarians and teetotalers. They differ from them in all sorts of ways that could influence their longevity. Therefore, if nutritional advice is to be believed, it must come from carefully conducted randomised trials.
If we are to rely on observational evidence, high quality research is required, and the signal must be substantial because there is so much bias in these studies. Top epidemiologists have stated that, because it is so easy to be fooled, any less than stunning results are almost impossible to believe.47 Some said that even a threefold risk increase is not persuasive, and that they can only be persuaded if the lower limit of the 95% confidence interval falls above a threefold increased risk.
When I examine claims made by psychiatrists by looking up the sources, I almost always find that the claims cannot be substantiated. To show you how this works, I examined one of the claimed risk factors for ADHD, low birth weight. I found a relevant article immediately by googling low birth weight ADHD, which mentioned that “Several studies have reported that children with a low or extremely low birth weight are as much as 3.8 times more likely to meet diagnostic criteria for ADHD.”48 This is bad science. If we describe several studies, we should not cherry-pick the one with the most extreme result but should say what they show on average, or what the median result was.
The authors quoted four studies, and I looked up the first one. It included 137 very low birth weight (VLBW) children that were compared at 12 years with a sample of matched peers for several psychiatric symptoms.49 The main risk was ADHD, which was diagnosed in 31/136 (23%) of the VLBW children, compared to 9/148 (6%) of peers.
The risk ratio was 3.75, but I calculated that the 95% confidence interval went from 1.85 to 7.58. This means that the true risk of getting an ADHD diagnosis is likely to be between 2 and 8 times higher for VLBW children than for normal children.
Assuming the result is correct, which we cannot know, as positive results get published more often than negative ones (and I happened to select the most positive one), we may calculate how big the study should have been if the lower limit of the confidence interval should exceed 3. The lower limit becomes 3, if I multiply all numbers by 10. Thus, the study should have been 10 times bigger to arouse any interest among top epidemiologists.
This is a general problem with observational studies. They are usually far too small, and considering their inherent biases, with the additional risk of selective publication of results that happened to be positive by chance, this means that most results from observational studies are misleading. Even if the studies are very large, they are often misleading, as we cannot eliminate the biases, no matter how we try to adjust for them statistically.
The VLBW study was biased. A table showed that parents of VLBW children were socioeconomically disadvantaged compared to the control group. Furthermore, the authors noted that parents with psychiatric disturbances were more likely to have children who were also vulnerable to psychological problems; that mothers of VLBW children were more depressed than mothers of other infants; and that most VLBW children had limited access to their mothers during the first six months of life. The authors found this of particular interest. So do I, as this could be the explanation for their findings rather than low birth weight.
It is not possible to adjust reliably for such differences with statistical methods. An ingenious study, in which a statistician used raw data from two randomised multicentre trials as the basis for observational studies that could have been carried out, showed that the more variables that are included in a logistic regression, the further we are likely to get from the truth.50 The statistician also found that comparisons may sometimes be more biased when the groups appear comparable than when they do not; that adjustment methods rarely adjust adequately for difference in case-mix; and that all adjustment methods can on occasion increase systematic bias. He warned that no empirical studies have ever shown that adjustment, on average, reduces bias.
His study may be the most important one I have come across in my whole career. But I have not met a single researcher who did not know him personally, that was aware of his highly important results.
This is not to say that observational studies cannot be useful. Many things cannot be studied in randomised trials and we therefore have no other option than to do observational research. But it is unacceptable that the textbooks almost always described the results of such studies as if they represented the truth, with no caveats.
Other flaws in the ADHD research
One textbook provided the sobering information that ADHD is defined arbitrarily as one end of a normal distribution curve, and that brain development is delayed but not qualitatively different from that in healthy children.18:229
If this is correct, we would expect more of those children born in December to have an ADHD diagnosis and be in drug treatment than those born in January in the same class, as they have had 11 fewer months to develop their brains. This is exactly the case. A Canadian study of one million school children showed that the prevalence of children in drug treatment increased pretty much linearly over the months from January to December,51 and 50% more of those born in December were in treatment.
There are other studies that show the same. This means that if we approach the children with a little patience that allows them to grow up and mature, far fewer would get an ADHD diagnosis.
The diagnosis arises primarily from teacher complaints and parents are often told that their kid cannot come back to school unless he or she is on an ADHD drug. A general practitioner told me that a schoolmistress had sent most of her pupils for examination on suspicion of ADHD.7:138 It was clearly she who was the problem, not the kids, but as soon as the kids are branded with ADHD, it relieves everyone of any responsibility or incentive to redress the mess they have created, either at school or at home.
We have decided as a society that it is too laborious or expensive to modify the kids’ environment, so we modify the kids’ brain instead. This is cruel, as I shall explain in Chapter 9. The United States spends over 20 billion dollars a year drugging children for ADHD, which is enough to pay the mid-career salaries of an extra 365,000 teachers.52 And this goes up and up. The number of children with an ADHD diagnosis increased by 41% in just 8 years, from 2003 to 2011.53
Only one of the textbooks mentioned any of the important studies of the prevalence of the ADHD diagnosis in school classes according to age.17:51 The belief in the false story about ADHD being a brain disease is so strong that it is close to impossible to correct the harmful narrative.
The indoctrination is very effective. In 2022, one of my colleagues gave a lecture in critical thinking for psychiatry residents. He asked them to review three studies.
One study showed that 16% of those with an ADHD diagnosis had genetic abnormalities (copy number variants), compared with 7% in the controls.54 The researchers concluded that ADHD was a genetic disease. The residents were asked if this small difference was significant and could be applied to ADHD as a diagnostic category.
The second study looked for a genetic abnormality in neuropsychiatric disorders and is often cited for providing evidence of it.55 The researchers reported that there was a common genetic component involved in the pathogenesis of five neuropsychiatric disorders. One of the disorders was ADHD. They found that those with ADHD were three times more likely to have this abnormality. But if you combine the data from two tables, you will find that only 0.3% had the genetic abnormality, so 99.7% didn’t have it. But because only 0.1% of controls had it, the odds ratio was three.
The third study found that children with an ADHD diagnosis have smaller brains than other kids.56 The effect size was 0.1, which means that patients with the diagnosis have a 47% chance of having a brain bigger than normal.57 The effect size is also called the standardised effect size. It is the effect divided by the standard deviation of the measurements. This allows comparisons of measurements on different but similar scales. If, for example a scale has a 10-fold greater range than another scale, the standard deviation will also be 10-fold bigger, and the effect sizes can therefore be combined in meta-analyses.
The residents emphasised that genetic differences were highly significant and said that the brain volume study suggested that ADHD was a neurodevelopmental disease.
My colleague was flabbergasted. He told the residents that the data showed that nearly all the kids diagnosed with ADHD didn’t have a genetic abnormality; that the odds ratio for the five-disorder study was meaningless; and that the brain volume study showed that there was a 96% overlap between kids with the diagnosis and kids without.57
The residents then got hostile. Didn’t the lecturer understand that ADHD and the other dis-orders were biological disorders; that they were illnesses like diabetes or cancer?
My colleague had seen much insanity in psychiatry, but he told me that this was the most hopeless thing he had ever experienced. It is frightening that such people are supposed to take care of psychiatric patients in an evidence-based fashion. They are clearly not able to do this, as it requires that you have a minimum understanding of science.
The study that claimed that children with an ADHD diagnosis have small brains has been widely condemned. Lancet Psychiatry devoted an entire issue to criticisms of the study. Allen Frances, chair of the DSM-IV task force (DSM is the Diagnostic and Statistical Manual of Mental Disorders, issued by the American Psychiatric Association), and Keith Conners, one of the first and most famous researchers on ADHD, re-analysed the data and found no brain differences.58
The original researchers wrote in the discussion that “our results coming from highly powered analysis, confirm that ADHD patients truly have altered brains, i.e. that ADHD is a disorder of the brain. This is a clear message for clinicians to convey to parents and patients, which can help to reduce the stigma that ADHD is just a label for difficult kids and caused by incompetent parenting.”56
The stupidity in this message is heart-breaking. One of the critics wrote in Lancet Psychiatry that “there is no point in conveying that a child with ADHD has a brain disorder.”59 Of course not. It is not true, and it does not reduce stigma to tell such nonsense to clinicians, parents and children; it increases stigma.
The American Academy of Child and Adolescent Psychiatry writes on its homepage:60 “ADHD is a brain disorder. Scientists have shown that there are differences in the brains of children with ADHD … some structures in the brain in children with ADHD can be smaller than those areas of the brain in children without ADHD.”
In September 2021, The World Federation of ADHD International Consensus Statement was published.61 It presented what the authors called “208 evidence-based conclusions about the disorder,” but several of these were incorrect, e.g. “When made by a licensed clinician, the diagnosis of ADHD is well-defined and valid” and that treatment with ADHD medications reduces substance abuse, educational underachievement, and criminal activity (see Chapter 9).
There were 80 authors, so most of them cannot have contributed much to the paper. They did not specify which contributions they made but many of them had numerous conflicts of interest in relation to the drug industry. The paper asserted that there is a “polygenic cause for most cases of ADHD, meaning that many genetic variants, each having a very small effect, combine to increase risk for the disorder. The polygenic risk for ADHD is associated with general psychopathology … and several psychiatric disorders.”
The great deception of doctors and the public occurs, among other reasons, because very small group differences compared to controls are represented as abnormalities found in individuals diagnosed with ADHD, even though the study data, when properly parsed, show that not to be true.57 Once the data are reviewed, it becomes clear that decades of research into possible abnormalities in genes, brain volume, and brain chemicals all turned up negative.
To see the list of all references cited, click here.
Mad in America hosts blogs by a diverse group of writers. These posts are designed to serve as a public forum for a discussion—broadly speaking—of psychiatry and its treatments. The opinions expressed are the writers’ own.
Mad in America has made some changes to the commenting process. You no longer need to login or create an account on our site to comment. The only information needed is your name, email and comment text. Comments made with an account prior to this change will remain visible on the site.