Lancet Psychiatry’s Controversial ADHD Study: Errors, Criticism, and Responses

Amid calls for a retraction, Lancet Psychiatry publishes articles criticizing the original finding and a response from the authors.

Peter Simons
5
2295

On February 17, 2017, Lancet Psychiatry published a controversial article that claimed to have found brain differences between children with a diagnosis of attention deficit hyperactivity disorder (ADHD) and children without. The authors, led by Martine Hoogman at the Donders Institute for Brain, Cognition and Behaviour, Netherlands, claimed that their evidence supported the theory that ADHD is a brain disorder.

In fact, Hoogman and colleagues go so far in their original article as to say “We confirm, with high-powered analysis, that patients with ADHD have altered brains; therefore ADHD is a disorder of the brain.”

This overstated conclusion ignited an immediate firestorm of controversy. Researchers as prominent as Allen Frances, chair of the DSM-IV task force (who wrote the book on diagnostic categories), joined to denounce the study. An article on which Frances is co-author explicitly stated:

“The most important argument against the authors’ conclusion that ‘patients with ADHD have altered brains’ is that it is not supported by their own findings.”

They add that the conclusion of Hoogman and colleagues “is wildly speculative and dangerously misleading at a time when ADHD is already overdiagnosed and overtreated with medication in high-income and middle-income countries.”

Photo Credit: Flickr

This was part of a larger push by researchers in numerous countries to examine the strange, problematic data presented by Hoogman and colleagues. The most obvious error in the original article was the swapping of the two variables in one analysis, which resulted in a chart showing that individuals with ADHD appeared to have higher IQs than healthy controls—a surprising finding that contradicted previous research.

In response to the criticism, Hoogman and colleagues re-examined their data and found that this was, in fact, an error—their findings were consistent with previous research suggesting that ADHD symptoms are associated with poorer performance on standardized IQ measures.

Researchers, led by Susanne Bejerot at Orebro University, Sweden, re-analyzed the corrected data and concluded that, when controlling for this new IQ data, many of the observed brain differences disappear.

“We found no significant difference between individuals with ADHD and those in the control group in any of the investigated areas of the brain when IQ difference is controlled for.”

In their response, Hoogman and colleagues acknowledge that “Because a slightly lower IQ can be a feature of ADHD, adjusting for IQ will remove disorder effects in brain regions associated with both ADHD and IQ.” That is, they acknowledge that when controlling for the differences in IQ, their supposed brain differences disappear. They argue that lower IQ is caused by ADHD and should not be statistically controlled. This is a controversial approach to ADHD research, as causation has not been demonstrated and low IQ is not a criterion for a diagnosis of ADHD. Thus, any brain differences found are likely to be due simply to differences in IQ, not to ADHD diagnosis.

Critics of the study (see MIA Report) also see this error as illuminating its fundamental problem: they contend that Hoogman and colleagues were too eager to prove their hypothesis and didn’t thoroughly examine what results might be supported by their data. Critics argue that this error should have been caught by Hoogman and colleagues, by the peer reviewers who approved the article, or by the editorial staff of the journal that published it. That such an unusual and compelling finding went completely unnoticed and unmarked indicates to critics that the authors, reviewers, and editorial staff were so biased by their perception of ADHD as a brain disorder that they ignored substantial evidence to the contrary.

Although this led to a petition to retract the article, the editorial staff of Lancet Psychiatry instead decided to publish a correction and then devote an entire issue to airing the criticisms and a rejoinder from Hoogman and colleagues.

Amongst the articles critical of the original study was an analysis led by Trudy Dehue at the University of Groningen, Netherlands. Dehue and colleagues found that the original results actually show that for adults, there was no difference in brain volume between those with ADHD and those without. Even for children, Dehue and colleagues argue that the small effect size found indicates that there was a considerable overlap in brain sizes between the two groups (95% of the two populations overlapped). That is, even on average there was so much overlap that only the extreme outliers were relevant.  Another way of looking at this finding is that 95 out of every 100 children diagnosed with ADHD had a brain volume that matched that of a child in the “healthy control” group.

According to Dehue and colleagues, the original findings “showed no significant differences in the brains of adults with ADHD, which suggests that the minor differences in children mostly vanish when they grow up. This finding could have been actual headline news given the claims by commercial pharma companies and sponsored experts that ADHD is a life-long disorder in need of life-long treatment.”

Dehue and colleagues, in fact, go so far as to explicitly state:

“There is no point in conveying that a child with ADHD has a brain disorder.”

Another article published by Lancet Psychiatry this month addressed the original finding that emotional processing areas of the brain were implicated as differing between children with an ADHD diagnosis and children without the diagnosis. The authors, Alison Poulton and Ralph Nanan at the University of Sydney, Australia, argue that, as before, Hoogman and colleagues should have highlighted this unexpected finding. Poulton and Nanan point out that there is no emotional component to a diagnosis of ADHD, and question why researchers would find such a brain difference. Their argument is that the diagnosis of ADHD often overlaps with other diagnoses commonly given in childhood, such as oppositional defiant disorder (ODD), the prominent feature of which is emotional dysregulation.

Poulton and Nanan argue that many of the findings of Hoogman and colleagues are actually due to the low specificity of the ADHD diagnosis and its overlap with other diagnoses. If the results are related to ODD rather than ADHD, this calls into question whether the data can be used to suggest that ADHD is a brain disorder.

Hoogman and colleagues simply agree with this criticism and state that they hope to investigate further. They do not address the implications that this has for the interpretation of their results.

The diagnosis of ADHD has been a controversial one. The primary criticism has been the loosening of diagnostic categories, which leads to overdiagnosis. Critics argue that the criteria for a diagnosis of ADHD are so loose that almost anyone could receive the diagnosis (one criterion is “feelings of restlessness,” for instance). Other studies have shown that it is the youngest children in classrooms who were more likely to receive a diagnosis, and they are more than twice as likely to receive stimulant medication than the oldest children. Researchers theorize that the youngest children struggle more in school and are at an earlier stage of development than children who are nearly a year older—which can make them more likely to fit the behavioral criteria for a diagnosis of a mental disorder.

A 2013 New York Times article exposed misleading pharmaceutical industry advertising practices (to both doctors and consumers), which have resulted in numerous FDA investigations and punishments, and linked them to increased ADHD overdiagnosis and overprescription.

Dr. Keith Connors, who helped shepherd the idea of ADHD into being and who conducted the first studies on methylphenidate (Ritalin), says that within the past few years he began to understand that his work has been disastrously misused. He says he “announced to stunned colleagues that the over-diagnosis of ADHD was “an epidemic of tragic proportions.”

“Tragic because many kids get the wrong diagnosis and really have a different problem that needs a different treatment: or they are normal youngsters given a treatment they don’t need; or the drugs prescribed for them are given away or sold to other students wanting a quick fix for studying or partying—a reason why schools and colleges now have huge numbers of students using stimulant drugs, and why emergency rooms are increasingly overwhelmed with overdosing youngsters.”

Hoogman and colleagues were obliged to respond to arguments by prominent researchers that their data supports the idea that ADHD is not a brain disorder. They had the good grace to admit some limitations, such as acknowledging that they had erred in reporting the IQ scores and that this would affect their results. However, despite their findings suggesting the exact opposite, they continue to argue that “ADHD is a disorder by all standards of psychiatric nosology.” That is, even if it is unsupported by their data, they claim that we must accept the common wisdom that ADHD is a brain disorder.

Hoogman and colleagues themselves sum up the argument of their critics nicely:

“The criticisms of small effect sizes raised by these letters imply that we should only use the term brain disorder when everyone with the disorder shows the same pattern of brain abnormalities. By that definition, no psychiatric disorder would be a brain disorder.”

 

****

Batstra, L., te Meerman, S. Conners, K., & Frances, A. (2017). Subcortical brain volume differences in participants with attention deficit hyperactivity disorder in children and adults. Lancet Psychiatry. http://dx.doi.org/10.1016/S2215-0366(17)30107-4

Bejerot, S., Nilsonne, G., & Humble, M. B. (2017). Subcortical brain volume differences in participants with attention deficit hyperactivity disorder in children and adults. Lancet Psychiatry. http://dx.doi.org/10.1016/S2215-0366(17)30160-8

Dehue, T., Bijl, D., de Winter, M., Scheepers, F., Vanheule, S., van Os, J. . . . Verhoeff, B. (2017). Subcortical brain volume differences in participants with attention deficit hyperactivity disorder in children and adults. Lancet Psychiatry. http://dx.doi.org/10.1016/S2215-0366(17)30158-X

Hoogman, M., Buitelaar, J. K., Faraone, S. V., Shaw, P., & Franke, B. (2017). Subcortical brain volume differences in participants with attention deficit hyperactivity disorder in children and adults – Authors’ reply. Lancet Psychiatry. http://dx.doi.org/10.1016/S2215-0366(17)30200-6

Hoogman, M., Bralten, J., Hibar, D.P., Mennes, M., Zwiers, M. P., Schweren, L. S. J. . . . Franke, B. (2017). Subcortical brain volume differences in participants with attention deficit hyperactivity disorder in children and adults: a cross-sectional mega-analysis. Lancet Psychiatry, 4, 310–19. http://dx.doi.org/10.1016/S2215-0366(17)30049-4

Poulton, A., & Nanan, R. (2017). Subcortical brain volume differences in participants with attention deficit hyperactivity disorder in children and adults. Lancet Psychiatry. http://dx.doi.org/10.1016/S2215-0366(17)30105-0

Support MIA

Enjoyed what you just read? Consider a donation to help us continue to produce content, provide up-to-date research news, offer continuing education courses, and continue building a community for exploring alternatives to the current paradigm of mental health. All donations are tax deductible.

$
Select Payment Method
Loading...
Personal Info

Credit Card Info
This is a secure SSL encrypted payment.

Donation Total: $20.00

5 COMMENTS

  1. Well – perhaps one valid observation?

    “BY THAT DEFINITION, NO PSYCHIATRIC DISORDER WOULD BE A BRAIN DISORDER”.

    Surely it is a first principle of ethical medical practice that to advise someone that they have a “brain disorder” demands that they have a reproducible and scientifically validated pathological diagnosis.

    It is psychologically, emotionally, socially, economically and personally devastating to be advised: “you have a brain disorder”.

    “THE GREATER THE IGNORANCE THE GREATER THE DOGMATISM.

    (Professor Sir William Osler.)

    TRM 123 Retired Physician.

  2. I have not read this article, and I am going straight to the comments. I know this article is about some imaging study (a commonplace tactic to sell the “scientificness” of psychiatry).

    People should know a few things when they step into the offices of these psych. people. You will never have any imaging done on you. As an individual, it is practically irrelevant to you.

    3 things happen when you enter into the offices of mental health workers:

    1.) You may be labelled (along with it may come years of disease mongering and internalisation of labels)

    2.) You may be prescribed drugs (along with it may come debilitating effects)

    3.) You may have social and moral judgements cast on you (this may not apply much to “outpatients”, but more so if you stay in one of these places for an extended duration for any purpose), and family members (who may or may not have your best interests in mind) and mental health workers together may end up making decisions for you.

    These are the 3 things people should always be aware of. And these are the only things that will happen to you in there.

    The imaging garbage is nonsense. It is practically irrelevant to the common man (except in the sense that sometimes it may be extremely practically relevant when the psych people use them to justify their methods and tell you “So, you don’t believe in scans and MRIs etc.”, in which case the appropriate response is to say “Do you do any imaging in clinical practice? No, right? Then let us not even go there). Behind psychiatry, there is a legal system, there are social systems of everyday life, there is a prison system, and the medical system. Be aware of these things, and of how psychiatry and its associated labels, laws, drugs etc. may be used against you directly or indirectly, whether by close ones or strangers.

  3. “The criticisms of small effect sizes raised by these letters imply that we should only use the term brain disorder when everyone with the disorder shows the same pattern of brain abnormalities. By that definition, no psychiatric disorder would be a brain disorder.”

    Excellent point! They have accidentally drawn the correct conclusion – no psychiatric disorder can qualify as a “brain disorder.” This is like saying that not all people with cancer should show the same indications of inappropriate growths, or that not all people with high blood pressure need to have higher than average blood pressure. Such idiocy is what passes for “science” in the world of psychiatry!

    —- Steve

  4. Roger Griggs, …. said, “There’s no way on God’s green earth we would ever promote stimulants directly to consumers”
    The doctor and the pharmaceutical Co. accuse each other of doing the damage.
    No God to lay judgement.
    Will society self implode from all the children being harmed?

  5. I left this comment for MIA and reproduce it here.
    “One only need ask oneself who benefits from the diagnosis of ADHD. Well nearly everyone except the Taxpayer. Jerome Kagan is correct. The pharmaceuticals benefit, The Psychiatrists benefit, It is a ready made excuse for failings in teaching, the school benefits because they get extra funding, the pupil benefits because again it is a ready made excuse for failure, the parents benefit because the child gets extra paid attention, the pupil gains further by been given aids such as Laptops free, extra time to take exams. The only loser loses big – the Taxpayer. Jerome Kagan is absolutely correct. THE BIG CLUE: FOLLOW THE MONEY.”