Editor’s Note: Over the next several months, Mad in America is publishing a serialized version of Peter Gøtzsche’s book, Critical Psychiatry Textbook. In this blog, he discusses how textbooks portray brain imaging data for psychiatric diagnoses and the flaws with that body of research. Each Monday, a new section of the book is published, and all chapters are archived here.
According to the psychiatric narrative, psychiatry is built on the biopsychosocial model of disease that takes biology, psychology, and socio-environmental factors into account when explaining why people fall ill.8
The reality is vastly different. Ever since the president of the US Society of Biological Psychiatry, Harold Himwich, in 1955 came up with the absurd idea that psychosis pills work like insulin for diabetes,4:46 biological psychiatry has been the predominant disease model.
Despite 15 years of intense studying, I have been unable to find any important contribution of biological psychiatry to our understanding of the causes of psychiatric disorders and how they should best be treated.
The strong belief in biological psychiatry is also dominant in the textbooks. There is a lot about brain scan studies and brain chemistry and comparatively little about traumas, other psychosocial factors, poverty, discrimination, and other poor life conditions, even though they are important determinants for psychiatric disorders.35,36,61
One textbook was particularly misleading as it noted that social causal factors such as poverty, loneliness, and housing shortages are of a more indirect nature and contribute to the maintenance of already established diseases.18:27
A little light shined through here and there. Elsewhere, in the same book, other psychiatrists contradicted this. They wrote that general improvements in housing standards, job opportunities, and family support have great importance for primary prevention, and that traumas, like losses and physical and emotional abuse, are important factors for development of psychopathology.18:293
Another book noted, with a reference,62 that childhood traumas are associated with elevated DNA methylation of brain-derived neurotrophic factor in patients with borderline personality disorder and that those who respond to psychotherapy have a decrease in DNA methylation.17:41 However, the quoted paper showed that, for all patients, psychotherapy significantly increased methylation. Thus, the information in the textbook was misleading, as one obviously cannot separate those who will respond beforehand from those that won’t respond. The authors of the paper even blamed the patients for the calamity: “Poor responders were mainly responsible [my emphasis] for the increase.”
The textbook authors went to great lengths to convince their readers that the origin of psychiatric problems should not primarily be sought in people’s living conditions but in the brain. Thereby they propagated the idea that psychiatric disorders are individual mishaps and not something that primarily comes from outside the individual and secondarily affects the brain.
We are told that biological psychiatry has created important results within genetics and psychopharmacology, and with imaging techniques,17:919 and that imaging studies in depression have led to increased knowledge of hippocampus’ role, which has produced clinically relevant results.17:910 Pretty conveniently, the authors “forgot” to tell us in what way the imaging studies have been useful for clinicians.
One of the textbooks explained that neuropsychiatry is a further development of what was formerly called biological psychiatry.17:207 But an erroneous idea does not become evidence-based or useful by giving it a new name, and to postulate that billions of people have wrong brains, which essentially is what biological psychiatry does, is as bad as it gets.
Schizophrenia and related disorders
The textbooks claimed it is indisputable that schizophrenia has a neurobiological background;20:401 that schizophrenia16:207,18:39,18:79 and affective disorders have an organic basis;18:39 and that MRI (magnetic resonance imaging) and PET (positron emission tomography) scans have shown brain atrophy and disturbed brain metabolism in patients with schizophrenia and depression.18:27
When declaring schizophrenia an organic disease, the psychiatrists focused on brain imaging studies and brain chemistry, and the information in the textbooks was often very detailed. For example, one textbook noted that patients with schizophrenia have enlarged ventricles, smaller temporal lobes (superior gyrus temporalis), smaller medial temporal structures (hippocampus, amygdala and parahippocampus) and smaller frontal lobes.19:227 In particular, the grey matter appeared to be affected. It was claimed that since several of these changes occur already at the onset of the disease, they are probably not a result of long-term medication.19:227
These claims are contradicted by studies that found that psychosis pills shrink the brain in a dose-related fashion and that the disease could not explain these changes,63,64 but the textbook authors avoided commenting on these well-known studies.
One of the textbooks admitted that some of the reduction in grey matter seen with PET scans or fMRI (functional magnetic resonance imaging, which measures the small changes in blood flow that occur with brain activity) may be caused by the use of psychosis pills but added that several changes occur already at the onset of the disease and that there are also brain changes in those who later develop psychosis.17:309 Another textbook noted that, although the brain changes were minor, they were also seen in people who have not received psychosis pills before.16:221
The problem with such statements is that brain scan studies are highly unreliable, as I shall explain in detail below. If any reliable studies had shown this, it would have been such a great triumph for biological psychiatry that we would have heard about them incessantly, but we do not, and in both cases, the authors did not give any references to their remarkable claims.
Another textbook claimed it was well substantiated that there are neuroanatomical changes; that psychotic patients have enlarged ventricles and 4% less grey matter than healthy people; and that first-episode patients also displayed this, albeit to a lesser degree than in chronic patients.20:405 On the other hand, the authors also noted that the findings were contradictory, with reference to a meta-analysis of over 18,000 subjects with schizophrenia,65 and they noted that, although there is a progressive loss of brain tissue over time, it is very difficult to separate causal factors, e.g. drugs and drug abuse.20:406
This honesty did not last long. The same authors claimed that untreated psychosis increases the loss of brain volume and that it is likely that psychosis pills can offer some protection. This has never been shown, and it is extremely unlikely. Psychosis pills do not protect the brain; they harm the brain in numerous ways (see Chapter 7). Many studies have shown that psychosis pills kill nerve cells,4:176,5:63 and they shrink the brain, too.63,64
For affective disorders, the textbook authors’ opinions were more divided than for psychoses. Some were highly confident that the diseases are biological, while others had reservations.
We are told that depressive conditions are associated with neurobiological changes; that there is nonspecific white matter change;17:357 that cognitive difficulties in affective disorders may be related to neurodegeneration;17:358 that MRI and PET suggest a significant biological component;18:113,18:122 that prolonged untreated depression may explain the brain atrophy that can be measured;18:124 and that bipolar children have decreased amygdala volume and an altered connection between the prefrontal cortex, the basal ganglia, and the limbic system.19:216
One book one noted that recurrent or prolonged depression causes atrophy of the hippo-campus.16:267,16:557 In the same book, however, other authors wrote that it was not clear if white matter hyperintensities in bipolar were caused by the disease or the treatment or were present before any of these.16:295
This was one of the very rare admissions in the books that the changes observed on brain scans might be caused by the drugs. Usually, this possibility was totally ignored, as it also is in scientific articles. An editor of one of the textbooks,18 professor Poul Videbech, published a meta-analysis in 2004 of imaging studies66 that reported that depression causes a reduction of 9% in the size of the hippocampus, which one of the textbooks quoted.20:433 Discussing the limitations of his study, Videbech noted that cross-sectional studies such as those he had included in the meta-analysis cannot conclude about causality. He asked: “Does the depression cause shrinkage of the hippo-campus or are subjects with small hippocampi susceptible to depression?”
It did not occur to Videbech that people with depression are treated with depression pills, and that it could be the pills that caused brain atrophy. He did not mention this possibility, not even when discussing confounders where he included stress and alcohol abuse. He noted that, in three studies, a smaller volume in the right hippocampus or reduced density in the left “was linked to poor response to antidepressant medication,” and that, if this result is confirmed, “it is clinically very interesting as a potential predictor of treatment response.”
I cannot make any sense out of this sentence. It seems to me that Videbech suggested that, perhaps in future, all depressed people should have a brain scan. This won’t happen.
Strangely, ADHD—one of the most controversial diagnoses in all of medicine—was claimed to be one of the psychiatric disorders with the strongest evidence for a neurobiological etiology.17:612 It was called a neurodevelopmental disorder,16:462 or a neuropsychiatric developmental disorder,17:610 characterised primarily by biological risk factors, and not primarily by exposure to psychosocial risk factors and stressful events in childhood.19:51 It was claimed that ADHD represents a cerebral organ dysfunction and that clinical and neuroradiological studies have shown dysfunctional activity in the frontal lobes.19:112
Historically, ADHD was called “minimal brain dysfunction” and the focus was on a structural brain damage no one had ever seen.17:610
The fact is that ADHD is a social construct and that no reliable studies have shown any biological origin for this construct, or that the brains of people with this diagnosis are different to the brains of other people.7,10 One textbook that noted that CT and MRI scans had shown less brain tissue and less white matter acknowledged that there are many methodological problems with imaging studies.17:612
In contrast, a chapter on ADHD written by two psychologists had no reservations.20:469 It claimed, with references, that patients diagnosed with ADHD have smaller size of especially the right caudate nucleus, cerebellum and the total volume of the brain;67 that they have less grey substance in the right caudate nucleus, ventromedial prefrontal cortex and rostral cingular gyrus, which are not related to the use of ADHD medication;68 and that fMRI scans have also shown differences to healthy people.69
It would be a waste of time to read these papers because the whole scanning literature is highly unreliable (see below on this page). But briefly, the first study was a meta-analysis of MRI studies that included all regions across all studies and found global reductions for ADHD subjects compared with control subjects, with an effect size of 0.41.67 An effect size this big is a measure of the amount of bias in the reviewed studies and not of true differences. In other words: garbage in, garbage out.
The second study was also a meta-analysis, of predominantly very small studies, which we know are highly unreliable.68 It included two datasets, and one had only 34 patients with ADHD in the studies, on average, the other only 16 patients.
The third study included 20 patients with ADHD.69
All three papers and similar ones should be ignored. The psychologists dressed themselves as serious scientists and then quoted pure garbage.
A textbook noted that brain imaging studies had shown changes in amygdala in children with anxiety disorders but mentioned that it was not known if this was the cause of the disorder or a consequence of it.19:146
The other textbooks had no such reservations. Two psychologists wrote that patients with OCD have a dysfunction in the brain’s frontostriatal circuit, which is the connection between the frontal lobes and the basal ganglia and thalamus, and that the metabolism in the right caudate nucleus was reduced if the patients had taken depression pills or had received cognitive behavioural therapy.20:479
Other authors wrote that patients with OCD had brain atrophy and increased grey matter but offered no references in support of this astonishing claim.17:418
We are told that the basal ganglia, thalamus and orbitofrontal part of the cortex are involv-ed;19:162 that some studies have shown normalisation of dopaminergic hyperactivity in striatum after treatment with depression pills or cognitive behavioural therapy;17:419 that imaging studies have shown overactivity of the orbitofrontal cortex and caudate nucleus in patients with OCD that disappeared on successful treatment with drugs or psychotherapy;16:364 and that effective drug or behavioural therapy can normalise the affected brain areas.19:162
The last two sentences are tautologies. They contain empty information like in the sentence: It will either rain tomorrow or it will not rain. If “effective” or “successful” treatment is used, the brain changes are normalised. If they are not normalised, the treatment was not effective, or the patient was treatment resistant. This is a win-win situation that seems to confirm something that is not correct, namely that there are brain changes in the first place.
Brain scan studies are highly unreliable
We should be highly sceptical towards the results of imaging studies. The textbooks did not convey much doubt but the one where all three editors were psychologists noted that they were aware of the limitations of the methods used in imaging studies and they questioned the findings that had been made.20:10
Another textbook noted that the findings obtained with structural and functional scans were inconsistent and varying, especially those obtained with functional MR scans that measure small changes in blood flow to various areas of the brain while the patient is given various tasks.17:329
This whole area is a mess of highly unreliable research.7:233
A 2009 meta-analysis found that the false positive rate in neuroimaging studies is between 10% and 40%.70 And a 2012 report written for the American Psychiatric Association about neuroimaging biomarkers concluded that “no studies have been published in journals indexed by the National Library of Medicine examining the predictive ability of neuroimaging for psychiatric disorders for either adults or children.”71
One good research paper can sometimes make hundreds of poor studies redundant. This is the case for a 2012 systematic review by Joshua Carp that surveyed the methodological state of the art in a random sample of 241 fMRI studies.72
Carp found that many of the studies didn’t report on critical methodological details about experimental design, data acquisition, or analysis, and many studies were underpowered. Data collection and analysis methods were highly flexible. The researchers had used 32 unique software packages, and there were nearly as many unique analysis pipelines as there were studies. Carp concluded that because the rate of false positive results increases with the flexibility of the design, the field of functional neuroimaging may be particularly vulnerable to false positives. Fewer than half of the studies reported the number of people rejected from analysis and the reasons for rejection, and the median sample size per group was only 15, which generates an enormous risk of selective publication of those results that happened to agree with the investigators’ prejudices. The order of processing procedures also permits substantial flexibility in the analyses.
Replication is essential for the trustworthiness of science, and scientific papers must report experimental procedures in sufficient detail that allows independent investigators to reproduce the experiments. This is far from the case in imaging studies.72
Carp published another important study in 2012.73 He sought to estimate the flexibility of neuroimaging analysis by submitting a single fMRI experiment to the many unique analysis procedures described in the literature. Considering all possible combinations of these strategies, he came up with 6,912 unique analysis pipelines.
“Nearly every voxel in the brain showed significant activation under at least one analysis pipeline. In other words, a sufficiently persistent researcher determined to find significant activation in virtually any brain region is quite likely to succeed. By the same token, no voxels were significantly activated across all pipelines. Thus, a researcher who hopes not to find any activation in a particular region (e.g., to rebut a competing hypothesis) can surely find a methodological strategy that will yield the desired null result … Selective analysis reporting may occur without the intention or even the awareness of the investigator. For example, if the results of a new experiment do not concord with prior studies, researchers may adjust analysis parameters until the ‘correct’ results are observed.”
In a multiple observer study published in 2020, the researchers had asked 70 independent teams to analyse the same dataset, testing the same 9 ex-ante hypotheses.74 The dataset included fMRI data from 108 individuals, each performing one of two versions of a task that was previously used to study decision-making under risk. The teams were asked whether each hypothesis was supported based on a whole-brain-corrected analysis (yes or no). On average across the 9 hypotheses, 20% of teams reported a result that differed from most teams, which was midway between complete consistency across teams and completely random results. This study demonstrated that analytical choices have a major effect on the reported results.
In 2021, researchers reported that after they cautioned in 2016 that there are so many sources or error in imaging studies that findings should not be considered definitive but only suggestive, 24 MRI studies had appeared in JAMA Psychiatry and 22 in the American Journal of Psychiatry describing differences in such scans in samples of psychiatric patients.75 All 46 studies concluded that their findings are evidence of changes in brain structure.
In 2022, other researchers used three of the largest neuroimaging datasets available including a total of around 50,000 individuals to quantify brain-wide association studies’ (BWAS) effect sizes and reproducibility as a function of sample size.76 The median sample size was only 23 people. The researchers found that BWAS reproducibility requires samples with thousands of people.
As a commentator wrote, the study showed that almost every person diagnosed with depression will have the same brain connectivity as someone without the diagnosis, and almost every person diagnosed with ADHD will have the same brain volume as someone without ADHD.77 Yet, in the small studies, correlations were almost always greater than 0.2 and sometimes much larger, which, as the researchers wrote, should not be believed.
The conventional method for dealing with this problem is to increase the threshold for statistical significance. However, this will backfire in these small MRI studies because it inadvertently ensures that only the largest—and thus the least likely to be true—brain differences end up passing the significance test and being published.
The experience of the Editor-in-Chief of Molecular Brain is also relevant to consider when assessing the merits of brain scanning studies in psychiatry. In 2020, he described what happened when he requested to see the raw data in 41 of the 180 manuscripts he had handled.78 Upon his requests, 21 of the 41 manuscripts were withdrawn by the authors, and he rejected a further 19 “because of insufficient raw data,” which suggested that the raw data might not exist, at least for some of the cases. Thus, only 1 of 41 papers (2%) passed his reasonable test.
Unfortunately, brain scan studies have a psychological component. People are more prone to believe what they do not understand, which means that the more the result is imbedded in unintelligible but seemingly advanced statistics, the more likely it is that the readers will believe it.
Researchers have coined the term “seductive allure of neuroscience explanations” (SANE), which is a real phenomenon. Several studies have shown that people show greater trust in studies with neuroscience language and graphs, especially if there are brain images.79,80
To see the list of all references cited, click here.
Mad in America hosts blogs by a diverse group of writers. These posts are designed to serve as a public forum for a discussion—broadly speaking—of psychiatry and its treatments. The opinions expressed are the writers’ own.
Mad in America has made some changes to the commenting process. You no longer need to login or create an account on our site to comment. The only information needed is your name, email and comment text. Comments made with an account prior to this change will remain visible on the site.