More than forty thousand papers have been published using functional magnetic resonance imaging (fMRI) technology to explore the brain. A new analysis of the common methods used in these studies is calling the entire field into question, however. The new study, published open-access in the Proceedings of the National Academy of Sciences, suggests that the methods used in fMRI research can create the illusion of brain activity where there is none—up to 70% of the time.
Functional magnetic resonance imaging (fMRI) has become an increasingly popular form of research in neuroscience, psychiatry, and psychology over the past twenty-five years. But, as Anders Eklund, the Swedish researcher who discovered the flaw in the fMRI software, points out, “surprisingly, its most common statistical methods have not been validated using real data.”
Eklund, along with his colleagues in Sweden and the UK, Thomas Nichols, and Hans Knutsson, investigated the software programs commonly used to analyze fMRI data, and they found that the assumptions made by these programs lead to a high degree of false positives, up to 70% compared to the expected 5%. False positives are significant as they can make it seem that a particular area of the brain is “lighting up” in response to stimuli, when in fact, nothing of the sort is occurring.
In 2009, neuroscientists at Dartmouth University demonstrated the false-positive effect when they placed a dead Atlantic salmon into the machine, and “showed it a series of photographs depicting human individuals in social situations.” The data produced by the fMRI made it appear as though “a dead salmon perceiving humans can tell their emotional state.”
Over the past year, there have been several attempts from brain imaging researchers to direct the field toward addressing the reproducibility and reliability problems in fMRI studies and come up with reforms. In January, a review article in the American Journal of Psychiatry worried about how unreliable fMRI studies were being translated into psychiatric practice.
The report’s authors, Daniel Weinberger and Eugenia Radulescu from John Hopkins University, who have published fMRI-based research themselves, warned that these studies “pose a serious risk of misinforming our colleagues and our patients.”
In their self-styled “cautionary note,” they encouraged the entire academic and medical field built up around fMRI research to face what they called a “mostly ignored ‘inconvenient truth.” “That conventional MRI does not allow us to make firm inferences about the primary biology of mental disorders and that we need to acknowledge this as a starting point in realizing the full value of MRI studies in psychiatry.”
Russ Poldrack, the director of the Center for Reproducible Neuroscience, having acknowledged the effects of improper data analysis in neuroscience, earned a grant from the Laura and John Arnold Foundation last year to develop a platform for sharing research data and analysis tools.
“Our hope,” Poldrack said in an interview with Vox, “is also that these free, powerful, and innovative computing tools will be an incentive that can get people to share their data so that others can try to reproduce their results or use the data to ask different questions than the original investigators may have been interested in.”
Eklund, A., Nichols, T.E. and Knutsson, H., 2016. Cluster failure: Why fMRI inferences for spatial extent have inflated false-positive rates. Proceedings of the National Academy of Sciences, p.201602413. (Full Text)