The FDA Allows Ineffective Devices to Come to Market

Six years ago, a new device hit the market: a small patch that kids could wear on their foreheads while sleeping to treat ADHD. Available by prescription, it produces pulses of electricity meant to stimulate the trigeminal nerve, which is connected to regions of the brain thought to be involved in ADHD. It’s marketed to parents and doctors as “FDA cleared” and “clinically proven.”

I’m generally skeptical of new treatments—in my tenure at Mad in America, I’ve seen psychiatry fail to live up to the promise of hyperbolic headlines too many times. So I wasn’t exactly surprised when a new study, with a larger sample and a sophisticated control group, found the device to be completely ineffective on more than a dozen measures, including ADHD symptoms as well as neurobiological effects.

Even so, when I first began working on this story, I hoped I might hear of science “self-correcting.” The point of repeating studies (“replicating” them, in science jargon) is that mistaken first impressions can be corrected by more rigorous studies. The scientific record can be fixed, and thus the public can proceed with a better understanding of what does and does not work.

What I found, though, is that the regulatory process doesn’t work that way. Instead, it seems designed to obfuscate whether the device actually works or not, with clearance determined instead mostly on whether the device is safe—and once something is cleared, it’s almost impossible to stop it from being sold to the public.

It turns out that there had only been one small randomized controlled trial of the device before this new one. Yet in 2019, the FDA gave manufacturer NeuroSigma clearance to begin marketing the device, Monarch eTNS, based in part on that study (which concluded the device was safe and effective). In 2024, the FDA cleared the second-generation version of the device based solely on its similarity to the first.

While the FDA does look at evidence of effectiveness when clearing new devices such as this, it doesn’t hold them to the same standard as it does for drugs, according to Kim Witczak, former consumer representative on the FDA Psychopharmacologic Drugs Advisory Committee. “The legal standard for devices is reasonable assurance of safety and effectiveness, which is a much lower and more flexible bar than what is required for medications. That distinction is often lost in public understanding,” Witczak said via email.

The result is that a medical device can receive FDA clearance mostly because it is deemed to be safe. With that clearance, the device can be marketed to treat a medical condition. And the public see that the device is being sold as “FDA cleared” to treat a specific illness like ADHD, and thus reasonably assume that the device actually works.

“Most physicians and patients have no idea this is how the system operates,” Witczak said.

She added, “It truly feels like the wild west. In the device pathway, the FDA is often deciding whether something is safe enough to try, not whether it has been proven to work.”

I’ve written about flaws in the FDA’s drug approval process before, but at least that process requires some specific evidence of efficacy. I expected device approval to be about the same. But as I investigated this story, I learned that the FDA’s clearance of devices doesn’t pass even that low bar.

In my search for answers, I sought out the lead investigator on the first Monarch eTNS study—the positive one that led to FDA clearance for the device. I spoke to James J. McGough, MD, professor of clinical psychiatry at the University of California, Los Angeles, via email. “Primarily, approval of devices deemed ‘non-significant risk’ is mostly based on safety,” he said. “My understanding is the FDA followed its usual regulations and procedure in granting approval.”

I also spoke to Katya Rubia, PhD, professor of cognitive neuroscience at King’s College London—the lead investigator on the new study that found the device ineffective. In a phone interview, she told me much the same: “The FDA attempts to approve devices based on safety, not efficacy. The main take-home message is that regulatory approval doesn’t mean a device works. It means it’s safe.”

In the case of TNS, “The evidence is pretty clear that it doesn’t work,” she said. Because of this, Rubia has written to the FDA, urging them to revoke the device’s clearance.

But that’s unlikely to happen, according to Erick Turner, MD, a former FDA reviewer and professor emeritus of psychiatry at Oregon Health and Science University, reached via email. “I harbor no illusions that the FDA would rescind approval,” he said. “This is not the first time the FDA has approved an intervention with questionable efficacy.”

Witczak agrees. “Once a device is cleared, the FDA does not typically revisit questions of effectiveness unless a safety issue emerges. Lack of effectiveness alone is rarely grounds for removing a device from the market.”

The Wild West of Device Regulation

The FDA regulatory process is a bit of a tangled web. The FDA’s procedures for approving devices are based on a three-tier system. Class I and II devices are considered low and moderate risk, respectively, and are usually “cleared” through the 510(k) pathway. (Class III devices are considered high risk, and have a higher bar to show that the benefits outweigh the risks; they are “approved” rather than “cleared” by the FDA.)

In order to use the 510(k) pathway, the manufacturer must show that the device is substantially similar to an existing device. If they can demonstrate this, they don’t have to provide any safety or efficacy data, and the device will be “cleared” for marketing. The assumption is that predicate device—the existing one that the new device is similar to—already provided enough evidence to the FDA.

Although Monarch eTNS is considered a Class II device, it was the first of its kind, and so it could not be approved through the 510(k) pathway. Instead, it went through the De Novo pathway. In the process, NeuroSigma did provide the FDA with data on efficacy, and so the FDA could be said to have considered the risks and benefits of this device.

The question raised by Rubia, however, is whether the data provided by NeuroSigma constituted enough evidence to demonstrate efficacy, or if the regulators acted too hastily in allowing the device to market.

Adding to that concern, according to Witczak, is that De Novo approvals like this set a precedent, making it easier for the next device to get clearance (through the 510(k) pathway) without having to show any evidence of efficacy.

“This means a device can be cleared based on limited early evidence, later shown not to work well, and still function as the regulatory foundation for additional products,” Witczak said.

“I am particularly concerned that the De Novo pathway creates new categories that can shape the market long after the initial clearance,” she added.

That is, even if future studies prove that Monarch eTNS doesn’t work, it will most likely remain on the market and serve as a pathway for further similar devices to be sold to families desperate for an ADHD treatment for their children.

The Monarch eTNS Studies

I was beginning to piece together the pattern: Monarch eTNS succeeded in a small initial study, which was enough to get clearance to market from the FDA, and the FDA was unlikely to revoke such clearance.

But now I was curious about how to reconcile the very different data from the two studies—that first study finding the device to be a powerful treatment for ADHD versus the new larger, more sophisticated study that contradicted that success, finding that the device doesn’t seem to work.¹

The first study—the one published by McGough—was a double-blind, randomized controlled trial in 62 children aged 8–12. McGough’s results showed that the group receiving the stimulation improved more than the control group, who received no stimulation; they also showed some neurobiological differences between the groups. After four weeks, about half of the children who received Monarch eTNS (52%) met criteria for “clinically meaningful improvement.” This is the study cited as evidence of the device’s success.

Rubia’s study was designed to replicate this finding in a larger sample (150 children). They included a wider age range (8–18 years old) and also included some kids taking stimulants (39.3% of the sample; none of the children in McGough’s study were on stimulants). Most importantly, the researchers used an active (“sham”) condition for the control group. While the intervention group received eight hours of regular stimulation overnight, the control group in Rubia’s study received 30 seconds of low-frequency stimulation every hour—just enough to so that the control group thought that they were also receiving the treatment.

The result? On more than a dozen measures of ADHD symptoms and neurobiological effects, there were no differences between the groups at four weeks. “TNS is a safe intervention,” the researchers concluded, “but does not demonstrate clinical efficacy for pediatric ADHD.”

 

Rubia was surprised at her own conclusion. “To be honest, we expected it to work. We were hoping to find an improvement,” she said.

They searched for an explanation for their negative result, wondering if the finding was being driven by older children or by children on medication (both differences from McGough’s study). But on all subgroup analyses, Rubia’s team concluded that there was no effect. Monarch eTNS just didn’t work.

According to Rubia, the only remaining explanation was that McGough’s study was flawed. And there was one obvious culprit: unblinding. In McGough’s study, the control group received no stimulation at all. This means that patients were probably able to determine whether they were receiving stimulation or not—and if that was the case, much of the improvement in the group receiving stimulation could be due to the placebo effect. “If you do this for four weeks, I think you would notice,” Rubia said.

“The placebo effect is much, much larger in neurotherapies,” Rubia told Mad in America. “We found in our studies, when we looked at the control group, the sham group, that the placebo effect is twice as large as for medication.”

McGough disagrees with her conclusion. “I think those authors were too quick to regard our results as a false positive due to insufficient blinding. We assessed blinding and the results held,” McGough told Mad in America. He added, “It is highly unlikely these are placebo effects.”

I was left with a contradiction. Rubia said that the first study didn’t account for unblinding, but McGough said they did. So what actually happened?

In truth, McGough’s study did assess blinding—but only at the one-week mark, when the study was just getting started. They didn’t assess blinding at the four-week mark when the final outcomes were measured.

I went to the FDA’s review of McGough’s study to see whether they noticed this potentially fatal flaw. It turns out that they too were unconvinced by the blinding and that they couldn’t assess whether the results were due to the placebo effect. They write of “the potential for unblinding subjects in the sham arm which may have resulted in patient bias.” They add that the study data “did not allow an assessment of the placebo response.”

As I reread the original study, I sensed another concern—conflicts of interest. Ian Cook, MD, listed on NeuroSigma’s website as their Chief Medical Officer, was an author on the initial study.

I asked McGough about Cook’s presence on the study. McGough argued that including an executive of the company marketing the device did not constitute a conflict of interest. “Dr. Cook was a faculty member at UCLA and had ended his involvement with NeuroSigma prior to the study’s publication,” he said. (Cook still appears as Chief Medical Officer on NeuroSigma’s website as of the writing of this article).

According to McGough, “He had provided some initial contributions to the concept of the study and study design, made several small contributions regarding technical aspects of TNS in the report, and did a final review of the manuscript. He played no role in participant recruitment, data collection, or data analysis. We acknowledged his relationship with NeuroSigma in the interest of full disclosure, but had absolutely no conflicts regarding study outcomes.”

Finally, I found that there is no real evidence for longer-term outcomes with the device, either. Rubia’s study included a six-month follow-up, but the children only used the device for four weeks; her six-month data did not show any differences between groups. McGough’s study included a 12-month open-label extension for kids who had already improved, but only three children completed it and there was no control group.

Ultimately, McGough told me he believes that Rubia’s negative results are an indicator that her study failed, not that the device didn’t work. “It is also important to keep in mind that many commonly used medications, in particular those used for hypertension, have had studies that fail to show differences from placebo. These studies are published much less frequently than those with positive results. However, these studies are not taken as evidence that a medication doesn’t work. They are merely considered to be failed trials. Given the differences in their protocol, it might well be that they simple conducted a failed trial.”

However, he said, “I agree that subsequent confirmation of our initial double-blind study is important.”

McGough is currently working on another study, using what he called “a more sophisticated sham” condition. He hopes that this study will provide more definitive information about Monarch eTNS. His belief is that neuroimaging will be able to predict which children will respond to the treatment.

In contrast, Rubia concluded—on the basis of some neurobiological data—that the device didn’t seem to be having any effect on the brain at all.

To Rubia, this means that the device didn’t work as advertised. Yet she still believes that ADHD is a neurobiological disorder and hopes that a future device will correct the underlying biology.

“The whole point of finding brain differences is to find biomarkers, which we can then target with treatment,” she said.

The Search for Biomarkers to Diagnose and Treat ADHD

Rubia is considered a pioneer in the use of fMRI to assess brain activity in kids with ADHD. Her research has been used to suggest that certain brain areas, such as the frontal regions and the basal ganglia, are under-activated and smaller in kids with ADHD.

But other researchers disagree. Despite Rubia’s claims, researchers have failed to identify biomarkers that can be used to distinguish kids with ADHD from those without, according to Sanne te Meerman, PhD, assistant professor at the University of Groningen.

“All we know is that there are many different factors that have a small relation to the behaviors that fall under the ADHD umbrella,” te Meerman told me via email. “The contextual ones have the strongest predictive value: more children from broken homes or divorced parents have a tendency to be more restless for which they may receive an ADHD classification. Or the youngest in the classroom… one of the strongest predictors of ADHD is birth-month. There are also biological factors but they are weaker. For instance, in those with ADHD their brains might mature a bit slower. But this is not a disorder in itself nor is it unique for those with ADHD.”

There is nothing wrong with searching for a biological cause. But the concern, according to te Meerman, is that these crumbs of neurobiology are promoted to the layperson and said to represent “science,” while the known, large impact of social and environmental factors is dismissed and ignored.

“Despite their small influence,” te Meerman said, “the biological factors are however strongly overrepresented in discourse.”

Rubia was one of the 82 authors on a controversial Lancet Psychiatry ADHD study led by Martine Hoogman, PhD, in 2017 that claimed to identify brain volume differences in children with ADHD. The researchers concluded that they had proven ADHD to be a biological disorder. “We confirm with high-powered analysis,” they wrote, “that patients with ADHD have altered brains; therefore ADHD is a disorder of the brain.”

Yet Hoogman’s conclusion was roundly critiqued by such notable figures as Allen Frances, MD, chair of the DSM-IV task force, and Keith Conners, PhD, who provides his name to a prominent diagnostic measure for ADHD. Conners was the first researcher to study dextroamphetamine (Dexedrine) and methylphenidate (Ritalin) to treat ADHD.

te Meerman was a co-author on a paper—with Frances and Conners, as well as Laura Batstra, PhD—that critiqued that 2017 brain differences study. “The most important argument against the authors’ conclusion that ‘patients with ADHD have altered brains’ is that it is not supported by their own findings,” they wrote.

This was especially startling coming from the two men who together could be considered largely responsible for the initial adoption and expansion of the ADHD diagnosis. “What makes Allen Frances and Keith Conners stand out is that they simply admitted their mistakes and, perhaps more importantly, made a real effort to correct those,” te Meerman said. “I am to this day deeply impressed and moved by that.”

The first problem the critics found was errors in Hoogman’s data; rows of data were switched and confidence intervals and p values were incorrect. Then, independent researchers accounted for IQ—a huge confounding factor in brain volume research—and the supposed brain difference disappeared. Researchers kept finding more and more critiques until Lancet Psychiatry ended up devoting a whole issue to articles debunking the original brain differences claim.

In a petition to Lancet Psychiatry to retract the study, Mad in America’s editors wrote: “In fact, the researchers found that the distribution of individual brain volumes in the ADHD and control groups almost entirely overlapped. This means that at an individual level, a person diagnosed with ADHD is almost as likely to have a brain volume above the normal “average” as to have a brain volume below that average. As such, the MRI data showed that brain volume size, on an individual level, is, in fact, not a distinguishing characteristic of ADHD.”

Lancet Psychiatry never retracted the study.

“The whole ordeal remains disgraceful to this very day,” said te Meerman. “A sincere apology and rewriting/withdrawal of the paper’s conclusion would be the only good solution.”

In fact, ADHD is one of the most contested diagnoses in modern psychiatry. To te Meerman, the lack of consistent, identifiable biological differences is no surprise, because the term ADHD does not refer to a discrete illness, but rather is a general label used to describe kids who have difficulties like sitting still in school.

“ADHD is just a name for a diverse group of people with overlapping behaviors and experiences,” te Meerman said.

“The word treatment implies ADHD is like an illness, but it’s not,” he added. “It is a name we can give (but that is not necessarily good practice or even helpful) to someone whose behavior we deem as restless or unruly… some people are more active by nature or have less talent for contemporary knowledge transfer in schools (reading books, listening and sitting still).”

So, if Monarch eTNS and other brain stimulation devices don’t work, how should ADHD be handled? After all, even if it’s not an identifiable “illness” per se, the kids labeled with this problem are certainly struggling.

According to Rubia, the solution is still stimulant drugs. “It’s still the best treatment,” Rubia said. Therapy and behavioral interventions haven’t shown consistent evidence of efficacy, she said, while exercise and meditation may help, but the effect size is small and they require lifestyle changes.

But Rubia and te Meerman both noted that parents don’t always want to use the drugs for their children—they impact children’s sleep, growth, and appetite, among other risks. Rubia recommends taking drug holidays and only using it when needed, to try to mitigate the adverse effects of the drugs.

Another problem, according to Rubia and te Meerman, is that there is no evidence of long-term efficacy. “Drugs like stimulants can help people temporarily to focus more but do not ‘treat’ ADHD,” te Meerman said. He added that the drugs don’t seem to help children’s grades improve—attention is improved, but it doesn’t seem to translate to learning.

If not drugs, what would be the best approach for a child struggling at school and diagnosed with ADHD? te Meerman recommends an individualized approach, asking the child to share what is making them struggle: the way the schoolwork is delivered, whether the child feels safe or is being bullied, whether there are problems at home, or whether the “underfunded schools with overworked teachers have difficulty accommodating” the child’s temperament.

“An ADHD classification mostly hides such contextual factors,” te Meerman said.

The goal, he said, would be to work with the child, together designing a solution that applies to the specific factors making them struggle.

For my part, this sounds like the principles on which therapy is founded, of working with a child to identify what’s going wrong and helping the child figure out how best to face or surmount those difficulties.

Indeed, this story—of regulatory approval of a device that hasn’t been shown to work—is really a chapter in a larger story. The device received approval based on the narrative presented by psychiatry’s disease model: that ADHD is a distinct disease, that it involves specific neurobiology, that most people with ADHD have a similar neurobiology, that a device can fix our brain patterns, and that this is the best way to help a child who’s struggling in school. Yet none of those assumptions have proven true.

And there is ongoing harm when the device fails to help: the child and parents are now left with the belief, which lies within the disease model, that ADHD is a permanent brain disorder that will resist treatment. That belief helps maintain the disease model narrative, and this story tells of an institutional failure to protect the child from such harm.