How Following the Trail of “Cutting Edge” and “Convenient” Can Distort Reality

James Schroeder, PhD
33
99

In the late 1990’s, the National Institute of Mental Health (NIMH) set out to provide the most extensive review ever conducted of the effectiveness of ADHD medications in children.  It was known as the Multisite Multimodal Treatment Study of Children with ADHD (MTA study).  In 1999, NIMH announced that after 14 months, well-constructed medication management programs provided better results than other treatments, including behavioral therapy.  But the study was not over, and the tables started to turn as detailed in the new book by Robert Whitaker and Lisa Cosgrove.  By the end of three years, medication not only provided no more benefits over other options, it actually predicted greater deterioration of symptoms.  In other words, children that were taking medications exhibited worse outcomes (e.g., more delinquency) than those who were not.  They also ended up six pounds lighter and an inch shorter than their unmedicated counterparts.  After six years, medication use was associated with increased hyperactivity, impulsivity, and oppositionality, and overall functional impairment.  Those who had used behavioral therapy alone were less likely to be depressed or anxious.

NIMH investigators were shocked.  Even after analyzing a number of potential confounding variables thought to contribute to the unexpected findings, they finally concluded, “The findings…were not consistent with views and expectations about medication effects held by many investigators and clinicians in the field.  That is, long-term benefits from consistent treatment were not documented…”

The investigators acknowledged that despite previously held beliefs, good science had demonstrated otherwise.  It seemed this large scale study would provide clear guidance for those using and prescribing drugs to kids once it was released to the public at large.  That never happened.  Multiple papers and titles did not report the negative findings, instead highlighting the short-term benefits while minimizing the negative outcomes.  The NIMH press release announced the great news, “Improvement Following ADHD Treatment Sustained in Most Children” although they left out that “sustained” only meant through 14 months.  Parents were assured by NIMH that despite having no clear gains after three years, medications were the way to go, even for the long-term.  Discussion of the negative results was very difficult to find.  In 2013, the American Psychiatric Association (APA) and the American Academy of Child & Adolescent Psychiatry (AACAP) published the ADHD Parents Medication Guide that continued to espouse the positive effects of medication found in the MTA study.  And yet, despite the assurance given to move forward with medication treatment, other large scale studies have continued to find disappointing long-term effects .

On August 17, 2015, Medscape published an article entitled, Apps Can Help Health Habits, But Proof Is Poor:  AHA Statement.  Similar to the publicizing of the MTA study, it was filled with confusing, and at times, contradictory statements and advice.  The article acknowledged that “evidence… [for mobile applications in reducing cardiovascular risk] is generally lacking.”  Nevertheless, the article stated “clinicians should still encourage patients to use these new tools…”  After review of 69 studies that failed to show clear effectiveness, the authors continue to repeat the same mantra:  “Clinicians shouldn’t ignore the emergence of mobile health technologies, despite the lack of data supporting their use” ; health care providers should acknowledge that “evidence is currently weak”, but “patients should be encouraged to use them…”  Most striking, the lead author, Dr. Lora Burke, stressed that despite the lack of evidence, “we need to embrace the challenge of producing (italics added) this needed evidence on how effective these new technologies are and how we can best adopt them in our practice to promote better health.”  Ironically, further gleaning from the actual 58 page report indicated that weight loss is largely dependent on participant engagement, and “declining engagement and attrition (often as high as 40-50%) are characteristics of the digital health interventions” (Pg. 11).

So let’s take a second to review.  Researchers acknowledge that the evidence for using apps to promote cardiac health is weak at best, and nil or detrimental at worst.  Yet, not only does the press release indicate that patients should be encouraged to use them regularly (even before research supports it), the authors have the audacity to say that we should work to “produce” research (e.g., do whatever it takes in regards to data or study manipulation) to support the conclusions that these and other authors desire and/or believe, even if the best science does not come to this conclusion.   This is very similar to critiques that many have lobbied against pharmaceutical companies, in that through use of questionable statistical techniques and suppression of negative studies, findings to support use of medications are often artificially “produced.”

Let’s use an example to illustrate how contrary this is to what we are taught about the scientific process.  If a relatively non-marketable entity, such as exercise, consumption of produce, journaling, or regular exposure to nature, was repeatedly found to not sustain long-term positive effects for a particular health outcome, it is safe to say that clinicians and researchers would not be consistently telling their patients to engage in these activities in the hopes (or desire) that they might just work.  Further research might still be warranted; however, after almost 70 studies, there is a decent chance that interest and funding may have “dried up.”  But at the very least, you can rest assured that doctors would not be advising children to eat vegetables or exercise 60 minutes a day if it didn’t help promote sustained health.  So why would the same professionals promote medications and technology without clear long-term benefits, and even with potential for harm?

In some ways, the answer is very simple.  Not only are technology and medications readily marketable (and not available without cost or patent restrictions), many involved have a vested  interest (often financially) in promoting them.  They also both follow the coveted pathway of a convenient and “cutting edge” mindset.  Whether we admit it or not, we are a culture that seeks out the quickest, seemingly most innovative solutions to time-honored problems, even when clear, direct, low-cost options are readily accessible.  Let’s admit it.  All of us at some time want a drive thru solution to a problem that really requires a four course, sit-down meal.  We want to believe that a quick fix exists.  We want to be assured that the easier and more progressive path is better, or at least just as good.  But along the way, we (and many experts) fall trap to beliefs that end up distorting the reality in ways that can really lead to harm, whether of a physical, psychological, social, or financial sort.

So, what does this mean for us as parents and professionals?  Well, first of all, it means that we really need to be careful readers of what we consume, and when we encounter contradictory messages, we should investigate further to understand the contradictions, not simply follow the ultimate advice given.  Secondly, when it comes to spending our time and money, we should invest it where we know it matters, for the long-term.  There are a number of interventions that do make a difference with regard to attention and health.  Although they can be challenging and require sustained effort, there is little doubt that they can make a tremendous difference if we work to utilize these practices over time.  Finally, we should be very careful about assuming that chic and cutting edge treatments are necessarily the best option.  Often times, it is only depicted that way because what truly is the best option (e.g., adjust sleep routines, getting together with a friend to run twice a week, reducing processed foods, limiting screen time) is difficult to market and impossible to restrict.

By the way, there was one more piece of information from the 58 page report that got left out of the Medscape article.  It was quoted as follows:

“Physical activity in the United States has significantly declined over the past 2 decades.  Since the late 1980s, the proportion of adult women who report no leisure-time activity has increased from 19.1% to 51.7%, and the proportion of adult men reporting no leisure-time activity rose from 11.4% to 43.5%.” (Pg 12)

I wonder what cultural shift could have been associated with this concerning trend?  Maybe the American Heart Association needs to consider that before recommending people spend more time utilizing technological innovations—which lack clear, supporting evidence, they might think about putting forth more effort to facilitate ways to help people reduce the use of unnecessary devices that could be the problem in the first place.

33 COMMENTS

  1. Good article James. It reads a bit like a crime novel.

    Reading this reminds me of how some researchers assert that medication should be a regular part of the treatment for “Borderline Personality Disorder”, based on the same type of short-term symptom suppression as described above for ADHD.

    I have spoken to dozens of “sufferers” and family members about seeking help for problems related to the BPD label, and I usually encourage them not to see a psychiatrist. This is not because I think all medication or psychiatrists are bad, but rather because over-medication and medicating for too long are likelier than not to occur. Unless the person is in danger, I encourage seeing a non-prescribing psychotherapist and to seek peer support.

    It is ironic that while it can be a legal risk to advise people not to see a psychiatrist, many psychiatrists are legally enabled to misrepresent what will help people with impunity.

      • “I always tell people to avoid psychiatry altogether. I can’t in good conscience look at people running the risk of being tortured and murdered.”

        Me too.

        Recommending anyone see a psychiatrist is just too dangerous, as people are not warned that they could lose all their human and legal rights simply by entering treatment.

        Every psychiatrist’s office should be forced to prominently display warnings and also to have patients sign informed consent documents acknowledging that they understand the basis of this medical specialty is NOT based in any science whatsoever, and that they are aware of the broader legal and social risks of handing power over their lives and wellbeing to a psychiatrist. This should be repeated every time a new prescription is written, but with the actual clinical trial results explained to the patient.

        That said, this is another interesting and useful article that highlights the influence commercial interests have over “research” and the broader medical profession’s recommendations.

        Too many doctors are now marketeers and “spin doctors” rather than practitioners of health care.

  2. “For the love of money is the root of all kinds of evil,” the real Bible explains the rational behind / for the ADHD espousing scientifically invalid DSM “bible.” And I have lots of DSM “mental illness” believing in-laws who wear those $100 plastic bracelets, and play with those apps. These baby boomer in-laws also partook in the short run for-profit only creation of the banking industry’s housing bubble, and other short run for-profit only banking industry crimes, like creating the derivatives markets, that will ultimately likely result in the total financial collapse of our monetary system.

    “For the love of money is the root of all kinds of evil,” is the problem behind all of today’s “psychopathic” corporations / industries – from the banking industry to the medical / pharmacutical industries. Thomas Jefferson even forewarned us of the perils of allowing these evil banks and corporations to take control of our society long ago.

    As one who prefers the four course healthy meals, to the fast food fix, and even raised two teenagers who refuse to eat McDonalds. But, in part because of my love of fine dining, refused to give up my hour a day of exercise, even after it was claimed to be a “sign of mania” by a psychiatrist. And, as one who healed myself from one of the psychiatric industry’s partially or completely (as in my case) iatrogenic DSM “major mental disorders” – with regular moderate exercise, proper diet, lots of time spent gardening in my little neck of the woods, journalling, and numerous other creative outlets. I completely agree, the quick fixes, due to worship of short run profits only, are absolutely the “root” of the “evil” behind the banks and all corporations / industries that currently control our society today.

    I hope and pray we some day return to a society where treating others as you’d personally like to be treated, once again becomes part of the wisdom of our society. Allowing the psychopathic, short-run-profit-only minded corporations and banks to rule our society is staggeringly unwise for all of humanity. Thomas Jefferson, and Jesus, were correct. Truly, it’s once again time to throw the money changers, and the “corporations that will grow up around them,” out of the temple – that is all of humanity.

  3. James:

    Good job of connecting the dots. I have long suspected that the issues such as biased research or deceptive marketing practices of Big Pharma are not limited to the field of mental health but rather, they exist in all branches of medicine. Your example about the technological gimmick for heart disease is perfect. The big difference is that people with heart disease are not involuntarily medicated with stations if they refuse treatment or wish to seek alternatives.

  4. It’s funny (well… not really) but many studies, including those reported and reviewed on MIA, include such statements. Or at best weird justifications: “we didn’t find that the drug A is effective against it but that is likely because our study has low statistical power or the moon cycle was in the wrong phase. We are sure that if we spend more time and money drug A will prove to be the miracle drug treating mental illness and solving global warming all the time while curing Africa’s hunger problem”.

  5. James, this was quite an interesting article, thanks for bringing this to my attention as I had not been previously aware of this study.

    On a minor note, I’m not entirely sure I agree that the finding that “this large scale study would provide clear guidance for those using and prescribing drugs to kids” is quite as clearcut as described in your summary. Notably

    1) We know that the 14 week time point was the primary endpoint of the trial. If the results had been positive at 36 months and negative at 14 months, I think we would see much criticism of the study authors here for ignoring the primary endpoint (14 months) and focusing on a secondary endpoint (3o months). Surely if focusing on a secondary endpoint that to obtain a more positive result is unacceptable, doing so when the secondary point is more negative is bad too.

    2) The results post-14 months are intrinsically less reliable because the kids did not stick with their assigned treatments. While the authors performed a post hoc analysis of the level of medication and adherence to assigned treatment in the post 14 month period and found “no effect of medication adherence”, once the kids moved into “regular community treatment” and adherence to the assigned treatment became optional, randomization was lost. Put another way, the kids whose parents decided to continue with intensive medical treatment might as a group differ from the kids whose parents stopped treatment.

    So while I think the 36 month data casts some doubt on the 14 month results, I don’t think they definitively contradict them. BTW, I thought this was a very interesting article, thank you for writing it.

    • Your point seems valid and is well taken, John. I only saw it after posting my comment below. Yes, from a strict scientific point of view, the “study” was not maintained as such after the 14-month period for the reasons you mention; therefore, the post-14-month follow up findings are mitigated. However, I believe the follow up strongly suggests the initial findings may be quite flawed. This seems consistent with short-term versus long-term research on other “psychiatric conditions” (none of which have any conclusive validity as “brain diseases”, to my knowledge), and the strong tendency in mainstream psychiatry to trumpet and base treatment standards on these short-term studies while ignoring or even denying the findings of long-term studies.

      • Sorry, likewise I responded to your comment below without seeing this one.

        Why do you think the original study design was flawed.

        I don’t know that ADD is a “brain disease”. I think that for most aspect of human behavior and cognition there is a range in which different people fall. If some people are at the extremes of the range, they may find it difficult to function. For example, I used to worry about random stuff so much it made it difficult to function. I took amitriptyline and it made it easier not to do that. When I got older it wasn’t such a problem, and I stopped. For me it worked well, for others it has not.

        I think “diagnoses” are fine as long as they are used as rough guides as to what helps some people with certain characteristics. When they are used to stigmatize, coerce, or fail to recognize individuality and freedom it becomes a problem. JMHO.

        • I like your understanding of how medication is best used – sounds like you and Joanna Moncrief would agree on a lot.

          I understand your point about the loss of randomization, though of course the real world works much more like the post-study period of time and less like the experimental conditions. Still, I would not consider this definitive, but there is a lot of other data published since that supports it. The Raine study was referenced on one link in the article, and it drew similar conclusions. The Montreal ADHD study also found that kids who stayed on medication over time did the same or worse on major outcome measures. There was a recent study comparing Finnish children and US children, where the rates of ADHD dignosis were similar, but stimulant treatment was much more common in the USA, and yet treatment outcomes were no worse and maybe even a bit better for the Finns. And then there was the OSU medication effectiveness study done somewhere in the early 2000s, where they found that no major outcome measure, social, educational, or emotional, was positively impacted by stimulant use, after looking at basically every study ever done on the subject that they could dredge up. So I think the jury is in on this one – stimulants do NOT improve long-term outcomes for kids. And it is to me disturbing that despite what is now a fairly solid set of literature data showing this to be the case, no one, and I mean basically no one, in the mainstream discusses this fact, and they still tell parents willy-nilly that “untreated ADHD” leads to delinquency, school dropout, drug use, teen pregnancy, and low self-esteem, without telling them that on average, stimulant treatment improves NONE of these measures over time.

          Psychiatrists have been trying to present themselves as being “scientific” by focusing on the brain and chemicals. They may appear “scientific” to the lay public, but if they were real scientists of any sort, they’d be willing to admit when their primary hypothesis is not supported by the data.

          —- Steve

          • Steve, I have not followed the ADHD area particularly closely, so I do not have an informed opinion about the overall evidence base.

            With respect to science and the brain: I agree, its a very complex organ and our understanding is very rudimentary. We have learned a few things: dopamine antagonists acutely reduce symptoms of psychosis; dopamine agonists are useful for controlling the symptoms of parkinson’s disease; acetylcholinesterase inhibitors can temporarily turn back the clock by a few months in people with Alzheimer’s disease; allosteric GABA agonists are useful for controlling seizures; and so on.

            I think a lot of the problems come in from efforts to force fit a single approach to every individual, and when drugs are used because they are (ironically) cheaper than more humanitarian, individually tailored approaches. I saw a lot of this in the foster care system.

          • Agree completely. The big error is believing that what is now defined as “mental illness” represents distinct entities that require distinct treatments, rather than suggesting the need for a thorough search for what is causing these manifestations to occur. Which requires humility and respect for the client and a willingness to recognize the complexity of both the brain and of human behavior. As in general health, drugs can be very useful in acute situations, but long-term use almost always becomes problematic over time, even with the most beneficial drugs. But of course, it’s hard to make massive profits on psychosocial interventions, and as long as capitalism drives medicine, we’ll get this kind of profit-driven oversimplification rather than a more considered look at the big picture.

          • John,

            I agree, the brain is “a very complex organ and our understanding is very rudimentary.” I don’t agree, however, across the board with, “We have learned a few things: dopamine antagonists acutely reduce the symptoms of psychosis.”

            I have corresponded with some who have found the neuroleptics (dopamine antagonists) do help with “psychosis.” However, when a person is misdiagnosed (according to the DSM-IV-TR, but not the DSM5) with “psychosis,” due to antidepressant withdrawal symptoms or ADRs, the addition of a neuroleptics can in fact cause a first ever “psychosis.”

            But my medical research has led me to conclude this “psychosis” inducing effect is possibly due to the antidepressant and antipsychotic drug interactions causing anticholinergic toxidrome, which is a known drug interaction that is known to cause “psychosis.”

            My point is, I’m quite certain the medical community is way to certain about their belief the neuroleptics cure “psychosis.” Since, especially when inappropriately used to “cure” the adverse effects of the antidepressants, the addition of a neuroleptic (even at a child’s dose in my case) it can in fact cause a first ever actual “psychosis.”

        • John, I didn’t say the original design study was flawed, although it might be (consider there was no true control group; i.e., a group for whom there was NO treatment, which would control for the placebo or Hawthorne effect…but I realize there may be ethical concerns in offering participants “no treatment”). What I meant was that the conclusions drawn by the researchers after the 14-month study appear flawed (overly benign and lacking in cautionary information regarding the intensive medication group), at least in light of subsequent follow up information. I believe if researchers and mental health practitioners had the well-being of children at heart first and foremost, they would make a strong effort to counter or at least question the initial finding that appeared so favorable to intensive medication treatment for ADHD. I see no such strong cautionary statements; in fact, as has been stated, they appear to have “spun” the latter findings in a way that tends to support the initial “drug-friendly” conclusions.

          Your point about your own experience is very well taken, and as Steve said in his reply, it brings to mind critical psychiatrist Joanna Moncrieff’s recommendations for a “drug-centered” rather than “disease-centered” approach to treating mental and emotional symptoms. Unfortunately, owing to the predominance of the current disease-centered approach, the concept of “informed consent” is regularly violated (in my opinion) and many people (including kids) do not have the option of discontinuing a drug when its harmful effects outweigh its helpful ones (if such effects were ever even present).

    • The primary endpoint may have been 14 months, but the study authors chose to follow up and what they found was disconcerting. Of course the results of the follow up should have been emphasized. Let’s say a follow up study of a drug found an increased number of heart attacks in those who took the study medication, certainly this should not only be reported but publicized. It is true that the children who stopped treatment may have differed from those who did not. But according to Whitaker and Cosgrove, at the end of 3 years the researchers found that if anything the off-med group had more severe symptoms early in the trial. My own personal feeling is these facts cast not some doubt but severe doubt on the benefit of ADHD meds for most children.

      • Well, just as a thought experiment, lets take a hypothetical drug intended to prevent heart attacks.

        The trial begins as a randomized double-blind trial with the primary endpoint of overall mortality at 2 years. It enrolls a population with known heart disease, of which 50% in each arm (drug, placebo) have had a previous heart attack. After 2 years 1% of those in the drug arm and 1.5% of those in the placebo arm have died, with almost all of the deaths being among those who had a previous heart attack.

        The subjects from the drug treatment arm are then rolled over into an open label follow up study of 5 years duration. They are neither encouraged to nor discouraged from continuing to take the study medication. 80% of the sicker patients (those with a prior heart attack) decide to continue on drug, but only 20% of the healthier patients do as they are less worried about their hearts.

        At 5 years follow-up, we compare mortality between the following two groups:

        1) A group in which 80% of the subjects are high risk and have been taking drug
        2) A group in which 80% are low risk and have not been taking drug.

        Group 1 has higher mortality. How do you interpret that result?

        • Sorry, I missed your point earlier about the severity analysis. Here is what the original authors had to say:

          “…we addressed in more depth the hypothesis that selection biases might account for the loss of relative superiority of medication. If the most severe cases, expected to have poorer outcome than the less severe cases, were preferentially treated with medication at the 36-month assessment, then this factor may account for its apparent loss of efficacy. The statistical method selected to evaluate this hypothesis was the propensity score analysis. This propensity analysis relies on the assumption that selection biases can be modeled as a simple linear combination of multiple, complex variables, such as ethnicity, previous experience with medication, treatment response, SES, and other variables, which may or may not be correct.

          Its interesting that this attempt to retrospectively compensate for absence of randomization did not find an explanatory effect, but like all such efforts, it depends on whether the authors correctly captured all the relevant variables. As such, it is a pale substitute for an RCT.

          • Your quote from the original authors says they don’t know why the ADHD medication stopped working. But it did stop. Other studies have also cast doubt on the efficacy of ADHD drugs. If I were a parent I’d be very reluctant to have my child take an ADHD med given the side effects of the drugs, the tremendous push by the pharmaceutical industry to promote the drugs, and the continuing questions about how well they work. An RCT wouldn’t necessarily be definitive since those kind of studies can be (and have been) manipulated.

            As an aside, I’m also very suspicious of the concept of ADHD. A psychologist once diagnosed my 10 year old grandson with this supposed condition. When it was pointed out that the child could concentrate on reading books he liked for hours at a time, she said that was only because he was interested in the books. If lack of interest makes it hard to concentate, then maybe almost all of us have ADHD.

          • Marie, that’s not what it says.

            For there to be proof that “the ADHD medication stopped working”, you would need to see a decline in the difference between a group treated with drug and a group treated with placebo over time. And it would have to be the same groups, not two groups whose composition varied over time in a completely unknown manner.

            What the quote says is that they attempted to mathematically adjust the results to compensate for the changing composition of the groups over time but that the “correction” applied may itself be incorrect, thus rendering the entire body of conclusions for what happened after month 14 suspect.

          • John, the researchers state that the drug apparently “lost efficacy.” They make an assumption that the most severe cases with expected worse outcomes took more medication, thus explaining the poor results. But they say their statistical analysis may not have been correct. I don’t see this as meaning that the whole body of conclusions as to what happened after month 14 is suspect, just that what they were trying to prove (the worst cases took more medication) is suspect. Maybe I’m wrong, but that’s the way I read it.
            My understanding is that the composition of the groups did not vary in a completely unknown manner. If children taking the drug at year 3 were doing worse than they were at month 14 and worse than children no longer taking the medication, you might question if the medication stopped working. This may not be definitive, but it’s certainly something to be explored further. And it casts doubt on the efficacy of the medication. Also, RCTs are not the only things that can be used to question the efficacy of a drug.

          • The other problem with this whole study is that ADHD doesn’t exist. It’s just a vague bunch of problematic childhood behaviors, which any kid can have to different degrees under stress, lumped together as if it were an illness. There’s no common biological nor psychological etiology behind the false label Adhd.

            If this study were repeated, the chances it would replicate are extremely low. It’s funny to see arguments over a study based on myth.

        • I interpret them as uninformative for many reasons. First of all the assumption that higher risk groups are more likely to stay on the drug is not necessarily true (especially when we’re talking about “ADHD” and not real illness). Secondly, the point of evaluating people in a follow-up without asking them if they continued on the drug or not sounds idiotic. Even if some of them lie not asking is just inexplicable.
          Btw, RCT for a psychoactive substance is a joke to begin with. Like someone can’t say they’re taking meth or cocaine:
          http://explainxkcd.com/wiki/index.php/1462:_Blind_Trials
          That will work well.

  6. Regarding the MTA study on ADHD treatments, this is what I found on the NIMH (who funded the study) website, under “Questions and Answers”:

    “Because their treatment after the end of the study was not controlled, it is not possible to draw accurate conclusions about the effectiveness of interventions beyond 14 months, or determine if treatment improves long-term functioning. However, the observations collected from these uncontrolled follow-up assessments can provide information about the long-term course of ADHD itself. These data are being analyzed and reported as they become available.3”

    Is this why they trumpeted the initial 14-month results (most favorable to intensive medication treatment) but have muted subsequent results that have shown either no difference between treatment groups or worse long-term results for those in the more intensive medication treatment group?

    No matter how you explain this, it seems to be a prime example of how money and guild interests have come to trump honesty and commitment first and foremost to the patient in health care–especially in psychiatry. It is particularly egregious when children’s brains are increasingly put at risk from this dishonesty!

  7. I don’t know that anybody “muted” it or how it says anything about “money and guild interests”. Fourteen months was the primary endpoint of the trial, and randomization was lost at all points after 14 months. At that point its basically a cohort study with potential for all sorts of confounding factors. For example, the parents of kids with especially severe ADD were probably more likely to keep them on the rigorous medication schedule that worked for the first 14 months than those with kids with milder symptoms. There’s no way to control for that simply by looking at the test scores of kids who continued to adhere to the regimen.