What is the Evidence for Empirically Supported Treatments in Psychology?

New meta-scientific review questions the evidence for the gold standard psychotherapies and empirically supported treatments.

Ayurdhi Dhar, PhD
11
4552

A new meta-scientific review published in a special edition of the Journal of Abnormal Psychology evaluates the evidence behind empirically supported treatments (ESTs) for mental disorders. The authors report finding mixed evidential support, which raises concerns regarding the efficacy of several forms of psychotherapy.

Led by John Kitchener Sakaluk of the University of Victoria and Alexander Williams of the University of Kansas, the researchers scrutinize the statistical evidence base underlying ESTs. They report that most ESTs have low power and replicability, with some of them demonstrating consistently poor performance across various metrics.

“Our analyses indicated that power and replicability estimates were concerningly low across almost all ESTs, and individually, some ESTs scored poorly across multiple metrics, with Strong ESTs failing to continuously outperform their Modest counterparts,” they write.

Creative Commons

Recently, researchers have documented the replication crisis in psychology, bringing increased scrutiny to psychological research. Researchers have cast doubts on the reliability of classic experiments, which form the foundation of the discipline. Additionally, other studies have questioned the efficacy of psychiatric medications like antidepressants and neuroleptics. Over the past decade, the mental health field has increasingly faced questions about its conventional treatments and outcomes, as service users and others have cast aspersions on its ethical standing and its scientific status. These concerns are reflected in the new efforts being made to improve the autonomy of patients in mental health care.

Psychotherapy itself lacks internal consensus with clinicians practicing over 500 different forms of therapy. Newer studies have shown that psychotherapy is less effective for those who are poor, have minority status, or are on antidepressants. Researchers have also questioned the ethical foundations and priorities of psychotherapy, with some critics wondering if it can ever escape its neoliberal roots whereby people are treated as entrepreneurial agents.

The authors of this review write that while questions about the efficacy of psychotherapy are of great significance, controlled trials that test this efficacy only started in the 1970s. In 1995, the American Psychological Association’s (APA) Division 12 Taskforce took the responsibility of creating and updating a list of therapies (now known as ESTs). The list was based on repeated controlled trials that were presumed to produce strong evidential support for that treatment. A few examples of some of these empirically supported treatments (ESTs) are Cognitive Adaptation Training for Schizophrenia, Interpersonal Psychotherapy for Bulimia Nervosa, and Psychological Debriefing for Post-Traumatic Stress Disorder, etc. EST approaches are often considered the gold standard for psychotherapy.

Traditional criteria for evaluating the efficacy of psychotherapies depends largely on statistical significance. Therapies that consistently show statistical significance over placebo, no treatment, or another treatment are classified by the APA as having a ‘Strong’ evidence base. Those classified as ‘Moderate’ have shown statistical significance at least once, and the ones considered ‘Controversial’ have demonstrated inconsistent results.

The authors contend that using statistical significance to measure the value of evidence can be problematic. They point to the controversy surrounding psychology’s replication crisis and suggest that the misuse of and misunderstandings surrounding null hypothesis significance testing has led to many of these problems. They state that almost 50% of psychological research contains at least one reporting error, while around 13% have a “gross” reporting error.

Other researchers have raised similar concerns citing that statistical significance does not always translate to clinical significance. This crisis is not restricted to experimental or social psychology but is also influencing clinical literature in medicine and psychiatry.

Additionally, the power of the tests that are used for statistical analysis is also under question. Power, which refers to whether a test can detect an effect when that effect actually exists, has been recommended to be at 80%. Still, reviews report that average power in social science studies is around 44%. As a result, the authors suggest numerous other metrics that cumulatively might create a more rigorous evidential base for ESTs.

This meta-scientific review investigates the evidential value of research that informs 79 different ESTs listed by APA’s Divisions 12. The primary concerns were to examine the evidential value underlying ESTs, mainly focusing on the comparison between the Strong and Moderate ones, and lastly, looking for improvements in the standard for evidential support for ESTs over time.

The authors found that a high number of statistical analyses were not fully reported by researchers making it challenging to re-analyze and verify the data they provided. They also found a lack of clarity in which tests the researchers considered to be fundamental or focal to determine the efficacy of an EST. Most importantly, they found inconsistencies in how well different ESTs performed on various metrics. They write:

“A small number of ESTs (e.g., both Cognitive Processing Therapy and Prolonged Exposure for PTSD) scored consistently well across all or most metrics, whereas a larger number of ESTs—including a number classified as Strong (e.g., Behavioral Activation for Depression, Cognitive Remediation for Schizophrenia, Dialectical Behavior Therapy for Borderline Personality Disorder)—performed relatively poorly across most or all of our metrics of evidential value. Low reporting quality in articles for other ESTs (e.g., Cognitive Behavioral Therapy for Insomnia, Family-Based Treatment for Bulimia Nervosa) made it impossible to calculate many metrics.”

The researchers did find that ESTs classified as Strong or Moderate had fewer reporting errors, and that methodological quality-testing efficacy has shown some improvement over time. ESTs like Exposure Therapy for Specific phobias showed strong efficacy across different metrics, while others like Family Psychoeducation for Schizophrenia consistently fared poorly.

They conclude that numerous problems, like unclear and incomplete reporting of statistical data, primary hypothesis, and focal tests by researchers, was a problem across the board. These studies also did not have 80% power (although improvements in power were observed over time) and “yielded replicability estimates that fell below what is currently normative for clinical research.”

The researchers further note that their analysis allows clinicians to quantify and observe the underlying support for different ESTs, which can have important clinical implications. At the same time, they state that answers about which EST has a stronger evidential base are still not clear or straightforward:

“Based on the available evidence, we don’t know if there are differences in the level of empirical support for ESTs, and we don’t know if ESTs offer benefit beyond that of other bona fide psychotherapies in treating patients with specific diagnoses.”

They suggest that both clinicians and patients should be open to changing the type of therapy in the face of negative or non-existent therapeutic results. This is especially critical now that one cannot merely trust an EST to be founded on substantial evidence.

While the statistical evidence behind ESTs is mixed, and there are many fractures in therapeutic schools of thought, studies have shown the importance of empathy in the success of psychotherapy. Similarly, research shows that common factors like the client’s perception of the therapist’s openness and authenticity are of greater significance than using a particular modality.

Given that psychotherapy is usually underfunded and has limited support from health insurance companies, these findings regarding their efficacy are especially significant for those considering treatment options.

 

****

Sakaluk, J.K., Williams, A.J., Kilshaw, R.E., & Rhyner, K.T. (2019). Evaluating the Evidential Value of Empirically Supported Psychological Treatments (ESTs): A Meta-Scientific Review. Journal of Abnormal Psychology, 128 (6), 500-309. (Link)

11 COMMENTS

  1. “Similarly, research shows that common factors like the client’s perception of the therapist’s openness and authenticity are of greater significance than using a particular modality.“

    It’s not a huge leap to understand why seeing someone who validates you as a person instead of someone who is clearly judging you would produce a better outcome at the end of “treatment”. Therapists who view their clients as disordered are more likely to think they know what’s “wrong” with the client and what the client needs to do to “fix” their problems, or believe a particular manualized therapy is the answer. Therapists who view their clients behaviors as ADAPTIVE are more likely to listen to what the client thinks their issues and needs are and allow the client to drive the therapeutic process.

    One is an expert-provider-driven process (One much like medicine has become with matching symptoms to treatments instead of causes) and the other is collaborative and cooperative. The problem is that collaboration doesn’t lend itself well to rigid manualized therapies. It requires a willingness on the part of the provider to view the client as the expert on their own lives and individual problems. It requires a humility that the provider is performing a service at the request and discretion of the client. And it requires knowing who the client really is. The client in many cases not being the person being treated but rather their parents (in the case of minors) or the institution they’re in (hospital, group home, outpatient service, prison), foster care, school, etc.

    In a lot of cases, the providers should really step back and ask, who am I really treating and who is this narrative of disorder actually serving? Therapists, unfortunately, do a great deal of damage by not actually serving the people sitting across from them.

  2. Thank you for the article Ayurdhi.
    From my own experience, it seems to me that failure to “treat” people successfully lies within the view that there is something or someone to treat.
    It starts with the individual and ends there. They are or have the problem.
    The “problems” are way beyond the client sitting on the chair, and even if problems can be identified, the person is not usually in control of others/situations/society within their life, whether in the present or situations from the past buried within their narrative.
    Even if we idealize that the person should work towards being responsible and change their responses, whether through meds or therapy, it often fails.

    I believe there is only one “therapy” that would have fairly good results and that is a system that diverts from being, feeling different, creating possibilities for successes, creating a space of acceptance for “failure”.
    But this is not feasible, since we don’t want to feel as if we are parenting adults who missed the boat.
    So, they go to therapy.
    I always enjoyed just getting out and a game of pool, or a dinner out, cleaning etc, much more than an hour of “therapy” once per week.
    It might be great if a “therapist” goes out with you once per week for a game of pool, or whatever one likes doing.
    Not sure why “therapists” don’t meet clients outside of their office. They might find their clients easier to “read” in an environment where the client feels comfortable.

    • Don’t know about Blueler streetphotobeing but regards sterilization its worth considering FASD (Foetal Alcohol Spectrum Disorder) facial recognition software and the attempt by the Western Australian Government to pass a Mental Health Act allowing the forced sterilization of children without parental consent. More “it’s for their own good” being done to the Aboriginal people of this country.
      And I thank those people who acted to stop this being passed before they ‘slipped one past’ the public who were being ill informed by our media.

  3. Testing psychotherapy to cure “mental illness” is as meaningless as testing exorcism to cast out demons.

    Exorcism and psychotherapy are cultural activities whose “effectiveness” depends on the social and personal congruence between the parties.

    It is not surprising that psychotherapy is more effective in wealthy people, since psychotherapy was born in liberal circles.

    The bursting of psychotherapeutic approaches attests to the diversity, complexity, syncretism and cosmopolitism of modern Western culture, unlike other older, more local and more homogeneous cultures.

    Psychotherapy is neither medical nor paramedical, it is a purely cultural activity which testifies to the time and the place in which we live.

    It is out of the question to reimburse psychotherapies, just as it is out of the question that the State or Social Security finance the Church; all this is only the corporatism of charlatans associated ready to submit to state control in order to enrich themselves.

    Down with the Rasputins!

    • “Psychotherapy is neither medical nor paramedical, it is a purely cultural activity which testifies to the time and the place in which we live.”

      This is an insightfully concise perspective of psychotherapy. I’d also add that any of these approaches are geared toward fitting in and adapting to the norm. Whereas challenging the norm is the ONLY way to create real and true change, at the core, and how we find our own voice, apart and distinct from those of others. And that includes “the therapist voice.”

  4. When two people come together in dialogue, they either harmonize or not. Harmony is healing for all concerned and leads to positive manifestation. The research is in the experience of feeling, and of living. If one of these two people is getting paid, that can easily throw things off balance, especially if the client is dissatisfied.

    Problem is, complaints against psychotherapists seem to be entirely discounted and the problem will always lie with the client, which of course is pure shadow projection, and that’s where the system is toxic, this “professional alliance” word against a client’s truth. When that happens, psychotherapy becomes a dangerous and harmful endeavor, creating insidious and life-derailing post traumatic stress from feelings of betrayal and utter powerlessness. Takes a lot of healing to clean up that mess. It can be really treacherous.

    • “When that happens, psychotherapy becomes a dangerous and harmful endeavor, creating insidious and life-derailing post traumatic stress from feelings of betrayal and utter powerlessness”

      I like what you’ve written here Alex. But I got to this point and thought about what was done to me and realised that it’s not “psychotherapy” that becomes dangerous, but the “psychotherapist”. All of the information I had provided to a “therapist” (if you chose to call an abuser that) was used against me.
      Create trauma via the needle phobia Boans has to justify restraint and injection with chemical restraints.
      Use his wife to damage his cause of action against the Clinic by interferring with his documents and allowing access to his computer/emails/communications with lawyers.
      Destroy all of his relationships with family/friends via the slander of “mental patient” and releasing confidential information……. and on it goes.
      This wasn’t the reified “psychotherapy” but the vicious evil minded “psychotherapist” whose paranoid delusions regarding some non existent accountability running rampant.

      The authorities don’t give a s&%t what she does to “patients” and I have the proof of this. And of course “patient” is anyone she says is her “patient” and thus allows the arbitrary detention of any citizen in the State, confirmed as lawful by our Chief Psychiatrist and Minister in writing. No observation, no standards to be met, no burden, just a phone call to Mental Health Services and police will pick the victim up for her and deliver them to whatever facility she would like. And then of course we are looking at what is considered ‘treatment’, because these guys really don’t seem to be having a lot of success when you objectively examine the folk who have been ‘treated’. Unless of course one considers damaging peoples brains, or stopping their heartbeat as the intended outcome, then they are highly successful.

      Like the guy who was handed back by police to Jeffrey Dahmer, any complaints will result in you being handed back to your abuser for ‘forced treatment’.

      And of course with a husband who is a psychiatrist to clean up the mess she was leaving in the lives of others, they could actually profit from the damage they were doing.

      I guess my argument is guns don’t kill, psychotherapists with guns kill lol.

      • I agree with you, Boans, that in the end this comes down to individuals who practice this kind of betrayal (whom I would not trust in any capacity whatsoever), but also, they are guided by their education, training, and this “mental health industrial culture,” which is unto itself in how it perceives and “defines” humanity. It is an entirely limited and narrow world view, not at all truth. I believe this is where the problems begin because obviously this industry is misguided at the core, on top of the overall corruption of it. Psychology is more often used as a weapon against people vs. using it as a clarity and truth-seeking tool. That’s rather common, I think.

        You’re also describing first hand the negative alliances I talk about, where the deck is so stacked against the client. To be scapegoated, gaslighted, and stonewalled is a HUGE life stressor, and when a society is doing it to their own people, then it is only sabotaging itself.

        Up to now, the world has been unwittingly supporting corruption because it’s been so covert and information has been distorted in so many ways. Plus, we’ve all been subject to social programming, and for so long we had no idea. But with all the waking up going on, I do believe this is about to change, certainly my deepest hope at this time. Won’t be an easy transition, but it’s so obviously necessary and, I truly believe, inevitable.

  5. “A new meta-scientific review published in a special edition of the Journal of Abnormal Psychology evaluates the evidence behind empirically supported treatments (ESTs) for mental disorders.”

    The first sentence of this blog is evidence that the US psychologists still have not garnered insight into the reality that NONE of the DSM “mental disorders” has ANY scientific validity whatsoever, as confessed to by leaders in their industry years ago.

    https://www.nimh.nih.gov/about/directors/thomas-insel/blog/2013/transforming-diagnosis.shtml
    https://www.wired.com/2010/12/ff_dsmv/

    The ADHD drugs and antidepressants create the “bipolar” symptoms, unbeknownst to our psychologists, of which I have medical proof, in addition to Whitaker’s well researched findings.

    https://www.alternet.org/2010/04/are_prozac_and_other_psychiatric_drugs_causing_the_astonishing_rise_of_mental_illness_in_america/

    And there are millions and millions of misdiagnosed “bipolar” patients, according to the DSM IV, but NOT the DSM5. This disclaimer was actually taken out of the DSM5, by the unethical within the psychiatric industry in 2013, likely to spite Robert Whitaker, and all those such misdiagnosed.

    “Note: Manic-like episodes that are clearly caused by somatic antidepressant treatment (e.g., medication, electroconvulsive therapy, light therapy) should not count toward a diagnosis of Bipolar I Disorder.”

    None of the psychologists’ “treatments” will ever help any of your clients, so long as you continue to defame your clients with your make believe, “bullshit,” and “invalid” DSM disorders, and have the psychiatrists neurotoxic poison your clients.

    The US psychologists really need to wake up to reality, walk away from the DSM, psychiatry, and start actually trying to help your clients in an empathetic and loving manner, if helping people is actually your goal.

    And until the psychologists repent and confess, and change their evil ways, they and their industry will never be anything but a systemic, primarily child abuse and rape covering up, bunch of criminals.

    https://www.indybay.org/newsitems/2019/01/23/18820633.php?fbclid=IwAR2-cgZPcEvbz7yFqMuUwneIuaqGleGiOzackY4N2sPeVXolwmEga5iKxdo
    https://www.madinamerica.com/2016/04/heal-for-life/

    And all this child abuse and rape covering up by the psychologists and psychiatrists is by DSM design.

    https://www.psychologytoday.com/us/blog/your-child-does-not-have-bipolar-disorder/201402/dsm-5-and-child-neglect-and-abuse-1

    The psychologists need to give up the “BS” psychiatric DSM “bible,” and get out of the child abuse covering up business. The problem is, of course, there really is no business for them, since their DSM belief system is “bullshit.” And since their ongoing, primarily child abuse covering up, crimes against humanity are a multibillion dollar business today, which is “too profitable” for the psychologists, psychiatrist, and mainstream doctors to want to end. The majority of psychologists today are functioning within a satanic system.

    My attitude is you all, individually, need to decide who you want to become. And all will be judged fairly for whom they chose to become. The psychologists and psychiatrists, who became god-complexed lunatics, due to be given unchecked power, who never repent and change their evil ways, will be judged as just that. But also all psychologists and psychiatrists, who repent, change their ways, and make proper amends to those they harmed, since you all do have malpractice insurance for this exact purpose, will also be judged fairly by God.

    Choose wisely, “mental health” workers.

  6. Waste of time. The only measure of whether or not any “therapy” is effective is whether the person receiving it thinks it’s effective. To think that one “therapeutic school” will magically be proven more effective, regardless of the therapist or the client, is simply a fantasy. Therapy is a HUMAN INTERACTION, not a mechanical undertaking that can be quantified and measured out like some weight of ground meat!

LEAVE A REPLY