Data Challenges Superiority of Manualized Psychotherapy

A recent systematic review comparing manualized psychotherapy to non-manualized psychotherapy has challenged the ongoing promotion of psychotherapy manuals as a necessary part of evidence-based treatments (EBTs). Researchers, Dr. Femke Truijens and colleagues in Europe, found that manualized psychotherapy is no more superior to psychotherapy delivered without a manual.

“Manualized treatment is not empirically supported as more effective than non-manualized treatment. While manual‐based treatment may be attractive as a research tool, it should not be promoted as being superior to non-manualized psychotherapy for clinical practice.”

Psychotherapy treatment manuals are intended to direct therapists in the application of their approach. Manualized treatments specify a theoretical basis, the number and sequencing of treatment sessions, the content and objectives of each session, and the procedures required to achieve the objective of each session. The use of manuals have been embraced, and at times required, by overseeing institutions such as the American Psychological Association (APA) and the National Institute for Health and Care Excellence (NICE).

“This requirement captures the assumption that it is more effective to apply manualized treatment than to provide treatment in a less or nonmanualized form. As this assumption seems vital to justify the dissemination of manual‐based EBTs to clinical practice, in this paper, we review the empirical evidence for this assumption,” Truijens and colleagues explain.

They note that in clinical practice, there has been pushback to manualized approaches and the utility of manuals has been critiqued. Scholars and psychotherapists have expressed concerns that manuals inhibit flexible application of approaches, and impedes on one’s ability to tailor therapy to individual needs or adapt interventions to multiple, or “comorbid” presentations of distress.

In addition, manuals tend to be constructed around diagnostic presentations such that specific approaches are delineated for specific “disorders.” Practitioners critique the feasibility of mastering each approach. One response to these concerns has been to encourage the flexible adaptation of treatment manuals. For example, through the use of “transdiagnostic” manuals.

Nevertheless, the research has been focused on how to apply manuals rather than on whether or not manualized approaches are more effective. To address this gap in the literature, the authors consider the following questions:

“Does the use of manuals actually increase therapy effectiveness? And should manuals, therefore, be embraced in clinical practice and training?”

Truijens and team sought to add to this discussion by reviewing the empirical evidence. They write, “Given the current requirement of manuals as the core of evidence-based psychotherapy, it seems crucial to substantiate this discussion with empirical evidence.”

In this systematic review, the research team evaluated whether or not manual-based psychotherapy was more effective than psychotherapy delivered without a manual. They also examined the efficacy of manualized and non-manualized psychotherapies as compared to no treatment, delayed-treatment, minimal treatment, or alternative treatment control groups. Lastly, they examined lower levels of therapist adherence to the manual. The hypothesis was that if manualized therapy is indeed more effective, then the extent to which the therapist adhered to the manual would be linked to effectiveness.

To explore these three hypotheses, Truijens and colleagues conducted a systematic review of the existing literature. For the first hypothesis, they examined six relevant empirical studies. Eight meta-analytic studies applied to the second hypothesis and one meta-analysis of 15 studies was used to explore the last hypothesis regarding manual adherence.

Their results did not support the superiority of manualized psychotherapy compared with non-manualized psychotherapy. The researchers review of the six articles comparing manualized and non-manualized therapy directly found that three studies yielded no significant difference between the two, two observed superiority of non-manualized therapy, and one supported manualized delivery. The one study that did support manualized psychotherapy was interpreted by the authors to have been started “from a single specific intervention that appeared to be exceptionally effective, regardless of the administration via a manual.”

When manualized and non-manualized psychotherapy was compared with no treatment, delayed treatment, minimal treatment, or alternatives, the superiority of manualized psychotherapy was also not conclusively supported. Out of the eight meta-analyses reviewed, three demonstrated an advantage of using manualized therapy, one indicated the superiority of non-manualized delivery, and four showed no significant difference. The authors interpreted these findings:

“Here, we have to remark that it is fairly complex to meaningfully compare effect sizes of treatments that are so different in nature, given their varied understanding and operationalization of treatment, control groups, diagnosis, and outcome. First and foremost, this underlines how the universal hypothesis of manual efficacy is in trouble with respect to empirical support, both as a direct and as a moderating factor.”

Finally, when therapist adherence to the manual was explored, results were similarly inconclusive. One meta-analysis found that the degree of therapist adherence to the manual did not affect outcomes. The remaining 15 studies provided unclear results. Truijens and team comment on these findings:

“As such, the suggestion that adherence and fidelity to treatment principles may impact a positive treatment outcome remains a worthwhile avenue for further research. However, as an indicator for ethe fficacy of the manual as a general principle for clinical practice, this conflicting body of evidence is insufficient.”

The findings of this study do not support the superiority of manual-use in psychotherapy. The authors write that the failure to corroborate this claim “points to a severe problem in the justification of EBT dissemination.” In their conclusion, Truijens and team encourages consideration beyond the question of “manual or no manual?” toward the components and steps of the therapy process required to attend to different people and different presentations.

“Based on this review, we are not inclined to call for more research to settle the dispute about manualization in general; rather, we urge both researchers and clinicians to go beyond the dichotomy, as the next step in understanding what works for whom in psychotherapy.”

 

****

Truijens, F., Zühlke‐van Hulzen, L., & Vanheule, S. (2018). To manualize, or not to manualize: Is that still the question? A systematic review of empirical evidence for manual superiority in psychological treatment. Journal of clinical psychology. DOI: 10.1002/jclp.22712 (Link)