A recent article traces a historical mismatch between clinical researchers’ aspirations in youth psychotherapy and the methods used to meet those aspirations, highlighting the importance of epistemic humility and the complexity of the research subjects. The researchers, Michael Mullarkey and Jessica Schleider, from Stony Brook University, write:
“For more than half a century, a single empirical question has remained a key driver for psychotherapy research: ‘What treatment, by whom, is most effective for this individual with that specific problem, and under which set of circumstances?”
Evidence-based interventions, however, never benefit all individuals equally. In the context of youth psychotherapy, there are no reliably detected moderators for any disorder. In articulating this persistent difficulty in attempts to universalize treatment results, the authors highlight three major mismatches between aspiration and research method in clinical research as an argument for epistemic humility and embracing complexity in youth psychotherapy research.
The complex etiology of youth mental health is well-documented. A diverse range of factors, including loneliness, racism, exercise, self-empowerment, and peer support, have been shown to affect youth mental health, requiring flexible approaches to treatment to improve treatment outcomes. Because of this etiological complexity, it has proven difficult for researchers to identify reliable moderators for treating youth with mental health challenges.
Acknowledging the complexity of youth mental health and the repeated failure of researchers’ attempts to find reliable moderators in youth psychotherapy requires that researchers refine their aspirations and research methods to fit the empirical reality of their subject matter. The authors identify three principles highlighting the mismatches between researcher aspirations and methods:
- Testing one moderator at a time will not reveal “what works for whom” in youth psychotherapy. In psychotherapy research, there are at least hundreds of candidate-moderators that may help us to personalize care; therefore, any individual treatment moderator is unlikely to have large effects. Indeed, “Many interaction effects identified as statistically significant provide negligible benefit in explaining how well individuals respond to a given treatment.”
- Most clinical trial samples are too small and non-representative to reveal what works for whom. Here, the overwhelming majority of individual trial participants identify as white and thus do not include enough people of color to examine differential effects within individual racial groups.
- We prioritize group-level design and statistics over approaches that directly assess what works for individuals. The desire for widely generalizable results thus elides the fact that there is evidence that clinical findings based on “between-subjects designs” may not be generalizable to individual people.
Thus, as the authors point out, “a large body of well-intentioned, painstaking clinical research has yielded few practical improvements for personalized youth psychotherapy.” Rather than persisting in these ineffective methodologies, researchers must embrace the complexity in the youth psychotherapy field and adopt scientific humility in their attempts to identify “what works for whom.”
Embracing humility must be a goal for individual researchers and institutions like journal editorial boards, grant review panels, and funding agencies. The authors conclude with five recommendations for rethinking research practice to maximize the chances of discovering what works for whom:
- Streamline and normalize clinical trial data sharing – with a national database, for example.
- Develop a “best practice” battery of psychotherapy research moderator variables.
- Capitalize on passive sensing technology to collect low-burden behavioral data – and recognize that generalizability of findings from group to individual is a continuum, not an on/off switch.
- Prioritize (very) large-scale clinical trials.
- Use models that can robustly test many high-dimensional moderators simultaneously.
For research across the diverse types of issues and treatments for youth tackling the “devastatingly hard” problem of finding reliable treatment moderators, “taking steps to center our goal of identifying “what works for whom” should take precedence over continuing to rely on standard approaches.”
Mullarkey, M., & Schleider, J. (2021). “Embracing Scientific Humility and Complexity: Learning “What Works for Whom” in Youth Psychotherapy Research.” Preprint under review. (Link)