Human Suffering as Numbers and Graphs: The Problem with Measuring Outcomes in Therapy

Patients arriving to their first therapy session are often met with a series of questionnaires before even getting to meet their therapist. The practice of regularly administering clinical surveys in therapy, known in the research literature as routine outcome monitoring (ROM), is becoming increasingly prevalent. Proponents of the practice claim that completing clinical surveys at every appointment (or some routine schedule) can help track progress in therapy and improve outcomes. As a result of this advocacy, government health services, university training courses, and a growing number of private clinics have made it mandatory for clinicians to administer outcome surveys and for patients seeking care to complete said surveys.

In a recent article, however, I examined the research-base, finding that outcome monitoring is less helpful than is generally thought, and how its growing predominance can be attributed to sociopolitical influences in our healthcare systems.

Oversold and Overhyped

One might be forgiven for believing that, given the ubiquity of the practice in clinical settings, routinely monitoring outcomes must be a well-established method of improving therapy. Certainly, there is a plethora of studies that show that participants’ scores on these measures improve over time when they complete them regularly. Most of these studies, however, are conducted in laboratory settings—and the few studies conducted in real-world contexts have been considerably less sanguine.

A 2016 Cochrane review concluded that there was insufficient evidence that regularly administering surveys contributed to patient improvements in primary care, and an updated review in 2020 concluded similarly that the scores from these surveys needed to be ‘interpreted with caution’. A more recent review of all existing meta-analyses determined that—after all the time and effort taken to administer surveys, complete them, input responses into databases, interpret scores, and incorporate the results into every session—the average patient could expect an 8% ‘improvement’ over standard care.

Even so, keep in mind the circularity of reasoning that these types of studies commit, in which ‘improvement’ is defined using the very measures that the study is seeking to evaluate. As many therapists would attest, scores on clinical questionnaires mean little on their own. Worse results do not necessarily indicate less progress, and indeed there are many reasons why patients who are doing well in therapy would report poorer scores. One of the core objectives of therapy is to help patients experience their repressed or dismissed feelings—with this in mind, reports of greater distress, dysregulation, or emotional turmoil are often indicative of a fruitful session.

Unpopular with both Therapists and Patients

Despite its growing prevalence, routine outcome monitoring remains a deeply unpopular practice amongst everyday practitioners, who experience it as a time-consuming and burdensome bureaucratic task. According to Farhad Dalal, reports from the United Kingdom’s IAPT suggest, for instance, that many therapists are made to take 15 or even 30 minutes out of their usual one-hour sessions in order to process the mandated measurement surveys.

Of even greater concern for practicing therapists, the results from these outcome surveys are also often employed as Key Performance Indicators (KPIs), used to monitor the performance of individual clinicians and the services they work in. In his book CBT: The Cognitive Behavioural Tsunami, Dalal documents how this has led to a systemic ‘gaming’ of results that is prolific throughout many healthcare organizations. Some of the ways that both clinicians and managers meet their KPIs include avoiding complex and severe patients (who are more likely to report worse scores), encouraging patients to report better scores, or simply manually adjusting results themselves. As a consequence of the enormous pressure on clinicians to meet performance metrics tied to ‘outcomes’, the scores themselves have become largely uninterpretable.

It would also seem that routinely completing outcome measures is as unpopular with patients as it is with clinicians. Several studies have highlighted patients’ diverse and astute reservations with the practice—with patients just as prone to view outcome monitoring as a pointless administrative exercise. In particular, patients with lower educational backgrounds or limited language skills have trouble deciphering the often overly intellectualized questions—unsurprising given that the vast majority of measures are written and developed by highly educated, middle-class, and often white academics. In some ways more discerning of the limitations of these surveys than the academics who designed them, many patients also report concerns about the real-world validity of measuring abstract and complex human emotions with concrete response categories.

Mental Health Clinicians: Contemporary Bureaucrats

In a profession where the distance of chairs, dictation of emails, and colours on the waiting room walls are scrutinized, it is surprising that relatively little attention was been paid to how regularly administering questionnaires might influence the relationship between therapist and patient. Especially when the therapist is the overseer of these questionnaires, this practice is bound to have an impact on how the patient conceives of the therapist as well as therapy itself.

In the field of psychotherapy, the term ‘transference’ refers to the attitudes, feelings, desires, and fantasies that a patient holds towards their therapist. These emotions are often a redirection of those held towards caregivers and other important figures in the past—hence the term ‘transference’. The act of administering surveys to a patient, and then monitoring the results of these surveys, unquestionably evokes some form of transference in the therapy room. Will patients be reminded of being handed exams at school, anxiously awaiting the results of their efforts? Or will they be worried about what their therapist will think of their answers, in the same way they worried about their parents’ reactions to their school reports? The process of outcome monitoring may, in fact, be more reminiscent of being performance managed at work—a tedious, monotonous, and perhaps personally-threatening exercise in structural power imbalances.

Transference, at least, can be addressed explicitly, and even provide grist for the so-called mill. Therapy itself though—its structure, purpose, frame, goals—is apt to take on a different meaning when it is bookended by measures at beginning or end. That the majority of clinical measures focus on the experience of ‘symptoms’ primes both patient and therapist to view their hour together as a space for addressing illness pathology. What measures don’t focus on symptoms usually focus on ‘function’—how engaged and productive one is in completing tasks of daily routine, as well as school or work attendance. Therapy becomes moulded to the frame of a capitalist system that views the purpose of all self-improvement endeavours—therapy included—as to improve productivity, conformity, and contribution to the economy. Therapists then become as much a supervisor of their patients’ daily performance as their manager is of their own—therapist and patient alike, together colluded in the task of meeting expectations.

Outcome Monitoring: Capitalist Realism in the Sphere of Psychotherapy

With all of this in mind then, why does routine outcome monitoring continue to be promoted, encouraged, and even mandated in various clinical settings?

Proponents of the practice argue that analyzing patient data on an aggregate scale helps guide policymaking and steer public investment. This argument is easily dismantled, however, when looking at the patterns of healthcare funding in countries where outcome monitoring is deployed—with those services receiving the most funding often demonstrating the worst outcomes.

While outcome monitoring of this kind might not be for the benefit of patients or the assistance of therapists, there are certainly those that benefit greatly from the operationalisation of human suffering. A victim of the ‘managerialization’ of public services, routine outcome monitoring provides the Key Performance Metrics (KPIs) from which upper-level executives imagine to assess worker productivity and optimize efficiency. In this ecosystem of neoliberal thinking, what public services have not yet been sold off are operated like business ventures. Hospital departments hold ‘business meetings’, patients are referred to as ‘consumers’, and austerity doctrine reigns. Meanwhile, a growing stratum of bureaucrats and administrators must be employed to manage all of this outcome data.

Outcome monitoring is likewise becoming popular in private clinics, again to help monitor worker productivity as well as to, indirectly, maximise revenue for owners—as is the dictum of offering help under capitalism. Many private clinics have also started to use outcome results for marketing material, eroding what little personal meaning these measures may have had for patients. One such practice boasts of their commitment to ‘data-driven care’ while another advertises their ‘client satisfaction results’ on social media.

The touch of irony here is that, as a practicing psychologist, I myself use outcome measures all the time. When these tools are used thoughtfully, with a sense of purpose and in collaboration between both participants in therapy, they can contribute immensely to the process of understanding oneself. The growing preponderance of these measures being mandated as top-down obligations, however, needs to be viewed for what it is: a product of neoliberal economic forces and an attempt to reify the human condition—for all its nuances, complications, undulations, and ordeals—into terms compatible with, and ultimately subjugable to, the market economy.