Apples and Oranges in Peer Support Research

Do you skip to the conclusions section at the end of research articles?

We’re certainly at a crossroads now about the future of peer support, and everyone’s looking for evidence for or against it. But if you’re trying to decide whether to continue or to cut peer support in your organization, you’ll need a critical eye as you make your way through the literature.

I recently came across a peer-reviewed journal article by White, Foster, and Marks, et al titled “The effectiveness of one-to-one peer support in mental health services: a systematic review and meta-analysis.” It seemed a smart choice for getting myself caught up on recent peer support outcomes quickly. But on closer inspection, I realized the conclusion is less than helpful for technical reasons, and I’d like to explain why.

First, I should say that I’m a person living with a psychiatric diagnosis. I’m a “peer.” I’ve worked in the behavioral health field for several decades, and almost all my career positions have been related to peer support. I also specialized in research and evaluation in my graduate studies because I’m both excited by and confident in the dynamic question-and-answer process of the scientific method. There’s so much we can do with data if we collect, analyze, share, and discuss it responsibly.

It’s critically important for us to understand what’s passing for “evidence” about peer support in research if we want it to have a future.

So, let’s look at this meta-analysis closely. Meta-analysis is cool because it combines information from many very similar studies. Ideally, the more data and information you have in research, the more confident you can be about results. And, as long as we’re comparing apples to apples and oranges to oranges, reading one meta-analysis should be nearly as good as reading all 23 articles, in this case.

White, Foster, and Marks, et al acknowledged a serious apples and oranges problem with their data, which we’ll look at, but the final two sentences of their conclusion read:

“[We found] a modest, positive effect of peer support on self-reported recovery and an absence of effect, in the evidence to date, on clinical outcomes. Again, [the review] indicates that reporting bias—incomplete reporting of outcomes—continues to undermine the quality of the evidence base as a whole.”

This damning conclusion was a shock to me, honestly. So, I resigned myself to spending the day with this study. What evidence warranted a description of peer support as only slightly useful (or fairly useless), and cast serious doubt on the entire evidence base on peer support?

These are my thoughts about the meta-analysis and its conclusion through my magnifying glass.

The term “reporting bias” in the conclusion suggests we’re cherry picking the data that’s reported. Is that because peer support outcome data don’t show what we’d like them to show? Or is it that the complaint is based on a gross misunderstanding of what peer support is and how it works? If clinical outcomes (apples) are not the same as recovery outcomes (oranges)—and they aren’t—the “missing” apples didn’t belong in the basket mixed up with oranges, anyway.

William A. Anthony defined recovery as “having meaning and purpose in life despite symptoms of mental illness.” We measure recovery concepts in peer support in terms of hope, self-empowerment, self-determination, and engagement in meaningful activity, for example, not scores on clinical inventories. We know we’ll continue to experience symptoms and use mental health services, but also that the quality of our lives can improve dramatically at the same time.

Peer support isn’t a clinical service provided by clinicians with clinical objectives. Measuring service use and symptoms of illness of people receiving peer support seems illogical, in my opinion. To be fair, the apples and oranges problem is a challenge that every meta-analysis grapples with, but the evidence for recovery from mental illness is not the absence of symptoms, and this single misunderstanding seems to account for an enormous amount of confusion for clinically oriented behavioral health professionals.

Can we say that because these researchers found that “there was an absence of effect on clinical outcomes,” peer support doesn’t work? It wouldn’t be logical to come to that conclusion, but a quick glance at the last few sentences of the article might lead someone to think so.

The next major apples and oranges problem has to do with the duties and relationships attributed to peer support specialists. This isn’t really just a simple apples and oranges problem for researchers, it’s an enormous and wildly exotic fruit salad. It doesn’t work to combine “peer support services” in one big basket when they’re not alike unless the differences are relatively unimportant. But these differences are particularly important.

At a national conference about 15 years ago, I was a member of a Q & A panel on peer support. A woman who identified herself as a Peer Support Specialist stepped up to the mic and said, “I feel like a failure. My team asked me to try to get consumers to take their medication. They thought I could do it better because I’m a peer. I can’t do it better, though. I’m terrible at it.” The co-opting of peer support specialists into roles and duties that don’t fit with their training and purpose is a grave issue that plagues us to this day.

A number of services in this meta-analysis were described as “paraprofessional case management” or “structured behavioural interventions.” These activities, by their very nature, seriously undermine a power-balanced peer support relationship, and they really should have been disqualified by the researchers as “too different” (apples, not oranges).

Even “person-centered” case managers have perceived power over information and resources in the eyes of people receiving case management services. Case managers serve as the gateway to knowledge, benefits, resources, privileges, and opportunities. They perform inspections, write assessments, and document reports that directly impact our housing, employment, and rehabilitation prospects, and their evaluations of our progress often shape our treatment plans.

But limiting the service of paraprofessionals, clinicians, and educators with lived experience of recovery from mental illness or addictions is not at all what I’m suggesting. It would be tragic to clamp down and discourage new ideas in the name of standardization. To correct the apples and oranges problems in research on peer support, we don’t need to disqualify, devalue, or eliminate any combination of peer status, job title, or job description from existence. But we must be able to describe and categorize who is doing what so that research outcomes for peer support are reliable, valid, and useful.

For example, a “peer” who is hired and performs the job duties of a Case Manager is a Case Manager. The mere status of being a “peer” should never be used to recategorize the person’s job title or function as “Peer Support.” They may use their lived experience to develop more trust or empathy in the supportive relationship. But renaming or reclassifying this person’s title and/or function to “Peer Support” without correcting the job description and balancing power in the working relationship is misleading, inappropriate, and has muddied the water terribly.

We could study new Case Manager job titles with new job descriptions that indicate the status of the Case Manager as a peer. Those hybrid entities are already out there and they’d be another excellent group to study. But the “Peer Support” category for peer workers, employers, funders, researchers, educators, and decision-makers should refer to a specific set of criteria so that Peer Support, as it is intended, has a future. Many others have written extensively on this topic, and their expert advice should shape a nationally recognized Peer Support job title and job description that employers and researchers follow very closely.

My basic suggestion for defining “Peer Support” job titles and job descriptions is a 3-item checklist:

• The employee publicly identifies as a “peer” who has been certified as a Peer Support Specialist by a reputable organization.
• The job title uses exclusive “Peer Support” wording.
• The job description aligns with the philosophy, training, skills, and practical applications of solid peer support theory.

There are well-known sources of accurate peer support outcome data, and we should hold researchers accountable for the quality of their data selections. I also understand that describing peer support roles and responsibilities feels like “codifying,” and that we’ve historically resisted labels of all kinds as a grass roots movement. But now researchers need our help, and we need theirs, too.

In my opinion, peer support has been around long enough to stand the test of description and measurement. Our terms, functions, and even our philosophy will change over time because we’re a dynamic group of people. Let’s work together to improve the accuracy and quality of peer support outcome data. We have nothing to fear, and a great deal to learn.