Mental Health Advocacy in California: Perspectives of Advocates and Decision-Makers

In partnership with the California Association of Mental Health Peer-Run Organizations (CAMHPRO), Live & Learn, Inc. conducted a survey on the impact of stakeholder advocacy on decisions affecting public mental health systems in California. The California Mental Health Stakeholder Advocacy Survey was designed by people with personal experience of the mental health system and related advocacy work from CAMHPRO, Live & Learn, Inc., and Shifa Consulting.

The objective was to pilot an approach to help CAMHPRO evaluate the impact of consumer advocacy in the state and to document the activities that advocates engage in (e.g., legislative testimony, demonstrations, campaigns). We asked advocates and decision-makers how these groups engage in advocacy, what their roles are, the goals of advocacy, and its perceived effectiveness in fostering change.

The survey was conducted in April 2016 and asked about advocacy activities in California in the past 6 months conducted by people who identify as consumers, survivors, ex-patients, or peers.  There were different versions of the survey for consumer advocates and for county decision-makers.

The California Mental Health Stakeholder Advocacy Survey is based on an activity logging approach developed by the Ontario, Canada Consumer/Survivor Development Initiative (CSDI).¹ The activity log obtained information about four categories of system-level activities:

The survey developed by Live & Learn, Inc. and CAMHPRO included questions about:

• The specific activities advocates engaged in
• The primary type of improvement advocates sought: local services, policy or legislation, societal attitudes, or networking for advocates
• The target system for change of the activity: mental health system, general health sector, community, and government

Discussion of Findings from the Survey:

Advocates in California engage in a wide variety of public relations, community planning, political advocacy, and action research advocacy activities. The most commonly reported activities are related to Public Relations, Community Collaborations, and Action Research, and less frequently to Political Advocacy.

The survey provides insight into the goals of advocacy. The most common improvement desired by advocacy was to affect policy and legislation, and the least common intention was to improve networking for advocates. The government sector was the most common target of advocacy intended to change policy and legislation.

While improving networking was not an intended outcome, it was most frequently rated by advocates an area in which their activities were effective. Policy and legislative change was   rated as effectively a minority of advocates and decision-makers.

While this approach needs further development and ideally should be combined with other approaches in order to understand the impact of advocacy, there are some potentially interesting patterns. For instance, these data raise questions such as: If policy change is a frequently desired improvement, but one of the lowest ranked in effectiveness, how could activities be changed or adapted to have more of an impact?

Conclusions and Recommendations about Measuring Advocacy

Stakeholder advocacy includes activities by people directly affected by policy decisions, with the goal of guiding or changing policy through collective action. People directly affected by mental health policymaking are frequently a marginalized, under-resourced group, and there is very little information available to support decisions about what kinds of advocacy activities are most effective for achieving what kinds of results under what circumstances.²

The principle reason for the scarcity of research evidence is that the advocacy process does not lend itself to traditional program evaluation methods.^1,3,4 It is usually complex and non-linear. Policy change usually occurs as a consequence of multiple groups and social forces working in diverse ways, often with different specific goals and interests, over a long period; this makes it difficult to identify specific cause and effect relationships.⁵

Though randomized controlled designs are generally considered the “gold standard” in evaluation, the complexity of the advocacy process precludes using this method in this context. A more suitable method is an activity logging approach, in which a system is established for recording advocacy activities and attempting to identify associated outcomes. The conventional approach to assessing advocacy outcomes is to enumerate activities (what evaluators refer to as “outputs”)—for example, numbers of meetings with legislators, media events held, petitions signed, position papers produced, etc.

Although the objective impact of advocacy cannot necessarily be judged using this type of method, it does allow for broad-based sample of perceived impact. When paired with other approaches, this may be helpful in understanding the impact of and investments in advocacy. A similar study, whose methods we adapted,  the Ontario Consumer/Survivor Development Initiative (CSDI)¹, found that the activity log had practical benefits for consumer/survivor initiatives. The data were used to bring attention to the systems change efforts of consumer/survivor advocates. The tool helped advocates recognize the breadth of their efforts and provided them a sense of credibility and accomplishment. The activity log was also useful in reporting to funders, organizational boards of directors, and other stakeholders.¹

The obvious limitation of enumerating activities is that it is usually impossible to determine which activities actually produced a particular change in policy. All activities likely have some effect. But the ability to assess the relative impact of each activity over time would allow for continuous refinement in choosing how and where to focus resources for maximum impact. Other research used single questions asking respondents to assess whether goals were achieved in order to measure advocacy effectiveness. Earlier research has shown self-report to be reliable, and there are typically few incentives for respondents to misrepresent.²

One difference in evaluating advocacy -in contrast to other types of activity- is the unpredictability of the process. Advocates typically respond to opportunities as they occur, such as new policy initiatives, leadership changes, etc. We therefore recommend that evaluation be approached from a learning perspective: that researchers and advocates share an understanding that advocacy is complex and usually unpredictable, and that evaluation is an ongoing process aimed at tailoring strategy to continuously improve advocacy outcomes.

 

Contributors to this report include:

• Laysha Ostrow, PhD, CEO of Live & Learn, Inc. ([email protected])
• Sally Zinman, Executive Director of CAMHPRO
• Dow Wieman, PhD of Human Services Research Institute
• Leah Harris, M.A. of Shifa Consulting
• Darby Penney, M.L.S., editor

* * * * *

To read the full report go to http://www.livelearninc.net/s/Calif-Advocacy-Survey-Report-2016.pdf

References:

1. Janzen, R., Nelson, G., Hausfather, N. & Ochocka, J. (2007). Capturing system level activities and impacts of mental health consumer-run organizations. American Journal of Community Psychology, 39(3-4):287-299.
2. Hoefer, R. (2005). Altering state policy: Interest group effectiveness among state-level advocacy groups. Social Work,50(3):219-227.
3. Chapman, S. (2001). Advocacy in public health: Roles and challenges. International Journal of Epidemiology, 30:1226-1232.
4. Devlin-Foltz, D., Fagen, M.C., Reed, E., Medina, R. & Neiger, B. L. (2012). Advocacy evaluation: challenges and emerging trends. Health Promotion Practices,13(5):581-586.
5. Brownson, R.C., Seiler, R. & Eyler, A.A. (2010). Measuring the impact of public health policy. Preventing Chronic Disease, 7(4): A77.