Facebook’s AI Suicide Prevention Tool Raises Concerns


From The Daily Dot: Last week, Facebook announced a global rollout of their new suicide prevention algorithm, which aims to identify users deemed at risk of suicide and refer them to resources. Many people in the mental health and disability communities are concerned about the potentially coercive implications of the new tool.

“Some have expressed fear that Facebook Community Operations team members could request that police check up on them over a post’s content. Claire Kooyman, a 34-year-old Boulder, Colorado resident, told the Daily Dot that she’s previously used her Facebook page as a way to reach out to friends when she’s struggling with her depression and anxiety. She’s found it to be a way to get positive reinforcement from others who experience the same symptoms, but worries about what she’ll share next, and if it’ll get flagged by Facebook’s new system.

Kooyman also expressed concern for flagged users who are then picked up on a mental health hold—an involuntary detention of someone who hasn’t committed a crime but is considered a danger to themselves or others. In some states, if psychiatric facilities are unavailable, then patients can be held in jail.”

Article →­


  1. I apparently got flagged in the early trials. A first responder arrived at 1145 pm at the private shelter where I was living a couple years ago. Talk about feeling super-intruded! The shelter’s street address was not registered in any public information system, nor did any members of the private FB chat know the address. Totally creepy. Needless to say, these conversations are now offline.

    Report comment

    • Oh yes, another well meaning “prevention” program that is actually another way to funnel people into the system for no benefit to them. I am sorry you had to be the beta version suspect.
      I do agree… they can’t find your house if it’s on fire and the residence listed, but they can find you no matter what to take your civil rights from you.

      Report comment

  2. Suicide prevention? Not hardly. Allowing people to talk about this would be far more preventative than making them feel ashamed, “crazy” and isolated because they are struggling. What drug company or member(s) of the illness maintenance system put up money to “fund” this? Who monitors it? And, if they can monitor this, what else have they been monitoring and for whom?

    Report comment