Child safety: Meta AI starts flagging teen distress signals 🛡️

Cliquez ici pour lire en français

Meta has announced a new feature that notifies parents when a teenager brings up suicide or self-harm in a conversation with Meta AI. Rolling out in the United States, the United Kingdom, Canada, and Australia, the tool pairs AI detection with human review before any alert goes out.

A heads-up for parents 👨‍👩‍👧

The feature lets Meta AI flag conversations where a teen expresses thoughts related to suicide or self-harm — even when it’s phrased indirectly or ambiguously.

When that happens, parents get a notification encouraging them to check in with their kid, along with resources and guidance on how to approach the conversation. What they don’t get is the actual content of the exchange between the teen and Meta AI.

A two-step detection process 🤖

The system works in two stages. First, an AI model scans conversations for signs of psychological distress. Every flagged conversation then goes through a human review before any notification reaches a parent.

Meta says the double-check is meant to minimize false positives while staying on the side of caution. When a teen’s intent is unclear, the company reportedly prefers to send an alert rather than risk missing a real crisis. Notifications go out via text, email, or in-app, depending on how parents have set up their parental supervision tools.

And when it’s an emergency 🚨

Beyond parental notifications, Meta is also working to extend a mechanism already used on Facebook and Instagram to Meta AI. When its systems detect that someone may be in immediate danger, the company can flag the case directly to emergency services.

That capability isn’t live yet for Meta AI conversations, but it’s already operating on Facebook and Instagram — and reportedly triggered more than 19,000 reports to emergency responders worldwide over the past year.

A response to mounting pressure on child safety 🛡️

The announcement lands amid growing scrutiny of how tech platforms protect young users. Meta has reportedly faced legal action in recent months over minor safety and the effects of its platforms on teenagers.

Regulators aren’t slowing down either: in Australia, social media has been off-limits to anyone under 16 since December, and authorities there have been pushing platforms to spell out exactly how they’re protecting young users. With this new feature, Meta is betting on tighter prevention and detection within its AI assistant itself.

Do you think AI can genuinely help keep minors safe online — or does this kind of monitoring raise real privacy concerns for families?