AI system
- System
-
Meta proactive self-harm detection on Instagram with alert routing to the Uttar Pradesh Police Social Media Center
- Model
-
Not assessed
- Release
-
Not assessed
- Application developer
-
Meta Platforms - Role
-
Not assessed
- Configuration
-
Not assessed
- Inputs and outputs
-
Not assessed
- Autonomy
-
Not assessed
- Workflow
-
Platform detection and review that flags self-harm posts and contacts a police nodal center with account phone and location details. Police nodal center with a standing arrangement to dispatch local officers immediately.
- Human oversight
-
Not assessed
- Relation to AI
-
Police sources say Meta sent the alert after the post. They do not state whether Meta's classifier or a user report flagged it. Meta's documentation and NPR describe the process as machine-learning detection with human review before contacting authorities. The record relies on that description and should be read with that caveat.
- AI contribution
-
Police sources say Meta sent the alert after the post. They do not state whether Meta's classifier or a user report flagged it. Meta's documentation and NPR describe the process as machine-learning detection with human review before contacting authorities. The record relies on that description and should be read with that caveat.
Benefits
- Outcome identifiers
-
Not assessed
- Benefit
-
3,011 lives saved statewide from 1 January 2023 to 31 May 2026
- Category
-
Improved Physical Health
- Status
-
Not assessed
- Beneficiary groups
-
People.
- Beneficiaries
-
Not assessed
- Reach
-
Not assessed
- Distribution and access
-
Not assessed
- Duration
-
Not assessed
- Expected?
-
Yes
- Costs and harms
-
Not assessed
Measurement
- Studies
-
Not assessed
- Study design
-
Not assessed
- Measurement
-
Not assessed
- Setting and sample
-
Not assessed
- Measures
-
3,011 lives saved statewide from 1 January 2023 to 31 May 2026
- Comparison
-
Not assessed
- Results
-
3,011 lives saved statewide from 1 January 2023 to 31 May 2026
- Additional value from AI
-
Not assessed
- Limitations
-
Not assessed
- Denominators
-
Not assessed
- Materials for replication
-
Not assessed
Money
- Kind of value
-
Not assessed
- Amount
-
Not assessed
- Currency and prices
-
Not assessed
- Period and unit
-
Not assessed
- Realization
-
Not assessed
- Recipients and payers
-
Not assessed
- Gross or net
-
Not assessed
- Included costs
-
Not assessed
- Comparison
-
Not assessed
- Method
-
Not assessed
- Return and payback
-
Not assessed
People and places
- Participants
-
Developer:
Meta Platforms. Operator:
Meta Platforms with the Uttar Pradesh Police Social Media Center (agreement signed March 2022). - Jurisdictions
-
Benefit: Meerut,
India. Developer:
United States. Operator:
United States. - Ownership and control
-
Not assessed
- Dependencies
-
Not assessed
- Cross-border connections
-
Not assessed
- Geopolitical implications
-
Not assessed
Follow-up
- Conditions for replication
-
Account details resolve to a reachable address within minutes. Local officers are available to respond immediately.
- Replication attempts
-
4 follow-up entries. Benefit recurred: 4.
- Later findings
-
No development dated after the 7 June 2026 case was found in the cited sources.
- Availability
-
Not assessed
- Institutional decision
-
Not assessed
- Grounds
-
Not assessed
- Action and result
-
Not assessed
- Success criterion
-
Not assessed
Evidence
- Source numbers
-
AIBD-0010-source-0001–0006
- Citations
-
Numbered references below.
- Language
-
Not assessed
- Source relationships
-
Independent, Developer
- Original evidence
-
Not assessed
- Access and versions
-
Not assessed
- Supported findings
-
Not assessed
- Evidence links
-
Not assessed
- Evidence strength
-
Not assessed
- Missing evidence
-
Not assessed
Grades
- Assessed benefits
-
People.
- Depth
-
People: 4.
- Reasons
-
People: Police found him in a critical condition after apparent ingestion, and he was treated in hospital within minutes of the alert. Police describe the outcome as a life saved.
- Observed or intended
-
Not assessed
- Rubric
-
Not assessed
- Assessor and date
-
Not assessed
- Alternative assessments
-
Not assessed
- Fill
-
Independently reported
Entry details
- Kind
-
Not assessed
- Related entries
-
Same developer and platform, Meta's Instagram, and the same at-risk user group: Instagram's recommendation of suicide and self-harm content was ruled to have contributed to Molly Russell's death, where the Meerut alert reached a user in time.
Instagram's Exposure of Harmful Content Allegedly Contributed to Teenage Girl's Suicide
- Field
-
Suicide prevention / online safety. Social media platforms and police.
- Purpose
-
Built to flag posts and live videos indicating suicide or self-harm risk for review and, in imminent cases, referral to emergency services. This benefit is that purpose.
- Dates
-
Date basis unassessed: 2026 (precision unassessed). Updated: 22 October 2026.
- Stage
-
Not assessed
- Version
-
SHA-256: 343d2f4067b7…
References
- Other identifiers
-
oecd_aim: 2026-06-08-7d81. AI Incident Database: 382.
- Category mappings
-
Not assessed
- Brief
-
Not assessed
- Revisions
-
7 revisions. Latest: 22 October 2026.
Sources
-
News report Independent author
PTI (2026). When every minute counts: Uttar Pradesh Police save youth attempting suicide in 8 minutes after Meta alert over Instagram post. Deccan Herald.
flags posts indicating possible self-harm and shares alerts with authorities
-
News report Independent author
The Hans India (2026). Cops save man in 8 mins after suicide alert on Insta.
Police said Meta sent a suicide alert regarding the post to the Social Media Centre at the police headquarters at 6.53 pm.
-
News report Independent author
Kaste, M. (2018). Facebook Increasingly Reliant On A.I. To Predict Suicide Risk. NPR (transcript republished by North Country Public Radio).
We also are able to use AI to coordinate a bunch of information on location to try to identify the location of that individual
-
News report Independent author
PTI (2023). Facebook helps UP Police avert two suicides in a week in state. The Tribune.
Meta Hqs USA immediately sends an alert to the social media centre of PHQ UP Police through phone and email
-
News report Independent author
PTI (2021). Man saved by Delhi Police after Facebook alerts on self-harm video. The Tribune.
received a call alert from Facebook's US office about a suspicious self-harm live video being posted by a male Facebook user located in Delhi
-
Press release Developer
Card, C. (2018). How Facebook AI Helps Suicide Prevention. Meta Newsroom.
In serious cases, when it's determined that there may be imminent danger of self harm, Facebook may contact local authorities.
Collections: Cross-border cases, Safeguards, Security and threat-watching.