WHEN AI MISREADS SURVIVORS
A Responsible AI Governance Project for Domestic Violence OrganizationsHow can AI systems create harm if they do not understand survivor behavior, trauma, coercive control, and safety?
Independent Responsible AI & Survivor-Safety Project
Responsible AI · Human Oversight · Survivor Safety · Privacy · Bias · Governance
THE GOVERNANCE PROBLEM
Survivors are often misread by people. AI must not be allowed to repeat that harm at scale.
Domestic violence services operate in environments where survivor behavior may be shaped by fear, trauma, coercive control, financial dependence, surveillance, concern for children, immigration status, housing insecurity, or the need to stay in contact with an abusive person.
An AI system may see only the observable behavior:
Delayed reporting · Missed appointments · Inconsistent timelines · Continued contact · No police report
Without the right context, those behaviors can be misread as:
Low credibility · Low urgency · Noncompliance · Reduced risk
That misclassification can then influence how information is summarized, prioritized, escalated, or acted upon.
The governance problem is therefore not simply whether the AI produces an accurate summary. It is whether the system understands enough context to avoid turning complex survivor behavior into harmful institutional assumptions.
HOW HARM CAN HAPPEN
REAL-WORLD WARNING SIGNS
Structured domestic-violence risk tools have already raised questions about what happens when complex survivor realities are reduced to categories and institutional actors place too much confidence in a score.
DASH | United Kingdom
Reported cases and subsequent research have raised concerns about situations in which people facing serious danger were not identified as high risk.
Governance lesson: structured assessment tools can miss danger when complex experiences are flattened into fixed categories or when a risk classification is treated as unquestionable truth.
VioGen | Spain
Spain's VioGen system has also prompted scrutiny concerning low-risk classifications, transparency, human review, and how changing circumstances are captured.
Governance lesson: even systems designed to improve decision-making require transparency, training, human review, override mechanisms, and continuing assessment.
These cases do not mean that a single tool alone caused a particular outcome.
They demonstrate something more important for governance:
when risk tools, institutional judgment, and protection systems fail together, the consequences can be severe.
Governance therefore has to begin before additional automation enters survivor-support workflows.
WHAT AI MAY SUPPORT AND
WHAT IT MUST NEVER DECIDE
Appropriate Support Functions
AI may be useful for administrative or assistive tasks such as:
-
summarizing intake notes
-
organizing information for advocates
-
surfacing possible support needs
-
suggesting follow-up questions
-
reducing administrative burden
Decisions That Must Remain HumanAI should not independently make:
-
credibility judgments
-
service-denial decisions
-
final risk classification
-
scase-closure or deprioritization decisions
-
decisions that could result in unsafe survivor contact
WHAT WE WANT TO BUILD
The goal is to turn these principles into practical governance tools that domestic violence organizations, advocates, technologists, designers, and Responsible AI professionals can actually use.
Responsible AI Case Study
A detailed example showing how AI can misread survivor behavior and where governance intervention is required.Domestic Violence AI Risk Register
Domestic Violence AI Risk Register
A structured map of potential harms, including false low-risk classification, credibility bias, privacy exposure, unsafe contact, language-access failures, and automation bias.
Survivor-Centered Design Checklist
A practical review tool for teams considering AI in intake, triage, case notes, communications, or other survivor-support environments.
Governance Guardrails
Clear boundaries defining what AI may support and what should never be automated in domestic violence settings.
Workshop & Discussion Guide
A resource designed to bring domestic violence advocates, technologists, nonprofit leaders, privacy professionals, and Responsible AI practitioners into the same conversation.
WHO THIS PROJECT IS FOR
The purpose is not to design AI for survivors without their perspective.
It is to create the governance conditions necessary for responsible discussion before technology becomes embedded in high-sensitivity support systems.
THE GOAL Build the guardrails before AI enters domestic violence workflows. Define boundaries. Be explicit about what AI may assist with and what it must never decide. Keep humans in the loop. Require trained advocate review before consequential actions. Protect privacy. Minimize sensitive data and design for the possibility of unsafe or monitored communications. Test for bias. Stress-test systems for assumptions involving trauma, language, culture, credibility, and non-linear survivor behavior. Create usable governance tools. Turn principles into risk registers, checklists, case studies, testing methods, and organizational controls. The goal is not to replace care with technology. It is to govern technology with care.
INVITATION TO COLLABORATE
This project is being developed as a practical Responsible AI governance framework for high-sensitivity survivor-support environments.I am interested in collaboration and discussion with domestic violence advocates, trauma-informed practitioners, AI governance professionals, privacy specialists, nonprofit leaders, researchers, designers, and others working at the intersection of technology, safety, human rights, and survivor support.
Interested in contributing to the framework or discussing the project?
CONTACT PIYA MITRA
Responsible Use
This project is designed for governance research, discussion, and collaboration. It is not a clinical, legal, emergency-response, or domestic-violence risk-assessment tool.Any real-world use of AI within survivor-support environments should involve trained advocates, appropriate legal and privacy review, and organization-specific policies and safeguards.Survivors are often misread by people. AI must not be allowed to repeat that harm at scale.