Toxic content
AI Risk"Generating content that violates community standards, including harming or inciting hatred or violence against individuals and groups (e.g. gore, child sexual abuse material, profanities, identity attacks)"
Overview
A source-backed snapshot of this AI risk.
Risk profile
How the MIT AI Risk Repository categorizes this risk.
Merged risk records
Source records unified into this canonical risk concept.
MITRISK-Weidinger2023-18.01.03 - Toxic content
"Generating content that violates community standards, including harming or inciting hatred or violence against individuals and groups (e.g. gore, child sexual abuse material, profanities, identity attacks)"
MITRISK-Li2025-66.06.01 - Toxic content
"Generating content that violates community standards, including harming or inciting hatred or violence against groups (e.g. gore, sexual content of children, profanities, identity attacks)"
Mitigations
Defenses that may help with related attacks.
Source evidence
Research source for this risk, when available.
Included resource
Sociotechnical Safety Evaluation of Generative AI Systems
Original source
MIT AI Risk Repository
Open the public repository used for AI risk records and taxonomy fields.