PromptRiskDBThreat intelligence atlas

Harm to individuals through fake content

AI Risk

"General- purpose AI systems can be used to increase the scale and sophistication of scams and fraud, for example through general- purpose AI- enhanced ‘phishing’ attacks. General- purpose AI can be used to generate fake compromising content featuring individuals without their consent, posing threats to individual privacy and reputation."

Overview

A source-backed snapshot of this AI risk.

Techniques1Attack methods connected to this risk.
Mitigations0Defenses that may help with related attacks.
Records2Source records unified into this concept.

Risk profile

How the MIT AI Risk Repository categorizes this risk.

Domain4. Malicious Actors & Misuse
Subdomain4.3 > Fraud, scams, and targeted manipulation
Entity1 - Human
Intent1 - Intentional
Timing2 - Post-deployment
CategoryMalicious Use Risks; Risks from malicious use
SubcategoryHarm to individuals through fake content

Merged risk records

Source records unified into this canonical risk concept.

2 recordsView all →

MITRISK-Bengio2024-49.01.01 - Harm to individuals through fake content

"General- purpose AI systems can be used to increase the scale and sophistication of scams and fraud, for example through general- purpose AI- enhanced ‘phishing’ attacks. General- purpose AI can be used to generate fake compromising content featuring individuals without their consent, posing threats to individual privacy and reputation."

Domain4. Malicious Actors & MisuseSubdomain4.3 > Fraud, scams, and targeted manipulationSourceInternational Scientific Report on the Safety of Advanced AIYear2024

MITRISK-Bengio2025-60.01.01 - Harm to individuals through fake content

"Malicious actors can use general- purpose AI to generate fake content that harms individuals in a targeted way. For example, they can use such fake content for scams, extortion, psychological manipulation, generation of non- consensual intimate imagery (NCII) and child sexual abuse material (CSAM), or targeted sabotage of individuals and organisations."

Domain4. Malicious Actors & MisuseSubdomain4.3 > Fraud, scams, and targeted manipulationSourceInternational AI Safety Report 2025Year2025

Related techniques

Attack methods connected to this risk.

1 recordView all related techniques →

Mitigations

Defenses that may help with related attacks.

View all mitigations →
No propagated mitigations. No defense is available through the connected attack methods.

Source evidence

Research source for this risk, when available.

View all sources →