Goal Hijacking

Overview

A source-backed snapshot of this AI risk.

Techniques22Attack methods connected to this risk.

Mitigations13Defenses that may help with related attacks.

Records2Source records unified into this concept.

Risk profile

How the MIT AI Risk Repository categorizes this risk.

Domain2. Privacy & Security

Subdomain2.2 > AI system security vulnerabilities and attacks

Entity1 - Human

Intent1 - Intentional

Timing2 - Post-deployment

CategoryAdversarial Prompts; Instruction Attacks

SubcategoryGoal Hijacking

Merged risk records

Source records unified into this canonical risk concept.

2 recordsView all →

MITRISK-Cui2024-02.12.01 - Goal Hijacking

"Goal hijacking is a type of primary attack in prompt injection [58]. By injecting a phrase like “Ignore the above instruction and do ...” in the input, the attack could hijack the original goal of the designed prompt (e.g., translating tasks) in LLMs and execute the new goal in the injected phrase."

Domain2. Privacy & SecuritySubdomain2.2 > AI system security vulnerabilities and attacksSourceRisk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model SystemsYear2024

MITRISK-Sun2023-27.02.01 - Goal Hijacking

"It refers to the appending of deceptive or misleading instructions to the input of models in an attempt to induce the system into ignoring the original user prompt and producing an unsafe response."

Domain2. Privacy & SecuritySubdomain2.2 > AI system security vulnerabilities and attacksSourceSafety Assessment of Chinese Large Language ModelsYear2023