AI Agent Context Poisoning - AI Security Technique
AI Security TechniqueAdversaries may attempt to manipulate the context used by an AI agent's large language model (LLM) to influence the responses it generates or actions it takes. This allows an adversary to persistently change the behavior of the target agent and further their goals. Context poisoning can be accomplished by prompting the an LLM to add instructions or preferences to memory (See Memory) or...
Overview
A source-backed snapshot of this AI security technique.
Adversaries may attempt to manipulate the context used by an AI agent's large language model (LLM) to influence the responses it generates or actions it takes. This allows an adversary to persistently change the behavior of the target agent and further their goals.
Context poisoning can be accomplished by prompting the an LLM to add instructions or preferences to memory (See Memory) or by simply prompting an LLM that uses prior messages in a thread as part of its context (See Thread).
Technique details
Identifiers, maturity, and source taxonomy for this technique.
- ATLAS ID
- AML.T0080
- Maturity
- demonstrated
- Priority score
- 88
Attack flow
How to read the public records connected to this technique.
Impact
Why this technique may deserve attention in the current dataset.
- Evidence leveldemonstrated
- Mapped defenses1 ATLAS mitigation records
- Public examples0 linked case study records
- Research risks13 related MIT AI Risk records above the confidence threshold
- Vulnerabilities0 linked CVE records
Mitigations
Defenses that may help against this attack.
AML.M0031 - Memory Hardening
Memory hardening can help protect LLM memory from manipulation and prevent poisoned memories from executing.
Case studies
Examples from public reports and exercises.
Source evidence
Original public records and references for this page.
Original source
Original source links
Open the public records and source datasets used for this page.