PromptRiskDBThreat intelligence atlas

Situational Awareness

AI Risk

"AI systems may gain the ability to effectively acquire and use knowledge about itsstatus, its position in the broader environment, its avenues for influencing this environment, and the potentialreactions of the world (including humans) to its actions (Cotra, 2022). ...However, suchknowledge also paves the way for advanced methods of reward hacking, heightened deception/manipulationskills, and an increased propens...

Overview

A source-backed snapshot of this AI risk.

"AI systems may gain the ability to effectively acquire and use knowledge about itsstatus, its position in the broader environment, its avenues for influencing this environment, and the potentialreactions of the world (including humans) to its actions (Cotra, 2022). ...However, suchknowledge also paves the way for advanced methods of reward hacking, heightened deception/manipulationskills, and an increased propensity to chase instrumental subgoals (Ngo et al., 2024)."

Techniques0Attack methods connected to this risk.
Mitigations0Defenses that may help with related attacks.
Records2Source records unified into this concept.

Risk profile

How the MIT AI Risk Repository categorizes this risk.

Domain7. AI System Safety, Failures, & Limitations
Subdomain7.2 > AI possessing dangerous capabilities
Entity2 - AI
Intent1 - Intentional; 2 - Unintentional
Timing3 - Other
CategoryDouble edge components; Dangerous capabilities in AI systems
SubcategorySituational Awareness

Merged risk records

Source records unified into this canonical risk concept.

2 recordsView all →

MITRISK-Ji2023-34.02.01 - Situational Awareness

"AI systems may gain the ability to effectively acquire and use knowledge about itsstatus, its position in the broader environment, its avenues for influencing this environment, and the potentialreactions of the world (including humans) to its actions (Cotra, 2022). ...However, suchknowledge also paves the way for advanced methods of reward hacking, heightened deception/manipulationskills, and an increased propensity to chase instrumental subgoals (Ngo et al., 2024)."

Domain7. AI System Safety, Failures, & LimitationsSubdomain7.2 > AI possessing dangerous capabilitiesSourceAI Alignment: A Comprehensive SurveyYear2023

MITRISK-Maas2023-53.02.01 - Situational awareness

"cases where a large language model displays awareness that it is a model, and it can recognize whether it is currently in testing or deployment;"

Domain7. AI System Safety, Failures, & LimitationsSubdomain7.2 > AI possessing dangerous capabilitiesSourceAdvancing AI Governance: A Literature Review of Problems, Options, and ProposalsYear2023

Related techniques

Attack methods connected to this risk.

View all related techniques →
No linked attack methods. No AI attack method is connected to this risk in the current data.

Mitigations

Defenses that may help with related attacks.

View all mitigations →
No propagated mitigations. No defense is available through the connected attack methods.

Source evidence

Research source for this risk, when available.

View all sources →