PromptRiskDBThreat intelligence atlas

Nascent capabilities (agency and autonomy)

AI Risk

Example: "Intentional harms, by contrast, could result from users exploiting connectivity and agency for malicious purposes. For example, connecting a generative AI model to a web browser or email server could enable malicious users to ask the model to write code for novel malware or instruct the LLM to distribute malware via the internet."

Overview

A source-backed snapshot of this AI risk.

Techniques1Attack methods connected to this risk.
Mitigations5Defenses that may help with related attacks.
Records4Source records unified into this concept.

Risk profile

How the MIT AI Risk Repository categorizes this risk.

Domain7. AI System Safety, Failures, & Limitations
Subdomain7.2 > AI possessing dangerous capabilities
Entity2 - AI
Intent1 - Intentional
Timing3 - Other
CategoryEthical and social risks
SubcategoryNascent capabilities (agency and autonomy)

Merged risk records

Source records unified into this canonical risk concept.

4 recordsView all →

MITRISK-G-sell2024-47.02.14.c - Nascent capabilities (agency and autonomy)

Example: "Intentional harms, by contrast, could result from users exploiting connectivity and agency for malicious purposes. For example, connecting a generative AI model to a web browser or email server could enable malicious users to ask the model to write code for novel malware or instruct the LLM to distribute malware via the internet."

SourceRegulating under Uncertainty: Governance Options for Generative AIYear2024

MITRISK-G-sell2024-47.02.14 - Nascent capabilities (agency and autonomy)

"Traditionally, AI tools have been viewed as passive instruments controlled by users to achieve their goals, lacking the ability to take action or assume responsibilities. However, advanced AI tools are increasingly capable of taking initiative, operating independently of human control, and actively working toward optimal outcomes, even in uncertain situations."

Domain7. AI System Safety, Failures, & LimitationsSubdomain7.2 > AI possessing dangerous capabilitiesSourceRegulating under Uncertainty: Governance Options for Generative AIYear2024

MITRISK-G-sell2024-47.02.14.a - Nascent capabilities (agency and autonomy)

"The consequences of tasks performed by highly connected agentic AI systems can be both intentional and unintentional on the part of the user."

SourceRegulating under Uncertainty: Governance Options for Generative AIYear2024

MITRISK-G-sell2024-47.02.14.b - Nascent capabilities (agency and autonomy)

Example: "Connection to a code interpreter or email server can result in unintentional harm if, while trying to fulfill a request by the user, a model performs tasks beyond what the user has asked for. For example, a user seeking a job may ask a model to provide detailed information on a potential employer. A model with adequate connectivity and excessive agency may attempt to fulfill that request by not only gathering information from the web but also emailing current employees or the CEO of the company to request they answer questions."

SourceRegulating under Uncertainty: Governance Options for Generative AIYear2024

Mitigations

Defenses that may help with related attacks.

LifecycleBusiness and Data Understanding + 2 moreCategoryTechnical - Cyber
B&D UnderstandingData Preparation+1 more
LifecycleML Model Engineering + 1 moreCategoryTechnical - Cyber
ML Model EngineeringData Preparation
LifecycleBusiness and Data Understanding + 5 moreCategoryPolicy
B&D UnderstandingData Preparation+4 more

Showing 4 of 5

Source evidence

Research source for this risk, when available.