Nascent capabilities (agency and autonomy)
AI RiskExample: "Intentional harms, by contrast, could result from users exploiting connectivity and agency for malicious purposes. For example, connecting a generative AI model to a web browser or email server could enable malicious users to ask the model to write code for novel malware or instruct the LLM to distribute malware via the internet."
Overview
A source-backed snapshot of this AI risk.
Risk profile
How the MIT AI Risk Repository categorizes this risk.
Merged risk records
Source records unified into this canonical risk concept.
MITRISK-G-sell2024-47.02.14.c - Nascent capabilities (agency and autonomy)
Example: "Intentional harms, by contrast, could result from users exploiting connectivity and agency for malicious purposes. For example, connecting a generative AI model to a web browser or email server could enable malicious users to ask the model to write code for novel malware or instruct the LLM to distribute malware via the internet."
MITRISK-G-sell2024-47.02.14 - Nascent capabilities (agency and autonomy)
"Traditionally, AI tools have been viewed as passive instruments controlled by users to achieve their goals, lacking the ability to take action or assume responsibilities. However, advanced AI tools are increasingly capable of taking initiative, operating independently of human control, and actively working toward optimal outcomes, even in uncertain situations."
MITRISK-G-sell2024-47.02.14.a - Nascent capabilities (agency and autonomy)
"The consequences of tasks performed by highly connected agentic AI systems can be both intentional and unintentional on the part of the user."
MITRISK-G-sell2024-47.02.14.b - Nascent capabilities (agency and autonomy)
Example: "Connection to a code interpreter or email server can result in unintentional harm if, while trying to fulfill a request by the user, a model performs tasks beyond what the user has asked for. For example, a user seeking a job may ask a model to provide detailed information on a potential employer. A model with adequate connectivity and excessive agency may attempt to fulfill that request by not only gathering information from the web but also emailing current employees or the CEO of the company to request they answer questions."
Mitigations
Defenses that may help with related attacks.
Showing 4 of 5
Source evidence
Research source for this risk, when available.
Included resource
Regulating under Uncertainty: Governance Options for Generative AI
Original source
MIT AI Risk Repository
Open the public repository used for AI risk records and taxonomy fields.
