APromptRiskDBThreat intelligence atlas
AI Security Technique

Exfiltration via AI Inference API - AI Security Technique

Adversaries may exfiltrate private information via AI Model Inference API Access. AI Models have been shown leak private information about their training data (e.g. Infer Training Data Membership, Invert AI Model). The model itself may also be extracted (Extract AI Model) for the purposes of [AI Intellec...

View mitigationsRead context
AI Security TechniquerealizedExfiltration

Record summary

A quick snapshot of what this page covers.

Tactics1Attacker goals connected to this method.
Mitigations3Defenses that may help against this attack.
AI risks6Research-backed risks connected to this topic.

Attack context

How this AI attack works in practice.

Adversaries may exfiltrate private information via AI Model Inference API Access. AI Models have been shown leak private information about their training data (e.g. Infer Training Data Membership, Invert AI Model). The model itself may also be extracted (Extract AI Model) for the purposes of AI Intellectual Property Theft.

Exfiltration of information relating to private training data raises privacy concerns. Private training data may include personally identifiable information, or other protected data.

ATLAS ID
AML.T0024
Priority score
69
Maturity: realized
Exfiltration

Mitigations

Defenses that may help against this attack.

AML.M0024 - AI Telemetry Logging

DeploymentMonitoring and Maintenance
LifecycleDeployment + 1 moreCategoryTechnical - Cyber

Telemetry logging can help identify if sensitive data has been exfiltrated.

AML.M0004 - Restrict Number of AI Model Queries

Business and Data UnderstandingDeployment+1 more
LifecycleBusiness and Data Understanding + 2 moreCategoryTechnical - Cyber

Limit the volume of API queries in a given period of time to regulate the amount and fidelity of potentially sensitive information an attacker can learn.

Case studies

Examples from public reports and exercises.

No case studies found. No public example is connected to this attack in the current data.

Related risks

Research-backed risks connected to this topic.

Attacking LLMs via Additional Modalities a

Confidence: 0.67
Domain2. Privacy & SecuritySubdomain2.2 > AI system security vulnerabilities and attacks

"LLMs can now process modalities other than text, e.g. images or video frames (OpenAI, 2023c; Gemini Team, 2023). Several studies show that gradient-based attacks on multimodal models are easy and effective (Carlini e...

Inference Attacks

Confidence: 0.67
Domain2. Privacy & SecuritySubdomain2.2 > AI system security vulnerabilities and attacks

"Inference attacks [150] include membership inference attacks, property inference attacks, and data reconstruction attacks. These attacks allow an adversary to infer the composition or property information of the trai...

Adversarial AI: Data and Model Exfiltration Attacks

Confidence: 0.67
Domain2. Privacy & SecuritySubdomain2.1 > Compromise of privacy by leaking or correctly inferring sensitive information

"Other forms of abuse can include privacy attacks that allow adversaries to exfiltrate or gain knowledge of the private training data set or other valuable assets. For example, privacy attacks such as membership infer...

Jailbreak of a multimodal model

Confidence: 0.67
Domain2. Privacy & SecuritySubdomain2.2 > AI system security vulnerabilities and attacks

"Current generation multimodal (e.g., vision and language) GPAI models are vulnerable to adversarial jailbreak attacks. These attacks can be used to automatically induce a model to produce an arbitrary or specific out...

Model extraction

Confidence: 0.67
Domain2. Privacy & SecuritySubdomain2.2 > AI system security vulnerabilities and attacks

"Data Exfiltration goes beyond revealing private information, and involves illicitly obtaining the training data used to build a model that may be sensitive or proprietary. Model Extraction is the same attack, only di...

Data exfiltration

Confidence: 0.67
Domain2. Privacy & SecuritySubdomain2.2 > AI system security vulnerabilities and attacks

"Data Exfiltration goes beyond revealing private information, and involves illicitly obtaining the training data used to build a model that may be sensitive or proprietary. Model Extraction is the same attack, only di...

Related CVEs

Known software flaws linked to this context.

No related CVEs. No software flaw is connected to this attack in the current data.

Source

Where this page information comes from.