Create Proxy AI Model - AI Security Technique
AI Security TechniqueAdversaries may obtain models to serve as proxies for the target model in use at the victim organization. Proxy models are used to simulate complete access to the target model in a fully offline manner. Adversaries may train models from representative datasets, attempt to replicate models from victim inference APIs, or use available pre-trained models.
Overview
A source-backed snapshot of this AI security technique.
Technique details
Identifiers, maturity, and source taxonomy for this technique.
- ATLAS ID
- AML.T0005
- Maturity
- demonstrated
- Priority score
- 65
Attack flow
How to read the public records connected to this technique.
Impact
Why this technique may deserve attention in the current dataset.
- Evidence leveldemonstrated
- Mapped defenses5 ATLAS mitigation records
- Public examples3 linked case study records
- Research risks0 related MIT AI Risk records above the confidence threshold
- Vulnerabilities0 linked CVE records
Mitigations
Defenses that may help against this attack.
AML.M0019 - Control Access to AI Models and Data in Production
Access controls on models APIs can reduce an adversary's ability to produce an accurate proxy model.
AML.M0001 - Limit Model Artifact Release
Limiting the release of model artifacts can reduce an adversary's ability to create an accurate proxy model.
AML.M0000 - Limit Public Release of Information
Limiting release of technical information about a model and training data can reduce an adversary's ability to create an accurate proxy model.
AML.M0002 - Passive AI Output Obfuscation
Obfuscating model outputs can reduce an adversary's ability to produce an accurate proxy model.
Showing 4 of 5
Case studies
Examples from public reports and exercises.
Confusing Antimalware Neural Networks
Cloud storage and computations have become popular platforms for deploying ML malware detectors. In such cases, the features for models are built on users' systems and then sent to cybersecurity company servers. The Kaspersky ML research team explored this gray-box scenario and showed that feature knowledge is enough for an adversarial attack on ML models.
They attacked one of Kaspersky's antimalware ML models without white-box access to it and successfully evaded detection for most of the adversarially modified malware files.
Evasion of Deep Learning Detector for Malware C&C Traffic
The Palo Alto Networks Security AI research team tested a deep learning model for malware command and control (C&C) traffic detection in HTTP traffic. Based on the publicly available paper by Le et al., we built a model that was trained on a similar dataset as our production model and had similar performance. Then we crafted adversarial samples, queried the model, and adjusted the adversarial sample accordingly until the model was evaded.
Face Identification System Evasion via Physical Countermeasures
MITRE's AI Red Team demonstrated a physical-domain evasion attack on a commercial face identification service with the intention of inducing a targeted misclassification. This operation had a combination of traditional MITRE ATT&CK techniques such as finding valid accounts and executing code via an API - all interleaved with adversarial ML specific attacks.
Source evidence
Original public records and references for this page.
Original source
Original source links
Open the public records and source datasets used for this page.