Record summary
A quick snapshot of what this page covers.
Attack context
How this AI attack works in practice.
- ATLAS ID
- AML.T0005
- Priority score
- 65
Mitigations
Defenses that may help against this attack.
AML.M0019 - Control Access to AI Models and Data in Production
Access controls on models APIs can reduce an adversary's ability to produce an accurate proxy model.
AML.M0001 - Limit Model Artifact Release
Limiting the release of model artifacts can reduce an adversary's ability to create an accurate proxy model.
AML.M0000 - Limit Public Release of Information
Limiting release of technical information about a model and training data can reduce an adversary's ability to create an accurate proxy model.
AML.M0002 - Passive AI Output Obfuscation
Obfuscating model outputs can reduce an adversary's ability to produce an accurate proxy model.
AML.M0004 - Restrict Number of AI Model Queries
Restricting the number of queries to the model decreases an adversary's ability to replicate an accurate proxy model.
Case studies
Examples from public reports and exercises.
Confusing Antimalware Neural Networks
Cloud storage and computations have become popular platforms for deploying ML malware detectors. In such cases, the features for models are built on users' systems and then sent to cybersecurity company servers. The Kaspersky ML research team explored this gray-box scenario and showed that feature knowledge is enough for an adversarial attack on ML models.
They attacked one of Kaspersky's antimalware ML models without white-box access to it and successfully evaded detection for most of the adversarially modified malware files.
Evasion of Deep Learning Detector for Malware C&C Traffic
The Palo Alto Networks Security AI research team tested a deep learning model for malware command and control (C&C) traffic detection in HTTP traffic. Based on the publicly available paper by Le et al., we built a model that was trained on a similar dataset as our production model and had similar performance. Then we crafted adversarial samples, queried the model, and adjusted the adversarial sample accordingly until the model was evaded.
Face Identification System Evasion via Physical Countermeasures
MITRE's AI Red Team demonstrated a physical-domain evasion attack on a commercial face identification service with the intention of inducing a targeted misclassification. This operation had a combination of traditional MITRE ATT&CK techniques such as finding valid accounts and executing code via an API - all interleaved with adversarial ML specific attacks.
Source
Where this page information comes from.
Original source
Original source links
Open the public records and source datasets used for this page.