Record summary
A quick snapshot of what this page covers.
Attack context
How this AI attack works in practice.
In addition to the attacks that take place purely in the digital domain, adversaries may also exploit the physical environment for their attacks. If the model is interacting with data collected from the real world in some way, the adversary can influence the model through access to wherever the data is being collected. By modifying the data in the collection process, the adversary can perform modified versions of attacks designed for digital access.
- ATLAS ID
- AML.T0041
- Priority score
- 43
Mitigations
Defenses that may help against this attack.
AML.M0009 - Use Multi-Modal Sensors
Using a variety of sensors can make it more difficult for an attacker with physical access to compromise and produce malicious results.
Case studies
Examples from public reports and exercises.
Backdoor Attack on Deep Learning Models in Mobile Apps
Deep learning models are increasingly used in mobile applications as critical components. Researchers from Microsoft Research demonstrated that many deep learning models deployed in mobile apps are vulnerable to backdoor attacks via "neural payload injection." They conducted an empirical study on real-world mobile deep learning apps collected from Google Play. They identified 54 apps that were vulnerable to attack, including popular security and safety critical applications used for cash recognition, parental control, face authentication, and financial services.
Face Identification System Evasion via Physical Countermeasures
MITRE's AI Red Team demonstrated a physical-domain evasion attack on a commercial face identification service with the intention of inducing a targeted misclassification. This operation had a combination of traditional MITRE ATT&CK techniques such as finding valid accounts and executing code via an API - all interleaved with adversarial ML specific attacks.
Source
Where this page information comes from.
Original source
Original source links
Open the public records and source datasets used for this page.