Record summary
A quick snapshot of what this page covers.
Attack context
How this AI attack works in practice.
Adversaries may discover the ontology of an AI model's output space, for example, the types of objects a model can detect. The adversary may discovery the ontology by repeated queries to the model, forcing it to enumerate its output space. Or the ontology may be discovered in a configuration file or in documentation about the model.
The model ontology helps the adversary understand how the model is being used by the victim. It is useful to the adversary in creating targeted attacks.
- ATLAS ID
- AML.T0013
- Priority score
- 36
Mitigations
Defenses that may help against this attack.
AML.M0002 - Passive AI Output Obfuscation
Suggested approaches:
- Restrict the number of results shown
- Limit specificity of output class ontology
- Use randomized smoothing techniques
- Reduce the precision of numerical outputs
AML.M0004 - Restrict Number of AI Model Queries
Limit the amount of information an attacker can learn about a model's ontology through API queries.
Case studies
Examples from public reports and exercises.
Face Identification System Evasion via Physical Countermeasures
MITRE's AI Red Team demonstrated a physical-domain evasion attack on a commercial face identification service with the intention of inducing a targeted misclassification. This operation had a combination of traditional MITRE ATT&CK techniques such as finding valid accounts and executing code via an API - all interleaved with adversarial ML specific attacks.
Source
Where this page information comes from.
Original source
Original source links
Open the public records and source datasets used for this page.