Discover AI Model Ontology - AI Security Technique
AI Security TechniqueAdversaries may discover the ontology of an AI model's output space, for example, the types of objects a model can detect. The adversary may discovery the ontology by repeated queries to the model, forcing it to enumerate its output space. Or the ontology may be discovered in a configuration file or in documentation about the model. The model ontology helps the adversary understand how the model is being used by t...
Overview
A source-backed snapshot of this AI security technique.
Adversaries may discover the ontology of an AI model's output space, for example, the types of objects a model can detect. The adversary may discovery the ontology by repeated queries to the model, forcing it to enumerate its output space. Or the ontology may be discovered in a configuration file or in documentation about the model.
The model ontology helps the adversary understand how the model is being used by the victim. It is useful to the adversary in creating targeted attacks.
Technique details
Identifiers, maturity, and source taxonomy for this technique.
- ATLAS ID
- AML.T0013
- Maturity
- demonstrated
- Priority score
- 36
Attack flow
How to read the public records connected to this technique.
Impact
Why this technique may deserve attention in the current dataset.
- Evidence leveldemonstrated
- Mapped defenses2 ATLAS mitigation records
- Public examples1 linked case study records
- Research risks0 related MIT AI Risk records above the confidence threshold
- Vulnerabilities0 linked CVE records
Mitigations
Defenses that may help against this attack.
AML.M0002 - Passive AI Output Obfuscation
Suggested approaches:
- Restrict the number of results shown
- Limit specificity of output class ontology
- Use randomized smoothing techniques
- Reduce the precision of numerical outputs
AML.M0004 - Restrict Number of AI Model Queries
Limit the amount of information an attacker can learn about a model's ontology through API queries.
Case studies
Examples from public reports and exercises.
Face Identification System Evasion via Physical Countermeasures
MITRE's AI Red Team demonstrated a physical-domain evasion attack on a commercial face identification service with the intention of inducing a targeted misclassification. This operation had a combination of traditional MITRE ATT&CK techniques such as finding valid accounts and executing code via an API - all interleaved with adversarial ML specific attacks.
Source evidence
Original public records and references for this page.
Original source
Original source links
Open the public records and source datasets used for this page.