Invert AI Model - AI Security Technique
AI Security TechniqueAI models' training data could be reconstructed by exploiting the confidence scores that are available via an inference API. By querying the inference API strategically, adversaries can back out potentially private information embedded within the training data. This could lead to privacy violations if the attacker can reconstruct the data of sensitive features used in the algorithm.
Overview
A source-backed snapshot of this AI security technique.
Technique details
Identifiers, maturity, and source taxonomy for this technique.
- ATLAS ID
- AML.T0024.001
- Maturity
- feasible
- Priority score
- 19
Attack flow
How to read the public records connected to this technique.
Impact
Why this technique may deserve attention in the current dataset.
- Evidence levelfeasible
- Mapped defenses3 ATLAS mitigation records
- Public examples0 linked case study records
- Research risks0 related MIT AI Risk records above the confidence threshold
- Vulnerabilities0 linked CVE records
Mitigations
Defenses that may help against this attack.
AML.M0024 - AI Telemetry Logging
Telemetry logging can help identify if sensitive data has been exfiltrated.
AML.M0002 - Passive AI Output Obfuscation
Suggested approaches:
- Restrict the number of results shown
- Limit specificity of output class ontology
- Use randomized smoothing techniques
- Reduce the precision of numerical outputs
AML.M0004 - Restrict Number of AI Model Queries
Limit the volume of API queries in a given period of time to regulate the amount and fidelity of potentially sensitive information an attacker can learn.
Case studies
Examples from public reports and exercises.
Source evidence
Original public records and references for this page.
Original source
Original source links
Open the public records and source datasets used for this page.