Record summary
A quick snapshot of what this page covers.
Attack context
How this AI attack works in practice.
Adversaries may infer the membership of a data sample or global characteristics of the data in its training set, which raises privacy concerns. Some strategies make use of a shadow model that could be obtained via Train Proxy via Replication, others use statistics of model prediction scores.
This can cause the victim model to leak private information, such as PII of those in the training set or other forms of protected IP.
- ATLAS ID
- AML.T0024.000
- Priority score
- 19
Mitigations
Defenses that may help against this attack.
AML.M0024 - AI Telemetry Logging
Telemetry logging can help identify if sensitive data has been exfiltrated.
AML.M0002 - Passive AI Output Obfuscation
Suggested approaches:
- Restrict the number of results shown
- Limit specificity of output class ontology
- Use randomized smoothing techniques
- Reduce the precision of numerical outputs
AML.M0004 - Restrict Number of AI Model Queries
Limit the volume of API queries in a given period of time to regulate the amount and fidelity of potentially sensitive information an attacker can learn.
Case studies
Examples from public reports and exercises.
Source
Where this page information comes from.
Original source
Original source links
Open the public records and source datasets used for this page.