APromptRiskDBThreat intelligence atlas
AI Security Technique

Invert AI Model - AI Security Technique

AI models' training data could be reconstructed by exploiting the confidence scores that are available via an inference API. By querying the inference API strategically, adversaries can back out potentially private information embedded within the training data. This could lead to privacy violations if the attacker can reconstruct the data of sensitive features used in the algorithm.

AI Security Techniquefeasible

Record summary

A quick snapshot of what this page covers.

Tactics0Attacker goals connected to this method.
Mitigations3Defenses that may help against this attack.
AI risks0Research-backed risks connected to this topic.

Attack context

How this AI attack works in practice.

ATLAS ID
AML.T0024.001
Priority score
19
Maturity: feasible

Mitigations

Defenses that may help against this attack.

AML.M0024 - AI Telemetry Logging

DeploymentMonitoring and Maintenance
LifecycleDeployment + 1 moreCategoryTechnical - Cyber

Telemetry logging can help identify if sensitive data has been exfiltrated.

AML.M0002 - Passive AI Output Obfuscation

DeploymentML Model Evaluation
LifecycleDeployment + 1 moreCategoryTechnical - ML

Suggested approaches:

  • Restrict the number of results shown
  • Limit specificity of output class ontology
  • Use randomized smoothing techniques
  • Reduce the precision of numerical outputs

AML.M0004 - Restrict Number of AI Model Queries

Business and Data UnderstandingDeployment+1 more
LifecycleBusiness and Data Understanding + 2 moreCategoryTechnical - Cyber

Limit the volume of API queries in a given period of time to regulate the amount and fidelity of potentially sensitive information an attacker can learn.

Case studies

Examples from public reports and exercises.

No case studies found. No public example is connected to this attack in the current data.

Source

Where this page information comes from.