APromptRiskDBThreat intelligence atlas
AI Case Study

Camera Hijack Attack on Facial Recognition System - AI Case Study

This type of camera hijack attack can evade the traditional live facial recognition authentication model and enable access to privileged systems and victim impersonation. Two individuals in China used this attack to gain access to the local government's tax system. They created a fake shell company and sent invoices via tax system to supposed clients. The individuals started this scheme in 2018 and were able to fr...

View attack chainRead summary
IncidentShanghai government tax office's facial recognition serviceTwo individualsResource DevelopmentReconnaissanceAI Model Access

Overview

Case steps8Steps described in the case record.
Techniques8Attack methods mentioned in the case steps.
Linked CVEs0Known vulnerabilities mentioned in the record.

Risk patterns

Patterns found in the case record and its linked vulnerabilities.

  • 1Dominant ATLAS tactic. Resource Development appears in 4 case steps.
  • 2Multiple attack methods. The case connects to 8 unique AI attack methods.

Procedure timeline

Search the case steps or filter them by attacker goal.

Resource Development4Reconnaissance1AI Model Access1Initial Access1Impact1
  7. Initial Access

    The attackers successfully evaded the face recognition system. This allowed the attackers to impersonate the victim and verify their identity in the tax system.

  8. Impact

    The attackers used their privileged access to the tax system to send invoices to supposed clients and further their fraud scheme.

Related CVEs

Known software flaws mentioned in the case record.

No related CVEs found for this case. Built from MITRE ATLAS case study records and listed case steps.

Mitigations

Defenses connected to the attack methods in this case.

AI Telemetry Logging

Implement logging of inputs and outputs of deployed AI models. When deploying AI agents, implement logging of the intermediate steps of agentic actions and decisions, data access and tool use, installation commands, and identity of the agent. Monitoring logs can help to detect security threats and mitigate impacts. Additionally, having logging enabled can discourage adversaries who want to remain undetected from utilizing AI resources.

Adversarial Input Detection

Detect and block adversarial inputs or atypical queries that deviate from known benign behavior, exhibit behavior patterns observed in previous attacks or that come from potentially malicious IPs. Incorporate adversarial detection algorithms into the AI system prior to the AI model.

Deepfake Detection

Apply deepfake detection algorithms against any untrusted or user-provided data, especially in impactful applications such as biometric verification, to block generated content. Detectors may use a combination of approaches, including: - AI models trained to differentiate between real and deepfake content. - Identifying common inconsistencies in deepfake content, such as unnatural facial movements, audio mismatches, or pixel-level artifacts. - Biometrics analysis, such blinking, eye movements, and microexpressions.

Input Restoration

Preprocess all inference data to nullify or reverse potential adversarial perturbations.

Model Hardening

Use techniques to make AI models robust to adversarial inputs such as adversarial training or network distillation.

Use Ensemble Methods

Use an ensemble of models for inference to increase robustness to adversarial inputs. Some attacks may effectively evade one model or model family but be ineffective against others.

Use Multi-Modal Sensors

Incorporate multiple sensors to integrate varying perspectives and modalities to avoid a single point of failure susceptible to physical attacks.

Sources

Original public records and references for this case.