Bypassing ID.me Identity Verification - AI Case Study
AI Case StudyAn individual filed at least 180 false unemployment claims in the state of California from October 2020 to December 2021 by bypassing ID.me's automated identity verification system. Dozens of fraudulent claims were approved and the individual received at least $3.4 million in payments. The individual collected several real identities and obtained fake driver licenses using the stolen personal details and photos of...
Overview
Risk patterns
Patterns found in the case record and its linked vulnerabilities.
- 1Dominant ATLAS tactic. AI Model Access appears in 1 case steps.
- 2Multiple attack methods. The case connects to 3 unique AI attack methods.
Procedure timeline
Search the case steps or filter them by attacker goal.
-
AI Model Access The individual applied for unemployment assistance with the California Employment Development Department using forged identities, interacting with ID.me's identity verification system in the process. The system extracts content from a photo of an ID, validates the authenticity of the ID using a combination of AI and proprietary methods, then performs facial recognition to match the ID photo to a selfie. [7] The individual identified that the California Employment Development Department relied on a third party service, ID.me, to verify individuals' identities. The ID.me website outlines the steps to verify an identity, including entering personal information, uploading a driver license, and submitting a selfie photo.
-
Initial Access
Step 2
Evade AI Model
The individual collected stolen identities, including names, dates of birth, and Social Security numbers. and used them along with a photo of himself wearing wigs to acquire fake driver's licenses. The individual uploaded forged IDs along with a selfie. The ID.me document verification system matched the selfie to the ID photo, allowing some fraudulent claims to proceed in the application pipeline.
-
Impact
Step 3
Financial Harm
Dozens out of at least 180 fraudulent claims were ultimately approved and the individual received at least $3.4 million in unemployment assistance.
Mitigations
Defenses connected to the attack methods in this case.
AI Telemetry Logging
Implement logging of inputs and outputs of deployed AI models. When deploying AI agents, implement logging of the intermediate steps of agentic actions and decisions, data access and tool use, installation commands, and identity of the agent. Monitoring logs can help to detect security threats and mitigate impacts.
Additionally, having logging enabled can discourage adversaries who want to remain undetected from utilizing AI resources.
Adversarial Input Detection
Detect and block adversarial inputs or atypical queries that deviate from known benign behavior, exhibit behavior patterns observed in previous attacks or that come from potentially malicious IPs. Incorporate adversarial detection algorithms into the AI system prior to the AI model.
Deepfake Detection
Apply deepfake detection algorithms against any untrusted or user-provided data, especially in impactful applications such as biometric verification, to block generated content.
Detectors may use a combination of approaches, including:
- AI models trained to differentiate between real and deepfake content.
- Identifying common inconsistencies in deepfake content, such as unnatural facial movements, audio mismatches, or pixel-level artifacts.
- Biometrics analysis, such blinking, eye movements, and microexpressions.
Input Restoration
Preprocess all inference data to nullify or reverse potential adversarial perturbations.
Showing 4 of 7
Source evidence
Original public records and references for this case.
Original source
Original source links
Open the MITRE ATLAS data and public references used for this case study.