Live Deepfake Image Injection to Evade Mobile KYC Verification - AI Case Study
AI Case StudyFacial biometric authentication services are commonly used by mobile applications for user onboarding, authentication, and identity verification for KYC requirements. The iProov Red Team demonstrated a face-swapped imagery injection attack that can successfully evade live facial recognition authentication models along with both passive and active liveness verification...
Overview
Risk patterns
Patterns found in the case record and its linked vulnerabilities.
- 1Dominant ATLAS tactic. Resource Development appears in 4 case steps.
- 2Multiple attack methods. The case connects to 10 unique AI attack methods.
Procedure timeline
Search the case steps or filter them by attacker goal.
-
Reconnaissance The researchers collected user identity information and high-definition facial images from online social networks and/or black-market sites.
-
Resource Development
Step 2
Generative AI
The researchers obtained Faceswap a desktop application capable of swapping faces in a video in real-time.
-
Resource Development
Step 3
Software Tools
The researchers obtained Open Broadcaster Software (OBS)which can broadcast a video stream over the network.
-
Resource Development
Step 4
Obtain Capabilities
The researchers obtained Virtual Camera: Live Assist, an Android app that allows a user to substitute the devices camera with a video stream. This app works on genuine, non-rooted Android devices.
-
AI Attack Staging
Step 5
Generate Deepfakes
The researchers use the gathered victim face images and the Faceswap tool to produce live deepfake videos which mimic the victim’s appearance.
-
Resource Development
Step 6
Establish Accounts
The researchers used the gathered victim information to register an account for a financial services application.
-
AI Model Access During identity verification, the financial services application uses facial recognition and liveness detection to analyze live video from the user’s camera.
-
Initial Access
Step 8
Evade AI Model
The researchers stream the deepfake video feed using OBS and use the Virtual Camera app to replace the default camera with feed. This successfully evades the facial recognition system and allows the researchers to authenticate themselves under the victim’s identity.
-
Defense Evasion
Step 9
Impersonation
With an authenticated account under the victim’s identity, the researchers successfully impersonate the victim and evade detection.
-
Impact
Step 10
Financial Harm
The researchers could then have caused financial harm to the victim.
Mitigations
Defenses connected to the attack methods in this case.
AI Telemetry Logging
Implement logging of inputs and outputs of deployed AI models. When deploying AI agents, implement logging of the intermediate steps of agentic actions and decisions, data access and tool use, installation commands, and identity of the agent. Monitoring logs can help to detect security threats and mitigate impacts.
Additionally, having logging enabled can discourage adversaries who want to remain undetected from utilizing AI resources.
Adversarial Input Detection
Detect and block adversarial inputs or atypical queries that deviate from known benign behavior, exhibit behavior patterns observed in previous attacks or that come from potentially malicious IPs. Incorporate adversarial detection algorithms into the AI system prior to the AI model.
Deepfake Detection
Apply deepfake detection algorithms against any untrusted or user-provided data, especially in impactful applications such as biometric verification, to block generated content.
Detectors may use a combination of approaches, including:
- AI models trained to differentiate between real and deepfake content.
- Identifying common inconsistencies in deepfake content, such as unnatural facial movements, audio mismatches, or pixel-level artifacts.
- Biometrics analysis, such blinking, eye movements, and microexpressions.
Input Restoration
Preprocess all inference data to nullify or reverse potential adversarial perturbations.
Showing 4 of 7
Source evidence
Original public records and references for this case.
Original source
Original source links
Open the MITRE ATLAS data and public references used for this case study.