ProKYC: Deepfake Tool for Account Fraud Attacks - AI Case Study
AI Case StudyCato CTRL security researchers have identified ProKYC, a deepfake tool being sold to cybercriminals as a method to bypass Know Your Customer (KYC) verification on financial service applications such as cryptocurrency exchanges. ProKYC can create fake identity documents and generate deepfake selfie videos, two key pieces of biometric data used during KYC verification. The tool helps cybercriminals defeat facial rec...
Overview
Risk patterns
Patterns found in the case record and its linked vulnerabilities.
- 1Dominant ATLAS tactic. Resource Development appears in 2 case steps.
- 2Multiple attack methods. The case connects to 8 unique AI attack methods.
Procedure timeline
Search the case steps or filter them by attacker goal.
-
Reconnaissance The bad actor collected user identity information.
-
Resource Development
Step 2
Generative AI
The bad actor paid for the ProKYC tool, created a fake identity document, generated a deepfake selfie video, and replaced a live camera feed with the deepfake video.
-
AI Attack Staging
Step 3
Generate Deepfakes
The bad actor used a mixture of real PII and falsified details with the ProKYC tool to generate a deepfaked identity document.
-
AI Attack Staging
Step 4
Generate Deepfakes
The bad actor used ProKYC tool to generate a deepfake selfie video with the same face as the identity document designed to bypass liveness checks.
-
Resource Development
Step 5
Establish Accounts
The bad actor used the victim information to register an account with a financial services application, such as a cryptocurrency exchange.
-
AI Model Access During identity verification, the financial services application used facial recognition and liveness detection to analyze live video from the user’s camera.
-
Initial Access
Step 7
Evade AI Model
The bad actor used ProKYC to replace the camera feed with the deepfake selfie video. This successfully evaded the KYC verification and allowed the bad actor to authenticate themselves under the false identity.
-
Defense Evasion
Step 8
Impersonation
With an authenticated account under the victim’s identity, the bad actor successfully impersonated the victim and evaded detection.
-
Impact
Step 9
Financial Harm
The bad actor used this access to cause financial harm to the victim.
Mitigations
Defenses connected to the attack methods in this case.
AI Telemetry Logging
Implement logging of inputs and outputs of deployed AI models. When deploying AI agents, implement logging of the intermediate steps of agentic actions and decisions, data access and tool use, installation commands, and identity of the agent. Monitoring logs can help to detect security threats and mitigate impacts.
Additionally, having logging enabled can discourage adversaries who want to remain undetected from utilizing AI resources.
Adversarial Input Detection
Detect and block adversarial inputs or atypical queries that deviate from known benign behavior, exhibit behavior patterns observed in previous attacks or that come from potentially malicious IPs. Incorporate adversarial detection algorithms into the AI system prior to the AI model.
Deepfake Detection
Apply deepfake detection algorithms against any untrusted or user-provided data, especially in impactful applications such as biometric verification, to block generated content.
Detectors may use a combination of approaches, including:
- AI models trained to differentiate between real and deepfake content.
- Identifying common inconsistencies in deepfake content, such as unnatural facial movements, audio mismatches, or pixel-level artifacts.
- Biometrics analysis, such blinking, eye movements, and microexpressions.
Input Restoration
Preprocess all inference data to nullify or reverse potential adversarial perturbations.
Showing 4 of 7
Source evidence
Original public records and references for this case.
Original source
Original source links
Open the MITRE ATLAS data and public references used for this case study.