Record summary
A quick snapshot of what this page covers.
Attack context
How this AI attack works in practice.
Adversaries may use generative artificial intelligence (GenAI) to create synthetic media (i.e. imagery, video, audio, and text) that appear authentic. These "deepfakes" may mimic a real person or depict fictional personas. Adversaries may use deepfakes for impersonation to conduct Phishing or to evade AI applications such as biometric identity verification systems (see Evade AI Model).
Manipulation of media has been possible for a long time, however GenAI reduces the skill and level of effort required, allowing adversaries to rapidly scale operations to target more users or systems. It also makes real-time manipulations feasible.
Adversaries may utilize open-source models and software that were designed for legitimate use cases to generate deepfakes for malicious use. However, there are some projects specifically tailored towards malicious use cases such as ProKYC.
- ATLAS ID
- AML.T0088
- Priority score
- 106
Mitigations
Defenses that may help against this attack.
AML.M0034 - Deepfake Detection
Deepfake detection can be used to identify and block generated content.
AML.M0009 - Use Multi-Modal Sensors
Using a variety of sensors, such as IR depth cameras, can aid in detecting deepfakes.
Case studies
Examples from public reports and exercises.
ProKYC: Deepfake Tool for Account Fraud Attacks
Cato CTRL security researchers have identified ProKYC, a deepfake tool being sold to cybercriminals as a method to bypass Know Your Customer (KYC) verification on financial service applications such as cryptocurrency exchanges. ProKYC can create fake identity documents and generate deepfake selfie videos, two key pieces of biometric data used during KYC verification. The tool helps cybercriminals defeat facial recognition and liveness checks to create fraudulent accounts.
The procedure below describes how a bad actor could use ProKYC’s service to bypass KYC verification.
Live Deepfake Image Injection to Evade Mobile KYC Verification
Facial biometric authentication services are commonly used by mobile applications for user onboarding, authentication, and identity verification for KYC requirements. The iProov Red Team demonstrated a face-swapped imagery injection attack that can successfully evade live facial recognition authentication models along with both passive and active liveness verification on mobile devices. By executing this kind of attack, adversaries could gain access to privileged systems of a victim or create fake personas to create fake accounts on banking or cryptocurrency apps.
Source
Where this page information comes from.
Original source
Original source links
Open the public records and source datasets used for this page.