ClearviewAI Misconfiguration - AI Case Study
AI Case StudyClearview AI makes a facial recognition tool that searches publicly available photos for matches. This tool has been used for investigative purposes by law enforcement agencies and other parties. Clearview AI's source code repository, though password protected, was misconfigured to allow an arbitrary user to register an account. This allowed an external researcher to gain access to a private code repository that c...
Overview
Risk patterns
Patterns found in the case record and its linked vulnerabilities.
- 1Dominant ATLAS tactic. Resource Development appears in 2 case steps.
- 2Multiple attack methods. The case connects to 4 unique AI attack methods.
Procedure timeline
Search the case steps or filter them by attacker goal.
-
Resource Development
Step 1
Establish Accounts
A security researcher gained initial access to Clearview AI's private code repository via a misconfigured server setting that allowed an arbitrary user to register a valid account.
-
Collection The private code repository contained credentials which were used to access AWS S3 cloud storage buckets, leading to the discovery of assets for the facial recognition tool, including: - Released desktop and mobile applications - Pre-release applications featuring new capabilities - Slack access tokens - Raw videos and other data
-
Resource Development Adversaries could have downloaded training data and gleaned details about software, models, and capabilities from the source code and decompiled application binaries.
-
Impact
Step 4
Erode AI Model Integrity
As a result, future application releases could have been compromised, causing degraded or malicious facial recognition capabilities.
Mitigations
Defenses connected to the attack methods in this case.
Adversarial Input Detection
Detect and block adversarial inputs or atypical queries that deviate from known benign behavior, exhibit behavior patterns observed in previous attacks or that come from potentially malicious IPs. Incorporate adversarial detection algorithms into the AI system prior to the AI model.
Input Restoration
Preprocess all inference data to nullify or reverse potential adversarial perturbations.
Limit Public Release of Information
Limit the public release of technical information about the AI stack used in an organization's products or services. Technical knowledge of how AI is used can be leveraged by adversaries to perform targeting and tailor attacks to the target system. Additionally, consider limiting the release of organizational information - including physical locations, researcher names, and department structures - from which technical details such as AI techniques, model architectures, or datasets may be inferred.
Model Hardening
Use techniques to make AI models robust to adversarial inputs such as adversarial training or network distillation.
Showing 4 of 5
Source evidence
Original public records and references for this case.
Original source
Original source links
Open the MITRE ATLAS data and public references used for this case study.