AI Model Tampering via Supply Chain Attack - AI Case Study
AI Case StudyResearchers at Trend Micro, Inc. used service indexing portals and web searching tools to identify over 8,000 misconfigured private container registries exposed on the internet. Approximately 70% of the registries also had overly permissive access controls that allowed write access. In their analysis, the researchers found over 1,000 unique AI models embedded in private container images within these open registrie...
Overview
Risk patterns
Patterns found in the case record and its linked vulnerabilities.
- 1Dominant ATLAS tactic. Initial Access appears in 2 case steps.
- 2Multiple attack methods. The case connects to 9 unique AI attack methods.
Procedure timeline
Search the case steps or filter them by attacker goal.
-
Reconnaissance The Trend Micro researchers used service indexing portals and web searching tools to identify over 8,000 private container registries exposed on the internet. Approximately 70% of the registries had overly permissive access controls, allowing write permissions. The private container registries encompassed both independently hosted registries and registries deployed on Cloud Service Providers (CSPs). The registries were exposed due to some combination of: - Misconfiguration leading to public access of private registry, - Lack of proper authentication and authorization mechanisms, and/or - Insufficient network segmentation and access controls
-
Initial Access The researchers were able to exploit the misconfigured registries to pull container images without requiring authentication. In total, researchers pulled several terabytes of data containing over 20,000 images.
-
Discovery
Step 3
Discover AI Artifacts
The researchers found 1,453 unique AI models embedded in the private container images. Around half were in the Open Neural Network Exchange (ONNX) format.
-
AI Model Access
Step 4
Full AI Model Access
This gave the researchers full access to the models. Models for a variety of use cases were identified, including: - ID Recognition - Face Recognition - Object Recognition - Various Natural Language Processing Tasks
-
Impact With full access to the model(s), an adversary has an organization's valuable intellectual property.
-
Persistence
Step 6
Poison AI Model
With full access to the model weights, an adversary could manipulate the weights to cause misclassifications or otherwise degrade performance.
-
Persistence With full access to the model, an adversary could modify the architecture to change the behavior.
-
Initial Access
Step 8
Container Registry
Because many of the misconfigured container registries allowed write access, the adversary's container image with the manipulated model could be pushed with the same name and tag as the original. This compromises the victim's AI supply chain, where automated CI/CD pipelines could pull the adversary's images.
-
Impact
Step 9
Evade AI Model
Once the adversary's container image is deployed, the model may misclassify inputs due to the adversary's manipulations.
Mitigations
Defenses connected to the attack methods in this case.
AI Model Distribution Methods
Deploying AI models to edge devices can increase the attack surface of the system. Consider serving models in the cloud to reduce the level of access the adversary has to the model. Also consider computing features in the cloud to prevent gray-box attacks, where an adversary has access to the model preprocessing methods.
Adversarial Input Detection
Detect and block adversarial inputs or atypical queries that deviate from known benign behavior, exhibit behavior patterns observed in previous attacks or that come from potentially malicious IPs. Incorporate adversarial detection algorithms into the AI system prior to the AI model.
Code Signing
Enforce binary and application integrity with digital signature verification to prevent untrusted code from executing. Adversaries can embed malicious code in AI software or models. Developers should also cryptographically sign SBOM and AIBOM components that track model or data provenance. Enforcement of code signing can prevent the compromise of the AI supply chain and prevent execution of malicious code.
Control Access to AI Models and Data at Rest
Establish access controls on internal model registries and limit internal access to production models. Limit access to training data only to approved users.
Showing 4 of 10
Source evidence
Original public records and references for this case.
Original source
Original source links
Open the MITRE ATLAS data and public references used for this case study.