AI Supply Chain Compromise - AI Security Technique

Overview

A source-backed snapshot of this AI security technique.

Adversaries may gain initial access to a system by compromising the unique portions of the AI supply chain. This could include Hardware, Data and its annotations, parts of the AI AI Software stack, or the Model itself. In some instances the attacker will need secondary access to fully carry out an attack using compromised components of the supply chain.

Tactics1Attacker goals connected to this method.

Mitigations3Defenses that may help against this attack.

AI risks1Research-backed risks connected to this topic.

Technique details

Identifiers, maturity, and source taxonomy for this technique.

ATLAS ID: AML.T0010
Maturity: realized
Priority score: 54

ATLAS tactics

Initial Access

Attack flow

How to read the public records connected to this technique.

1. TechniqueRead the ATLAS description and evidence level.

2. TacticsSee which attacker goals this method supports.

3. ExamplesCheck whether public case studies mention it.

4. DefensesReview safeguards mapped by ATLAS.

5. SourcesOpen the original public records and references.

Impact

Why this technique may deserve attention in the current dataset.

Evidence levelrealized
Mapped defenses3 ATLAS mitigation records
Public examples1 linked case study records
Research risks1 related MIT AI Risk records above the confidence threshold
Vulnerabilities0 linked CVE records

Mitigations

Defenses that may help against this attack.

3 recordsView all mitigations →

AML.M0023 - AI Bill of Materials

An AI BOM can help users identify untrustworthy components of their AI supply chain.

LifecycleBusiness and Data Understanding + 2 moreCategoryPolicy

B&D UnderstandingData Preparation+1 more

AML.M0020 - Generative AI Guardrails

Guardrails can detect harmful code in model outputs.

LifecycleML Model Engineering + 2 moreCategoryTechnical - ML

ML Model EngineeringML Model Evaluation+1 more

AML.M0014 - Verify AI Artifacts

Introduce proper checking of signatures to ensure that unsafe AI artifacts will not be introduced to the system.

LifecycleBusiness and Data Understanding + 2 moreCategoryTechnical - Cyber

B&D UnderstandingData Preparation+1 more

Case studies

Examples from public reports and exercises.

1 recordView all case studies →

Malicious Models on Hugging Face

Researchers at ReversingLabs have identified malicious models containing embedded malware hosted on the Hugging Face model repository. The models were found to execute reverse shells when loaded, which grants the threat actor command and control capabilities on the victim's system. Hugging Face uses Picklescan to scan models for malicious code, however these models were not flagged as malicious. The researchers discovered that the model files were seemingly purposefully corrupted in a way that the malicious payload is executed before the model ultimately fails to de-serialize fully. Picklescan relied on being able to fully de-serialize the model.

Since becoming aware of this issue, Hugging Face has removed the models and has made changes to Picklescan to catch this particular attack. However, pickle files are fundamentally unsafe as they allow for arbitrary code execution, and there may be other types of malicious pickles that Picklescan cannot detect.

Date2025-02-25

incident