Compromised PyTorch Dependency Chain - AI Case Study
AI Case StudyLinux packages for PyTorch's pre-release version, called Pytorch-nightly, were compromised from December 25 to 30, 2022 by a malicious binary uploaded to the Python Package Index (PyPI) code repository. The malicious binary had the same name as a PyTorch dependency and the PyPI package manager (pip) installed this malicious package instead of the legitimate one. This supply chain attack, also known as "dependency...
Overview
Risk patterns
Patterns found in the case record and its linked vulnerabilities.
- 1Dominant ATLAS tactic. Initial Access appears in 1 case steps.
- 2Multiple attack methods. The case connects to 3 unique AI attack methods.
Procedure timeline
Search the case steps or filter them by attacker goal.
-
Initial Access
Step 1
AI Software
A malicious dependency package named
torchtritonwas uploaded to the PyPI code repository with the same package name as a package shipped with the PyTorch-nightly build. This malicious package contained additional code that uploads sensitive data from the machine. The malicioustorchtritonpackage was installed instead of the legitimate one because PyPI is prioritized over other sources. See more details at this GitHub issue. -
Collection
Step 2
Data from Local System
The malicious package surveys the affected system for basic fingerprinting info (such as IP address, username, and current working directory), and steals further sensitive data, including: - nameservers from
/etc/resolv.conf- hostname fromgethostname()- current username fromgetlogin()- current working directory name fromgetcwd()- environment variables -/etc/hosts-/etc/passwd- the first 1000 files in the user's$HOMEdirectory -$HOME/.gitconfig-$HOME/.ssh/*. -
Exfiltration All gathered information, including file contents, is uploaded via encrypted DNS queries to the domain
*[dot]h4ck[dot]cfd, using the DNS serverwheezy[dot]io.
Mitigations
Defenses connected to the attack methods in this case.
Code Signing
Enforce binary and application integrity with digital signature verification to prevent untrusted code from executing. Adversaries can embed malicious code in AI software or models. Developers should also cryptographically sign SBOM and AIBOM components that track model or data provenance. Enforcement of code signing can prevent the compromise of the AI supply chain and prevent execution of malicious code.
Control Access to AI Models and Data at Rest
Establish access controls on internal model registries and limit internal access to production models. Limit access to training data only to approved users.
Use Ensemble Methods
Use an ensemble of models for inference to increase robustness to adversarial inputs. Some attacks may effectively evade one model or model family but be ineffective against others.
Source evidence
Original public records and references for this case.
Original source
Original source links
Open the MITRE ATLAS data and public references used for this case study.