Malware Prototype with Embedded Prompt Injection - AI Case Study
AI Case StudyCheck Point Research identified a prototype malware sample in the wild that contained a prompt injection, which appeared to be designed to manipulate LLM-based malware detectors and/or analysis tools. However, the researchers did not find the prompt injection to be effective on the models they tested. The malware sample, called Skynet, was uploaded to VirusTotal by a user in the Netherlands. It attempts several sa...
Overview
Risk patterns
Patterns found in the case record and its linked vulnerabilities.
- 1Dominant ATLAS tactic. Resource Development appears in 2 case steps.
- 2Multiple attack methods. The case connects to 8 unique AI attack methods.
Procedure timeline
Search the case steps or filter them by attacker goal.
-
Resource Development
Step 1
LLM Prompt Crafting
The bad actor crafted a malicious prompt designed to evade detection.
-
Resource Development
Step 2
Develop Capabilities
The threat actor embedded the prompt injection into a malware sample they called Skynet.
-
Execution
Step 3
Direct
When the LLM-based malware detection or analysis tool interacts with the Skynet malware binary, the prompt is executed.
-
Defense Evasion
Step 4
Evade AI Model
The LLM-based malware detection or analysis tool could be manipulated into not reporting the Skynet binary as malware. Note: The prompt injection was not effective against the LLMs that Check Point Research tested.
-
Defense Evasion The Skynet malware attempts various sandbox evasions.
-
Credential Access
Step 6
Unsecured Credentials
The Skynet malware attempts to access
%HOMEPATH%\.ssh\id_rsa. -
Collection
Step 7
Data from Local System
The Skynet malware attempts to collect
%HOMEPATH%\.ssh\known_hostsandC:/Windows/System32/Drivers/etc/hosts. -
Exfiltration The Skynet malware sets up a Tor proxy to exfiltrate the collected files. Note: The collected files were only printed to stdout and not successfully exfiltrated.
Mitigations
Defenses connected to the attack methods in this case.
AI Telemetry Logging
Implement logging of inputs and outputs of deployed AI models. When deploying AI agents, implement logging of the intermediate steps of agentic actions and decisions, data access and tool use, installation commands, and identity of the agent. Monitoring logs can help to detect security threats and mitigate impacts.
Additionally, having logging enabled can discourage adversaries who want to remain undetected from utilizing AI resources.
Adversarial Input Detection
Detect and block adversarial inputs or atypical queries that deviate from known benign behavior, exhibit behavior patterns observed in previous attacks or that come from potentially malicious IPs. Incorporate adversarial detection algorithms into the AI system prior to the AI model.
Control Access to AI Models and Data at Rest
Establish access controls on internal model registries and limit internal access to production models. Limit access to training data only to approved users.
Deepfake Detection
Apply deepfake detection algorithms against any untrusted or user-provided data, especially in impactful applications such as biometric verification, to block generated content.
Detectors may use a combination of approaches, including:
- AI models trained to differentiate between real and deepfake content.
- Identifying common inconsistencies in deepfake content, such as unnatural facial movements, audio mismatches, or pixel-level artifacts.
- Biometrics analysis, such blinking, eye movements, and microexpressions.
Showing 4 of 9
Source evidence
Original public records and references for this case.
Original source
Original source links
Open the MITRE ATLAS data and public references used for this case study.