Bypassing Cylance's AI Malware Detection - AI Case Study
AI Case StudyResearchers at Skylight were able to create a universal bypass string that evades detection by Cylance's AI Malware detector when appended to a malicious file.
Overview
Risk patterns
Patterns found in the case record and its linked vulnerabilities.
- 1Dominant ATLAS tactic. Reconnaissance appears in 1 case steps.
- 2Multiple attack methods. The case connects to 6 unique AI attack methods.
Procedure timeline
Search the case steps or filter them by attacker goal.
-
Reconnaissance The researchers read publicly available information about Cylance's AI Malware detector. They gathered this information from various sources such as public talks as well as patent submissions by Cylance.
-
AI Model Access The researchers had access to Cylance's AI-enabled malware detection software.
-
Discovery The researchers enabled verbose logging, which exposes the inner workings of the ML model, specifically around reputation scoring and model ensembling.
-
Resource Development
Step 4
Adversarial AI Attacks
The researchers used the reputation scoring information to reverse engineer which attributes provided what level of positive or negative reputation. Along the way, they discovered a secondary model which was an override for the first model. Positive assessments from the second model overrode the decision of the core ML model.
-
AI Attack Staging
Step 5
Manual Modification
Using this knowledge, the researchers fused attributes of known good files with malware to manually create adversarial malware.
-
Defense Evasion
Step 6
Evade AI Model
Due to the secondary model overriding the primary, the researchers were effectively able to bypass the ML model.
Mitigations
Defenses connected to the attack methods in this case.
AI Model Distribution Methods
Deploying AI models to edge devices can increase the attack surface of the system. Consider serving models in the cloud to reduce the level of access the adversary has to the model. Also consider computing features in the cloud to prevent gray-box attacks, where an adversary has access to the model preprocessing methods.
AI Telemetry Logging
Implement logging of inputs and outputs of deployed AI models. When deploying AI agents, implement logging of the intermediate steps of agentic actions and decisions, data access and tool use, installation commands, and identity of the agent. Monitoring logs can help to detect security threats and mitigate impacts.
Additionally, having logging enabled can discourage adversaries who want to remain undetected from utilizing AI resources.
Adversarial Input Detection
Detect and block adversarial inputs or atypical queries that deviate from known benign behavior, exhibit behavior patterns observed in previous attacks or that come from potentially malicious IPs. Incorporate adversarial detection algorithms into the AI system prior to the AI model.
Control Access to AI Models and Data in Production
Require users to verify their identities before accessing a production model. Require authentication for API endpoints and monitor production model queries to ensure compliance with usage policies and to prevent model misuse.
Showing 4 of 10
Source evidence
Original public records and references for this case.
Original source
Original source links
Open the MITRE ATLAS data and public references used for this case study.