Evasion of Deep Learning Detector for Malware C&C Traffic - AI Case Study
AI Case StudyThe Palo Alto Networks Security AI research team tested a deep learning model for malware command and control (C&C) traffic detection in HTTP traffic. Based on the publicly available paper by Le et al., we built a model that was trained on a similar dataset as our production model and had similar performance. Then we crafted adversarial samples, queried the model, and adjusted t...
Overview
Risk patterns
Patterns found in the case record and its linked vulnerabilities.
- 1Dominant ATLAS tactic. AI Attack Staging appears in 3 case steps.
- 2Multiple attack methods. The case connects to 6 unique AI attack methods.
Procedure timeline
Search the case steps or filter them by attacker goal.
-
Reconnaissance
Step 1
Pre-Print Repositories
We identified a machine learning based approach to malicious URL detection as a representative approach and potential target from the paper URLNet: Learning a URL representation with deep learning for malicious URL detection, which was found on arXiv (a pre-print repository).
-
Resource Development
Step 2
Datasets
We acquired a command and control HTTP traffic dataset consisting of approximately 33 million benign and 27 million malicious HTTP packet headers.
-
AI Attack Staging
Step 3
Create Proxy AI Model
We trained a model on the HTTP traffic dataset to use as a proxy for the target model. Evaluation showed a true positive rate of ~ 99% and false positive rate of ~ 0.01%, on average. Testing the model with a HTTP packet header from known malware command and control traffic samples was detected as malicious with high confidence (> 99%).
-
AI Attack Staging
Step 4
Manual Modification
We crafted evasion samples by removing fields from packet header which are typically not used for C&C communication (e.g. cache-control, connection, etc.).
-
AI Attack Staging
Step 5
Verify Attack
We queried the model with our adversarial examples and adjusted them until the model was evaded.
-
Defense Evasion
Step 6
Evade AI Model
With the crafted samples, we performed online evasion of the ML-based spyware detection model. The crafted packets were identified as benign with > 80% confidence. This evaluation demonstrates that adversaries are able to bypass advanced ML detection techniques, by crafting samples that are misclassified by an ML model.
Mitigations
Defenses connected to the attack methods in this case.
Adversarial Input Detection
Detect and block adversarial inputs or atypical queries that deviate from known benign behavior, exhibit behavior patterns observed in previous attacks or that come from potentially malicious IPs. Incorporate adversarial detection algorithms into the AI system prior to the AI model.
Control Access to AI Models and Data at Rest
Establish access controls on internal model registries and limit internal access to production models. Limit access to training data only to approved users.
Control Access to AI Models and Data in Production
Require users to verify their identities before accessing a production model. Require authentication for API endpoints and monitor production model queries to ensure compliance with usage policies and to prevent model misuse.
Deepfake Detection
Apply deepfake detection algorithms against any untrusted or user-provided data, especially in impactful applications such as biometric verification, to block generated content.
Detectors may use a combination of approaches, including:
- AI models trained to differentiate between real and deepfake content.
- Identifying common inconsistencies in deepfake content, such as unnatural facial movements, audio mismatches, or pixel-level artifacts.
- Biometrics analysis, such blinking, eye movements, and microexpressions.
Showing 4 of 10
Source evidence
Original public records and references for this case.
Original source
Original source links
Open the MITRE ATLAS data and public references used for this case study.