Search Open Technical Databases - AI Security Technique
AI Security TechniqueAdversaries may search for publicly available research and technical documentation to learn how and where AI is used within a victim organization. The adversary can use this information to identify targets for attack, or to tailor an existing attack to make it more effective. Organizations often use open source model architectures trained on additional proprietary data in production. Knowledge of this underlying a...
Overview
A source-backed snapshot of this AI security technique.
Adversaries may search for publicly available research and technical documentation to learn how and where AI is used within a victim organization. The adversary can use this information to identify targets for attack, or to tailor an existing attack to make it more effective. Organizations often use open source model architectures trained on additional proprietary data in production. Knowledge of this underlying architecture allows the adversary to craft more realistic proxy models (Create Proxy AI Model). An adversary can search these resources for publications for authors employed at the victim organization.
Research and technical materials may exist as academic papers published in Journals and Conference Proceedings, or stored in Pre-Print Repositories, as well as Technical Blogs.
Technique details
Identifiers, maturity, and source taxonomy for this technique.
- ATLAS ID
- AML.T0000
- Maturity
- demonstrated
- ATT&CK external ID
- T1596
- Priority score
- 103
Attack flow
How to read the public records connected to this technique.
Impact
Why this technique may deserve attention in the current dataset.
- Evidence leveldemonstrated
- Mapped defenses1 ATLAS mitigation records
- Public examples8 linked case study records
- Research risks0 related MIT AI Risk records above the confidence threshold
- Vulnerabilities0 linked CVE records
Mitigations
Defenses that may help against this attack.
AML.M0000 - Limit Public Release of Information
Limit the connection between publicly disclosed approaches and the data, models, and algorithms used in production.
Case studies
Examples from public reports and exercises.
Exposed ClawdBot Control Interfaces Leads to Credential Access and Execution
A security researcher identified hundreds of exposed ClawdBot control interfaces on the public internet. ClawdBot (now OpenClaw) “is a personal AI assistant you run on your own devices. It answers you on the channels you already use … , plus extension channels. … It can speak and listen on macOS/iOS/Android, and can render a live Canvas you control.”[1] The researcher was able to access credentials to a variety of connected applications via ClawdBot’s configuration file. They were also able to invoke ClawdBot’s skills by prompting it via the chat interface, leading to root access in the container.
The researcher searched Shodan[2] to identify Clawdbot instances exposed on the public internet, some without authentication enabled. The researcher demonstrated that the ClawdBot’s authentication mechanism could be bypassed due to a proxy misconfiguration.
With access to ClawdBot’s control interface, they were then able to access ClawdBot’s configuration, which contained credentials to a variety of other services. Across various exposed instances of ClawdBot, they identified Anthropic API Keys, Telegram Bot Tokens, Slack Oauth Credentials, and Signal Device Linking URIs. The researcher prompted ClawdBot directly via the chat interface, which led to exposure of its system prompt. They were also able to get ClawdBot to execute commands via it’s bash skill, which at least in once instance led to root access in the ClawdBot container.
The researcher noted a broad range of other impacts they could have had with this level of access, including:
- Manipulation of user chat history with the ClawdBot AI agent
- Exfiltration of conversation histories of any connected messaging services
- Impersonation of users by sending messages on their behalf via connected messaging services
References
Attack on Machine Translation Services
Machine translation services (such as Google Translate, Bing Translator, and Systran Translate) provide public-facing UIs and APIs. A research group at UC Berkeley utilized these public endpoints to create a replicated model with near-production state-of-the-art translation quality. Beyond demonstrating that IP can be functionally stolen from a black-box system, they used the replicated model to successfully transfer adversarial examples to the real production services. These adversarial inputs successfully cause targeted word flips, vulgar outputs, and dropped sentences on Google Translate and Systran Translate websites.
Microsoft Edge AI Evasion
The Azure Red Team performed a red team exercise on a new Microsoft product designed for running AI workloads at the edge. This exercise was meant to use an automated system to continuously manipulate a target image to cause the ML model to produce misclassifications.
Botnet Domain Generation Algorithm (DGA) Detection Evasion
The Palo Alto Networks Security AI research team was able to bypass a Convolutional Neural Network based botnet Domain Generation Algorithm (DGA) detector using a generic domain name mutation technique. It is a generic domain mutation technique which can evade most ML-based DGA detection modules. The generic mutation technique evades most ML-based DGA detection modules DGA and can be used to test the effectiveness and robustness of all DGA detection methods developed by security companies in the industry before they is deployed to the production environment.
Showing 4 of 8
Source evidence
Original public records and references for this page.
Original source
Original source links
Open the public records and source datasets used for this page.