If your company runs an LLM gateway or a self-hosted AI app, it probably stores API keys for several model providers. According to Amazon, that is exactly what attackers' AI agents are looking for: they scan the internet for exposed AI apps, pull out the keys, test that they still work, and add them to a shared pool. The unsettling part is that none of this needs a new trick. It only needs you to have forgotten a basic setting.
What Amazon caught on its decoy servers
Amazon's threat intelligence team runs a network of decoy sensors called MadPot, set up to look like popular LLM services. From October 3 to early November 2026, attacker agents sent requests to it and, by accident, included the whole campaign history: the operator's instructions and the findings the agents reported. Amazon published its write-up on November 13, and The Hacker News covered it on November 16.
The records point to ARTEX, an open-source AI penetration testing framework (Amazon has moderate confidence in that attribution). Targets include LLM gateways and proxies such as One API, New API and LiteLLM, and AI app builders such as Dify, FastGPT, Langflow, RAGFlow, Open WebUI, LobeChat and Ollama. They are attractive because one weak admin page can expose the keys of many provider accounts at once.
One caveat matters: Amazon did not observe those attacks as they happened and cannot verify the outcomes the agents reported. The records show what the attackers were going after.
Familiar doors, left open
The playbook in the records lists weaknesses most development teams have met before:
- Default admin credentials, such as LiteLLM's default master key.
- Unfinished installs that an outsider can take over.
- Admin and provider-settings endpoints that need no login.
- SSRF through URL-fetch features, used to reach cloud metadata services.
- Secrets left in client-side JavaScript, source maps, debug routes and environment files.
To find leaked keys, the agents also searched GitHub, npm and PyPI packages, container image layers, notebooks and Telegram channels. Amazon's own conclusion is blunt: the techniques are not new, AI just makes the attacker faster and more persistent.
A separate case from Cloudflare shows the consequences. An attacker used an account-level API token from a CI/CD workflow, with no IP restriction or rate limit, and sent 400 database dump requests within an hour. A common root cause across several incidents was static CI/CD tokens that never expire.
Seven checks for your engineering team this week
These apply to any system that calls an AI provider, including internal chatbots and RAG apps.
1. Look for keys in browser-side code
After building a Next.js app, scan the static output:
grep -rEn "(sk-|AKIA|api[_-]?key)" .next/static | head
find .next/static -name "*.map"Environment variables only reach the browser when prefixed with NEXT_PUBLIC_, so never put an AI key in one of those. Call the AI provider from a server-side route. If you find .map files in production, review the productionBrowserSourceMaps setting.
2. Probe for sensitive files from outside
for p in /.env /.git/config /debug /api/config; do
curl -s -o /dev/null -w "$p %{http_code}\n" https://your-domain$p
doneYou want 404 or 403. A 200 needs fixing immediately.
3. Replace every default credential
For self-hosted LiteLLM, Dify, Open WebUI or Langflow, confirm the master key and admin account have unique values and that the setup wizard was finished. A half-completed install is an open invitation.
4. See which ports face the internet
sudo ss -tlnp
sudo ufw status verbose
docker ps --format "table {{.Names}}\t{{.Ports}}"A port bound to 0.0.0.0:xxxx listens on every interface. Admin and inference services belong behind an authenticated reverse proxy or a VPN, not exposed directly.
5. Limit keys and tokens
Give each AI provider key a spending cap and minimal permissions. CI/CD tokens should be short-lived, restricted by IP and narrow in scope; prefer temporary credentials over static tokens.
6. Block SSRF in URL-fetch features
If your app lets users enter a URL to import content (common in RAG document ingestion), reject internal address ranges and cloud metadata addresses before making the request.
7. Alert on spend and unusual usage
Amazon recommends watching for misuse through unexpected spending or unusual usage. A sudden jump in the AI bill is often the earliest sign a key has been taken.
If you suspect a key leaked, revoke it and issue a new one first, and investigate afterwards. Deleting it from the code is not enough, because Git history keeps it.
Related reading
This topic follows on from a few earlier posts: API key leaks in AI agent logs, the exposed ARTEX tool and 800+ servers without passwords, and GitHub secret scanning updates.
Start by listing every place that holds an AI key: apps, LLM gateways, CI/CD pipelines, container images. If you want a third party to review that architecture, get in touch with the Besoft team and we can look at it together.
Sources
- Amazon Security Blog, Amazon Threat Intelligence observes threat actors running cybercrime campaigns with ARTEX AI agents, Nov 13, 2026: https://aws.amazon.com/blogs/security/amazon-threat-intelligence-observes-threat-actors-running-cybercrime-campaigns-with-artex-ai-agents/
- The Hacker News, Amazon Uncovers AI Agent Campaign Targeting LiteLLM, Dify, and Other Apps for API Keys, Nov 16, 2026: https://thehackernews.com/2026/11/amazon-uncovers-ai-agent-campaign.html
- Cloudflare Blog, Adapting to AI penetration testing orchestrators, Nov 13, 2026: https://blog.cloudflare.com/adapting-to-ai-penetration-testing-orchestrators/






