JustujuAI, layer by layer.Open in the interactive map

Responding to a compromised agent

What to do when an AI agent has been hijacked or misbehaves: stop it, cut its access, find out what it touched, fix the cause and recover.

Layer 8 · Agents · Run time · Pending expert review, written 2026-11-02

Read first: Agent permissions & sandboxing, The agent loop.

SimpleA plain explanation, one diagram and an analogy, with where the analogy breaks.

Guess first: is switching the agent off enough?

Show the answerNo. Its keys may still work, and what it changed stays changed. Lock it out and check what it did.

Sooner or later an agent will be fooled or broken. Plan for that day before it comes.

First, stop it: pause the agent and switch off its keys and tokens, so it cannot keep acting.

Then look at its log: every tool it called, every file it changed, every message it sent. That shows what it touched.

Find how it was fooled, such as a poisoned web page, a bad tool or a stolen key, and close that door.

Finally, restore what was changed, tell the people affected, and tighten the agent's limits so the same trick fails next time.

Stoppause the agent1Revokeits keys andtokens2Traceread its actionlog3Fixclose the way in4Recoverrestore and tellpeople5
Steps when an agent goes wrong
  1. Stop: pause the agent
  2. Revoke: its keys and tokens
  3. Trace: read its action log
  4. Fix: close the way in
  5. Recover: restore and tell people

Like: A fire drill for a robot employee: know where the off switch is and which doors to lock.
Where it breaks: A fire stops when it runs out of fuel. A hijacked agent keeps acting until someone cuts its access, and it may have copies of keys elsewhere.

How it worksAn interactive demo first, then the mechanism: light maths, pseudo-code and trade-offs.

Prepare. You can only investigate what you recorded: log every tool call with inputs, outputs, the model's context sources and the identity it acted as. Keep a kill switch per agent, and an inventory of each agent's credentials and permissions.

Contain. Pause the agent, revoke or rotate its tokens and keys, disable its tool servers and any scheduled triggers, and quarantine memory or context stores that may hold injected instructions.

Investigate. Reconstruct the sequence from logs: which content carried the injection, which tools ran, what data was read or sent. Check for persistence such as altered configuration, poisoned memory, or new credentials it created.

Recover and learn. Restore changed systems from known-good states, notify affected parties as policy and law require, and remove the leg that made the attack possible. General incident-response practice applies; the agent is one more identity and one more system.

Trade-offs

ResearchWhere the simple picture breaks, open problems and primary sources.

Practice. NIST's revised incident-response guidance places response inside overall risk management, which applies to agents as to any system (NIST SP 800-61r3, 2025).

Agent-specific threats. OWASP's Top 10 for Agentic Applications (December 2025) and its earlier agentic threat list cover memory poisoning, tool misuse and privilege compromise, each of which changes what responders must check (OWASP, 2025; OWASP, Feb 2025).

Cases. A destructive prompt planted through an overscoped build token shipped in an AI coding extension release; the vendor says it was malformed and did not run, revoked the credentials, pulled the version and shipped a clean update (AWS-2025-015).

Open questions. How to log model context without storing too much sensitive data, and how to prove an agent's memory is clean after an incident.

Security on this concept: 2 threats Pending expert review

Attackers run AI agents against their targets Pending expert review

Exposed
  • Coding and computer-use agents that can run scans and exploits
  • Organisations targeted at machine speed
Attacks
  • Splitting an intrusion into small, harmless-looking tasks for an agent
  • Letting the agent do most of the reconnaissance and exploitation, with people approving only key steps
Real case
Incident: AI-orchestrated espionage campaign (2025-11)
Controls
  • Providers: detect and ban misuse; share indicators
  • Defenders: patch fast and watch for bursts of automated activity
  • Use AI on defence too, with a person approving actions
Owner
CISO, with AppSec
Reference
AML.T0124 Autonomous Attack Orchestration

An agent with too much power is told to destroy Pending expert review

Exposed
  • Files, cloud resources and accounts an agent can change
  • Extensions and settings that tell the agent what to do
Attacks
  • Slipping a destructive instruction into an agent's prompt or extension
  • Abusing broad permissions to delete files or cloud resources
Real case
Incident: Wiper prompt planted in an AI coding extension release (AWS says it was malformed and did not run) (2025-07)
Controls
  • Least privilege: give the agent only the tools and scopes this task needs
  • Ask a person before deleting or changing anything important
  • Run agents in a sandbox with backups and an undo path
Owner
AppSec, with Platform/Cloud
Reference
LLM03:2026 Excessive Agency
AML.T0101 Data Destruction via AI Agent Tool Invocation

All layers in the security view

Examples as of 2026-11-10

Sources

  1. NIST SP 800-61 Rev. 3: Incident Response Recommendations and Considerations for Cybersecurity Risk Management (2025-04-03)
  2. OWASP Agentic AI: Threats and Mitigations (2025-02-17)
  3. Security Update for Amazon Q Developer Extension for Visual Studio Code (AWS-2025-015) (2025-07-23)
  4. MITRE ATLAS data, release 2026.10 (the data behind atlas.mitre.org) (2026-10-28)
  5. OWASP Top 10 for Agentic Applications (2026 edition, published December 2025) (2025-12-09)
  6. Disable agent identities in your tenant (Microsoft Entra Agent ID, Microsoft Learn) (ongoing)
  7. Secure autonomous agents with Conditional Access (Microsoft Entra ID, Microsoft Learn) (ongoing)

More in Layer 8 · Agents