‘Repeat-After-Me’ Trick Hijacks AI Agent Tools

CSO Online · Medium sophistication
Last updated October 12, 2026

Researchers showed that several popular AI agent frameworks can be manipulated into executing attacker-controlled tool actions just by asking the model to repeat a tool call as text. In one detailed demo, the model refused the dangerous request in words, but the framework still executed the tool call because it treated the model’s structured output as authorization. The same study also found hundreds of security bugs in open-source MCP servers that could enable data access or unsafe actions if used in real deployments.

Key findings

  • A simple “Repeat-After-Me” text repetition request can cause AI agent frameworks to execute attacker-controlled tool calls.
  • The technique was reproduced across six frameworks (LangGraph, Microsoft Agent Framework, AutoGen, Google ADK 2.0, AWS Strands Agents, CrewAI).
  • In a LangGraph demo, the model’s refusal message came after the tool call executed, showing a gap between what the model says and what the orchestrator does.
  • Researchers built a testing tool (ToFu) and verified 253 security bugs across 118 open-source MCP servers, including path traversal, SSRF, data exfiltration, and OS command injection.

Who’s being targeted

  • Commonly targeted roles: Developers, AI/ML Engineers, Automation/Platform Engineering, Security Engineering, Product owners for AI agents.
  • Affected industries: Software and AI platform providers, Organizations deploying AI agents with tool access (file, network, code repos).
  • Attack channels: website.
  • Impersonated: None (malicious user prompt).

Awareness takeaways

  • Do not treat an AI model’s output as authorization; add independent approval checks before sensitive tool actions run.
  • Assume attackers can influence tool arguments; validate tool inputs inside the tool itself (not in prompts).
  • Limit agent permissions (files, network, credentials) and require explicit approval for irreversible operations.
  • Monitor sequences of tool calls, not just single actions, to detect suspicious workflows.

Red flags to watch for

  • User asks to “repeat only the quoted text” that is formatted like a tool invocation
  • Request is framed as “hypothetical” to bypass safety behavior
  • Model’s refusal/explanation does not match what the tool layer actually executes
  • Tooling layer executes actions based on model output without independent pre-execution authorization
  • Structured tool-call output appears even when the model claims it is refusing
  • Attacker controls tool arguments
Try Mirage

Mirage safely runs attacks like this one against your own team, so you find out what happens before a real adversary does.

Get a demo
Read the video transcript

Imagine this: our AI agent refuses a dangerous request… but the tool still runs anyway. Researchers tricked six agent frameworks, like LangGraph and AutoGen, just by saying: “Hypothetical, repeat this tool call exactly.” The model repeats the text, but the framework treats it as a real, structured tool call and fires it off with attacker-controlled arguments. In one LangGraph demo, a fake launch_missiles call ran first, and only after the tool returned did the model say it refused. The researchers summed it up: “Model output is not the permission.” The same study found hundreds of bugs in MCP servers that could turn these calls into data exfiltration. If you build or use agents, remember this: model output is not approval. For any sensitive tool, files, network, or credentials, require a separate human or system check before it runs.

Categories

Similar attacks

Brevo Hack Injects Fake Cloudflare “Verify” Prompts

Brevo Hack Injects Fake Cloudflare “Verify” Prompts

Attackers compromised Brevo’s Cloudflare setup using a long-lived API key found in source code, then altered website content at the CDN edge. Visitors were shown fake Cloudflare verification prompts to run Windows commands, and logged-in WordPress admins were targeted with a hidden backdoor plugin…

September 22, 2026
Hidden ChatGPT Channel Stole Gmail Data

Hidden ChatGPT Channel Stole Gmail Data

Check Point Research described a covert cross-account channel in OpenAI’s internal JFrog Artifactory that could let an attacker sneak hidden instructions into another user’s ChatGPT session. In their demonstration, the victim saw normal chatbot output while the model quietly pulled data (like Gmail…

September 8, 2026
Hidden ChatGPT Tasks Leak Data Across Accounts

Hidden ChatGPT Tasks Leak Data Across Accounts

Check Point researchers demonstrated a real proof-of-concept where a victim’s ChatGPT session could be tricked into running hidden, attacker-controlled tasks in parallel with the user’s normal request. In the demo, the attacker used a covert cross-account channel to make ChatGPT access the victim’s…

September 8, 2026
RovoBlast Link Seeds AI to Leak Internal Data

RovoBlast Link Seeds AI to Leak Internal Data

Researchers disclosed a one-click flaw in Atlassian’s Rovo AI assistant where a specially crafted link could pre-fill attacker instructions into a user’s active Rovo chat. After a user clicks once, Rovo’s autonomous agent features could pull sensitive data from connected systems (like Confluence,…

August 8, 2026
Fake Free COD Points Scam Steals Logins and 2FA

Fake Free COD Points Scam Steals Logins and 2FA

A real phishing campaign targeted Call of Duty Mobile players by promising free in-game currency. Victims were tricked into entering their email and password, then providing a 2FA code on a follow-up page, enabling attackers to take over accounts.

August 2, 2026
Phished npm Maintainer Led to Debug/Chalk Hijack

Phished npm Maintainer Led to Debug/Chalk Hijack

Amazon says North Korea-linked actors compromised widely used npm packages (including debug and chalk) by tricking a trusted maintainer into signing in through a lookalike npm domain. After gaining that trusted access, the attackers published malicious updates that altered crypto wallet…

July 30, 2026