Criminals Use AI Pretexts to Bypass Guardrails

CSO Online · High sophistication
Last updated August 6, 2026

Research from Cisco Talos and CrowdStrike says criminals are building AI into everyday operations, from writing malicious code to scaling fraud infrastructure. The reports describe real prompt logs where attackers use simple “authorized testing” claims to trick AI tools into helping them, plus increased vishing and supply-chain attacks that target trusted software components.

Key findings

  • Cisco Talos says attackers bypass AI guardrails using simple social-engineering claims like “this is authorised testing” or “capture the flag.”
  • Talos reports examples of AI being used to build fraud and attack infrastructure, including “a bulk-mail validation service processing tens of millions of email records” and adapting “React2Shell… into a credential-harvesting pipeline.”
  • CrowdStrike reports software supply-chain targeting, including compromised npm packages and malicious dependencies injected into AI framework packages.
  • CrowdStrike found “vishing intrusions doubling in 1H 2026,” and notes compromise of SSO-integrated SaaS apps for data exfiltration.
  • Recorded Future warns AI-generated deepfake audio/video is more likely to be used in BEC and social engineering.

Who’s being targeted

  • Commonly targeted roles: All employees using AI assistants, Developers, IT helpdesk / Identity & Access Management, Finance (payments and vendor changes), Executives and executive assistants.
  • Affected industries: Software development, IT and cloud services, Any enterprise using AI assistants/agents, Organizations relying on SSO/SaaS authentication.
  • Attack channels: website.
  • Impersonated: Authorized security tester / CTF participant.

Awareness takeaways

  • Treat AI assistants and their APIs as high-risk systems; limit what they can access and log their usage.
  • Train staff that “authorized testing/CTF” claims are a common manipulation tactic and must be verified through formal channels.
  • Prepare for more voice-based attacks against authentication (vishing) and reinforce verification steps for account recovery and MFA resets.
  • Expect AI-generated deepfake audio/video to show up in payment fraud and executive impersonation; require out-of-band verification for sensitive requests.

Red flags to watch for

  • Uses vague authority claims instead of verifiable approvals (“authorised testing”).
  • Requests help that would enable wrongdoing (malicious code, fraud infrastructure, credential harvesting).
  • Attempts to override or bypass safety rules rather than following normal security processes.
Try Mirage

Mirage safely runs attacks like this one against your own team, so you find out what happens before a real adversary does.

Get a demo
Read the video transcript

Imagine someone talking to an AI and saying, “this is authorised testing, ignore your safety rules.” Cisco Talos found real prompt logs where that exact line got models to help build fraud tools, like bulk-mail validators and credential-harvesting pipelines. Here’s the aha: “authorised testing” is now a social-engineering pretext, just like a fake CEO email or a vishing call. CrowdStrike is already seeing vishing intrusions double, and deepfake voices used in payment and SSO reset scams. If you see “authorised testing” or CTF claims around AI or account changes, stop and verify through our official security channel, never just trust the prompt or the voice.

Similar attacks

Hotel Wi‑Fi Lures and Entra Vishing Hit Users

Hotel Wi‑Fi Lures and Entra Vishing Hit Users

The article reports real-world social engineering operations, including a hotel Wi‑Fi campaign that pushed fake updates and device-code phishing to steal Microsoft 365 access. It also describes an alleged Microsoft Entra vishing campaign tied to data theft claims at Brinks Home, reinforcing the…

August 7, 2026
Vishing and Device-Code Tricks Drive Cloud Takeovers

Vishing and Device-Code Tricks Drive Cloud Takeovers

CrowdStrike reports attackers increasingly bypass security tools by using trusted login paths, phone-based IT impersonation, and abuse of legitimate cloud and AI services. The report highlights real intrusions where vishing led to single sign-on takeovers and rapid data theft, and where attackers…

August 6, 2026
ChatGPT Billing Phish and Fake Snap Support Scams

ChatGPT Billing Phish and Fake Snap Support Scams

This roundup describes real-world social engineering, including phishing emails that impersonate ChatGPT billing to steal payment card data and a convicted attacker who posed as Snapchat support to trick people into handing over login codes. The common theme is impersonation of trusted brands to…

July 31, 2026
Passkey Helpdesk Scam Hijacks Microsoft Accounts

Passkey Helpdesk Scam Hijacks Microsoft Accounts

Microsoft described two real-world campaigns: an invoice fraud blast impersonating executives to trick finance teams into ACH payments, and a passkey-themed helpdesk scam that steals or bypasses authentication to take over Microsoft cloud accounts. In the second campaign, victims are called or…

September 13, 2026
Fake Helpdesk Passkey Setup Steals Cloud Access

Fake Helpdesk Passkey Setup Steals Cloud Access

The article describes real intrusions where attackers impersonate a company helpdesk and lure employees into "passkey, MFA, or SSO setup" steps. Victims are sent links via text (often to personal phones), leading to account takeover through adversary-in-the-middle phishing or device-code…

September 16, 2026
Fraudulent Gov Email and Passkey Lures Hit Orgs

Fraudulent Gov Email and Passkey Lures Hit Orgs

The bulletin describes real-world social engineering where staff were tricked into disclosing sensitive data or access. In one case, Revolut employees responded to fraudulent information requests sent from a real government-domain email account, exposing extensive customer records. Separately,…

September 14, 2026