Hackers Could Weaponize Email AI to Impersonate CEOs

Barracuda researchers simulated how an attacker who already compromised one employee mailbox could use the account’s built-in email AI assistant to hide evidence, learn org context, and draft convincing internal phishing emails. In their proof of concept, the attacker used an invoice-themed link to hijack the CEO’s session and then used the CEO mailbox to request a wire transfer change to a new bank account.

Key findings

• Once an email account is compromised, the attacker can use the built-in email AI assistant as a “living off the land” tool.
• The AI assistant can be prompted to create inbox rules to hide security-related emails (stealth/persistence).
• The AI assistant can summarize sensitive conversations and org structure to help the attacker choose believable pretexts.
• Attackers can use the AI assistant to draft internal spearphishing emails in the compromised user’s writing style to increase trust.
• A successful click can route to an adversary-in-the-middle proxy to steal session tokens and bypass MFA.
• After CEO takeover, the attacker can use the CEO AI assistant to identify upcoming payments and craft a bank-change wire request (BEC) that passes authentication checks.

Who’s being targeted

• Commonly targeted roles: Executive leadership (CEO and assistants), Finance / Accounts Payable, All employees (email users), IT / Security operations (mailbox and AI assistant administrators).
• Affected industries: Any organization using email platforms with built-in AI assistants, Corporate finance (wire payments), Executive leadership.
• Attack channels: email, website.
• Impersonated: Employee account holder (attacker using a compromised mailbox), Trusted internal employee (attacker replying from their compromised mailbox), CEO (attacker operating from the CEO’s compromised mailbox).

Awareness takeaways

• Treat internal emails as untrusted when they contain links or payment instructions, even if they come from a real coworker’s mailbox.
• Finance teams should require out-of-band verification for any bank detail change or wire redirection request.
• Monitor for suspicious mailbox rule creation (especially rules that delete or move ‘sign-in’/security emails) as an early sign of account takeover.
• Limit and audit AI-assistant access and logging, because attackers can use assistants for reconnaissance and drafting highly convincing messages in the victim’s style.

Red flags to watch for

• Unexpected inbox rules that delete or move ‘sign-in’ or security alert emails
• Missing security notification emails that coworkers normally receive
• Unexplained configuration changes attributed to the user
• Invoice/confirmation links inserted into an email thread unexpectedly
• Pressure to approve/confirm via a link rather than established finance systems
• Link leads to a login flow or unexpected webpage rather than an internal approved tool
• Any bank-detail change request sent only by email (even from a real executive mailbox)
• Request references a real payment but changes the account at the last minute
• Unusual urgency or bypass of normal vendor/bank-change verification steps