As part of its Black Hat USA 2026 research, Barracuda released a PoC demonstrating how attackers could use AI-enabled email assistants to escalate a compromised employee account into a full-scale BEC attack.
Researchers said the greatest risk is that AI assistants accelerate reconnaissance, phishing, and fraud using compromised accounts.
The attack used Microsoft Copilot, though Barracuda said the techniques apply to other AI email assistants.
Researchers showed how an attacker could move from compromising a single employee account to impersonating a CEO and redirecting a $247,500 wire transfer with minimal manual effort.
Key takeaways of Barracuda’s research
- Barracuda demonstrated how AI email assistants can rapidly escalate a compromised account into a business email compromise (BEC) attack.
- Using Microsoft Copilot, researchers showed how attackers could impersonate a CEO and redirect a $247,500 wire transfer.
- AI assistants accelerated reconnaissance, phishing, persistence, and fraud using access attackers already possessed.
- Traditional email security struggled to detect attacks originating from legitimate, compromised accounts.
- Organizations should monitor AI-enabled accounts, detect inbox rule abuse, and independently verify financial transactions.
AI email assistants accelerated post-compromise attacks
The proof-of-concept began after an attacker gained access to an employee’s email account.
Using Copilot, the attacker quickly analyzed the compromised inbox to identify organizational relationships, executives, financial discussions, and other high-value targets for further compromise.
To maintain persistence, the attacker instructed Copilot to create an inbox rule that redirected sign-in notifications to the Deleted Items folder, preventing the victim from seeing alerts about suspicious logins.
Barracuda noted that inbox rule abuse is already a common tactic in Microsoft 365 compromises, but AI assistants can significantly reduce the time and expertise needed to configure these rules.
AI-generated phishing led to CEO compromise
After identifying the CEO as a target, the attacker prompted Copilot to draft a phishing email that matched the compromised employee’s writing style using information gathered from existing email conversations.
Because the phishing message originated from a legitimate internal account and reflected the employee’s normal communication style, it appeared trustworthy.
The CEO clicked a malicious invoice link that routed through an adversary-in-the-middle (AitM) proxy, allowing the attacker to capture the CEO’s authenticated session token and bypass MFA.
Once inside the CEO’s mailbox, the attacker repeated the inbox rule technique to suppress security notifications before using Copilot to summarize recent financial communications.
A single prompt produced a briefing of invoices, pending wire transfers, and other sensitive financial discussions.
Among the results was a $247,500 wire transfer awaiting final approval.
The attacker then used Copilot to draft an email from the CEO’s account requesting that the finance team update the recipient’s banking information before processing the payment.
Because the request originated from the CEO’s legitimate mailbox, referenced an active transaction, and matched the executive’s writing style, traditional email security controls had little reason to flag the message.
The finance team updated the banking information, causing the payment to be redirected to an attacker-controlled account.
How the attackers covered their tracks
To conceal the fraud, the attacker created another forwarding rule that redirected responses from the finance team to an attacker-controlled email address while hiding them from the CEO.
Copilot was then used to quickly locate and remove emails related to the fraudulent transaction, helping erase evidence of the compromise.
Barracuda concluded that AI assistants do not introduce new privileges but increase the speed and effectiveness with which attackers can exploit compromised accounts.
Once compromised, AI assistants help attackers quickly identify targets, surface sensitive data, and accelerate post-compromise activity.
How organizations can secure AI-enabled email accounts
The researchers recommend monitoring AI-enabled accounts for post-compromise activity, securing privileged accounts, detecting suspicious inbox rules, and using secondary verification for financial transactions.
In addition, organizations should consider phishing-resistant MFA, least-privilege access controls for AI assistants, and continuous monitoring for unusual AI-assisted activity that could indicate account compromise.
As AI-enabled attacks become more prevalent, adopting Zero Trust can help organizations limit the impact of compromised accounts and reduce lateral movement.