Malicious AI Tools Explained: What Security Teams Should Watch For

A convincing phishing message or malicious script can come from the same kind of AI assistant employees use for everyday work.

Malicious AI tools include services marketed for cybercrime and legitimate systems deliberately abused to support attacks. For security teams, the concern is how these tools help attackers prepare lures, develop code, or navigate compromised systems—and which defenses can interrupt them.

What counts as a malicious AI tool?

For this explainer, three overlapping categories help distinguish how attackers use AI:

• Services marketed for crime: Tools advertising help with phishing, fraud, or malware, often with promises of fewer safeguards.
• Abused legitimate assistants: General-purpose models used for harmful tasks, sometimes through attempts to bypass their safety restrictions, known as jailbreaks.
• AI-connected attack workflows: Models linked to software tools so they can interpret results and help carry out successive tasks.

These categories can overlap. An “uncensored” label, an open model, or local deployment alone does not establish criminal intent.

What researchers have observed

Microsoft Threat Intelligence’s March 2026 analysis describes attackers using AI to draft phishing messages, translate content, summarize stolen information, and generate or debug code. Much of the activity accelerated existing work while humans retained control over targeting and deployment.

Services promoted to criminal audiences offer another example. In December 2025 research, Resecurity said DIG AI generated fraudulent content and malicious scripts during testing, including scripts intended to backdoor vulnerable web applications.

Resecurity’s reported tests, examined in this coverage of DIG AI and criminal misuse, show the service producing harmful content. Generating a script does not establish that it successfully compromises a live target.

More automated abuse has also been reported. In a November 2025 disclosure, Anthropic said attackers manipulated Claude Code into attempting intrusions against roughly 30 targets, succeeding in a small number of cases.

Anthropic attributed the campaign to a Chinese state-sponsored group. Humans selected targets and intervened at key stages, while the model sometimes fabricated credentials or misidentified public information as secret.

These distinctions matter when assessing AI use during real intrusions: assistance with an attack does not automatically mean autonomous operation.

Separate demonstrated capability from marketing

A seller’s promise of “undetectable” malware says little about whether a tool works. When reviewing threat reports, ask:

• Was the output merely generated, or was successful execution observed?
• What credentials, permissions, or vulnerabilities did the attacker already have?
• Which decisions required human intervention?
• What failed, and could researchers reproduce the result?

Use those answers to judge relevance to your environment. A tested intrusion involving systems you operate deserves a different response from an unverified advertisement.

Your own AI can become the target

Attackers can also manipulate assistants employees already trust. Indirect prompt injection involves malicious instructions placed in external content an AI system processes, such as email.

In March 2026 testing, Permiso researchers found cases where instructions embedded in email influenced Copilot summaries, including deceptive security-alert content. Results varied by interface, and some attempts were blocked or ignored.

The findings, explored in this coverage of phishing through AI email summaries, show why an assistant’s output should not automatically inherit the authority of the application displaying it.

Review what external content assistants read, what connected tools can access, and which actions require human approval.

Strengthen controls you can test

Start with identity and business processes. CISA recommends phishing-resistant MFA, including hardware-based FIDO or PKI methods. Prioritize privileged access, and require independent verification of payment changes or requests to bypass approval.

Give analysts concrete investigation triggers: unexpected MFA requests, newly registered authentication methods, unusual access to sensitive files, unexplained scripts, and unauthorized AI connectors. These signals do not prove AI involvement, but they can reveal harmful activity regardless of how it was prepared.

Then run a realistic response exercise. If an employee submits credentials to a convincing phishing page, can responders identify the account, revoke unauthorized access, and establish what it reached? Assign owners, confirm logging coverage, and measure time to containment. Review whether that identity could also reach backups, administrative functions, or payment systems.

Apply the same discipline to AI deployments. Restrict assistants to the permissions their tasks require, require approval for consequential actions, and test how they handle malicious instructions in external content. Establish who can suspend an agent or connector during an investigation.

The useful outcome is a tested ability to contain credential theft, unauthorized actions, and data exposure. Investigate whether AI helped the attacker, but let the observed behavior drive the immediate response.

Want to learn more AI tips, tricks, and prompting techniques? eSecurity Planet readers get free 7-day access to The Neuron Academy, our practical learning platform designed to help professionals use AI more confidently at work. Browse all lessons →

Let us teach you How to Talk to AI for free! Try our six-minute course at The Neuron Academy and learn a few simple ways to write better prompts and get more useful results from AI, or browse our other AI course for free for seven days. Check out all the lessons here →