AI vs. AI: Microsoft Blocks Phishing Attack Using LLM-Generated Obfuscated Code

Microsoft Threat Intelligence has revealed details of a credential phishing campaign that likely harnessed AI-generated code to enhance its obfuscation techniques and slip past traditional defenses. While the campaign was ultimately blocked, it illustrates how cybercriminals are beginning to integrate large language models (LLMs) into their toolkits, raising the stakes for defenders worldwide.

According to Microsoft’s analysis, “the activity obfuscated its behavior within an SVG file, leveraging business terminology and a synthetic structure to disguise its malicious intent.” The code was so complex that Microsoft’s own Security Copilot concluded it was “not something a human would typically write from scratch due to its complexity, verbosity, and lack of practical utility.”

The attackers crafted SVG attachments disguised as PDFs, embedding malicious scripts behind what looked like a business performance dashboard filled with invisible chart elements. Business-related terms such as “revenue,” “operations,” and “risk” were concatenated into hidden attributes, later decoded into executable payloads.

The campaign, detected on August 18, leveraged a compromised small business email account to distribute its lures. Messages were self-addressed — the sender and recipient were the same — while the real victims were hidden in the BCC field, a tactic designed to “bypass basic detection heuristics.”

The attachment, named “23mb – PDF-6 pages.svg,” was designed to appear like a standard PDF file, tricking users into opening it. Once launched, the SVG redirected victims to a fake CAPTCHA page intended to build trust before leading to a credential-harvesting portal.

Microsoft Security Copilot identified several indicators suggesting the obfuscation was machine-generated:

• “Overly descriptive and redundant naming” (e.g., processBusinessMetricsf43e08)
• “Modular and over-engineered code structure” resembling textbook AI output
• “Verbose, generic comments” written in formal business language
• “Formulaic obfuscation techniques” like staged data transformation
• “Unusual use of CDATA and XML declaration” — technically correct but unnecessary

These traits, Microsoft noted, are consistent with AI/LLM-generated code.

Despite the sophistication, Microsoft Defender for Office 365’s AI-powered protections successfully stopped the attack. As the report stresses, “AI-enhanced threats, while evolving, are not undetectable… an attacker’s use of AI often introduces new artifacts that can be leveraged for detection.”

Detection was aided by multiple factors: suspicious file naming, unusual use of SVG as a payload, redirect patterns, and session tracking behaviors on the phishing site. These infrastructural and behavioral signals remained visible despite the attacker’s obfuscation attempts.

While this campaign was limited and primarily targeted US-based organizations, it signals a growing trend. “Like many transformative technologies, AI is being adopted by both defenders and cybercriminals,” Microsoft warns.

Related Posts:

• SVG Phishing Surge: How Image Files Are Being Weaponized to Steal Credentials
• SVG Files Weaponized: Phishing Attacks Embed HTML Code
• Sophos Uncovers Rising Threat of SVG-Based Phishing Attacks
• SVG Files: The Emerging Vector of Cyber Threats
• SVG Attacks: How GULoader Malware Sneaks into Your Network