Adversarial Examples in AI: Attacks, Defenses & the 2026 Landscape

Adversarial examples are one of the most critical unsolved problems in modern machine learning. These are inputs that have been deliberately perturbed — often by changes imperceptible to the human eye — to cause a model to make an incorrect prediction. Since Szegedy et al. first demonstrated the phenomenon in 2014, adversarial examples have evolved from a research curiosity into a first-class security concern that now threatens production AI systems across every domain, from autonomous vehicles to large language models. This guide provides a comprehensive technical overview of the adversarial ML landscape as of mid-2026.

What Are Adversarial Examples?

An adversarial example is an input to a machine learning model that an attacker has intentionally designed to cause a misclassification or erroneous output. Formally, given a classifier f and a legitimate input x with ground-truth label y, an adversarial example x' satisfies f(x') ≠ y while x' is constrained to lie within a small ε-ball around x under some distance metric (typically L_∞, L₂, or L₀). The constraint ensures the perturbation is small enough to evade human detection.

The seminal 2014 paper by Szegedy et al. at ICLR revealed that state-of-the-art neural networks classified images with high confidence but were completely fooled by tiny perturbations that were visually indistinguishable from natural noise. Goodfellow et al. (2015) subsequently proposed the Fast Gradient Sign Method (FGSM), a simple one-step attack that demonstrated the linear nature of adversarial vulnerability in high-dimensional spaces: neural networks, counterintuitively, are vulnerable precisely because of their linearity in high dimensions, not their nonlinearity. This insight reshaped how the research community understood model robustness.

Since then, a vast taxonomy of attack algorithms has emerged. The most influential include Projected Gradient Descent (PGD), an iterative variant that remains the de facto standard for evaluating robustness; the Carlini-Wagner attack, an optimization-based approach that produces minimal-norm perturbations; and AutoAttack, an ensemble of complementary attacks designed for reliable robustness evaluation. In 2026, compositional attack frameworks (Radionov & Levina, Scientific Reports 2026) now treat attacks as modular systems synthesized from interchangeable functional blocks, enabling automated discovery of novel attack configurations that outperform hand-crafted benchmarks.

Why They Matter for AI Safety

Adversarial examples are not merely a theoretical curiosity — they have direct and growing implications for deployed AI systems. In safety-critical domains such as autonomous driving, a stop sign perturbed with a small sticker can cause a vision model to classify it as a speed-limit sign. In medical imaging, imperceptible perturbations injected into an X-ray or MRI scan can flip a model's diagnosis from malignant to benign. In cybersecurity, network intrusion detection systems (NIDS) that rely on deep learning experience accuracy drops from 97% to below 30% under targeted white-box attacks, as demonstrated in a 2026 study published in Discover Computing.

The economic and safety stakes are enormous. A 2025 survey in Artificial Intelligence Review catalogued vulnerabilities across four critical industries: automotive, digital healthcare, electrical power and energy systems, and LLM-based NLP. The survey found that adversarial robustness remains the largest gap between academic research and industry deployment. Many production systems still lack any adversarial defense, and even state-of-the-art defenses are regularly broken by adaptive attackers who can tailor their strategy to the defense mechanism.

The UK NCSC's Guidelines for Secure AI System Development treat adversarial ML (AML) as a distinct threat category requiring dedicated mitigation throughout the ML lifecycle. Similarly, NIST's AI 100-2 E2025 report provides a comprehensive taxonomy that classifies attacks along dimensions including attacker knowledge, goals, objectives, and stage in the ML lifecycle. The consistent message from every major standards body is clear: adversarial vulnerability is a first-order safety and security issue.

Attack Taxonomy: White-Box vs Black-Box

Adversarial attacks are classified along several key dimensions. The most fundamental distinction concerns the attacker's knowledge of the target model.

White-box attacks assume the attacker has complete knowledge of the model architecture, parameters, training data, and gradients. This is the strongest threat model and gives rise to the most efficient attacks. PGD, FGSM, and Carlini-Wagner are white-box methods. In a white-box setting, the attacker can compute exact gradient directions and craft minimal perturbations with high success rates. The 2026 study on NIDS cited earlier demonstrated that white-box PGD attacks reduced multi-class classification accuracy from 97.11% to as low as 29.45% for certain target classes.

Black-box attacks are more realistic: the attacker has no access to model internals and can only query the model's outputs. Subtypes include:

• Score-based (soft-label) attacks, where the attacker observes full output probabilities. The Square Attack, identified as the strongest black-box score-based attack in the 2025 BlackboxBench evaluation, achieves high success rates with limited queries.
• Decision-based (hard-label) attacks, where only the final class label is available. The Boundary Attack and HopSkipJump are canonical examples.
• Transfer attacks, where an adversary trains a local surrogate model and crafts perturbations that transfer to the black-box target. The 2026 VCP-Attack achieved a 95.6% attack success rate against GPT-4o using transferable perturbations optimized via contrastive projection.

Gray-box attacks occupy the middle ground. The attacker may know the model architecture but not the parameters, or they may have limited query budgets. Query-assisted gray-box transfer auditing, as explored in the 2026 compositional-attack literature, provides more realistic robustness evaluations than pure black-box or white-box assumptions.

Attack Taxonomy: Targeted vs Untargeted

Beyond the knowledge dimension, attacks are classified by their goal.

Untargeted attacks aim only to cause any misclassification. The attacker's objective is f(x') ≠ y for any label other than the ground truth. These are easier to mount and require smaller perturbation budgets. Most early adversarial example research focused on untargeted evasion.

Targeted attacks are more constrained and more dangerous. The attacker aims for f(x') = y_target, a specific incorrect class chosen in advance. Targeted attacks require larger perturbations and more sophisticated optimization, but they are essential for evaluating worst-case risks. The 2026 NIDS study found that adversarial training partially restored accuracy after targeted attacks — from 29.45% to 71.96% for PortScan targeting — but recovery varied significantly by class, highlighting class-conditional robustness gaps.

In the LLM domain, targeted prompt injection is the dominant threat model. The attacker wants the model to execute a specific action — send an email, reveal a password, execute code — rather than merely produce a garbled or refusal output. The 2026 AutoInject framework, using a 1.5B-parameter reinforcement learning policy, achieved 77.96% attack success rates against Gemini 2.5 Flash and 21.88% against Meta's SecAlign-70B, a model fine-tuned specifically to resist injection. These results confirm that targeted attacks remain effective even against state-of-the-art defenses.

Real-World Examples: Image Classifiers

Image classifiers remain the canonical testbed for adversarial attacks, and the physical-world threat continues to escalate. In 2018, Kurakin et al. demonstrated that adversarial perturbations printed on paper and photographed with a smartphone could fool classifiers. Since then, physical attacks have only grown more sophisticated.

Stop sign attacks. The classic example: adding small black-and-white stickers to a stop sign causes a real-time object detector to classify it as a 45 mph speed-limit sign. This is not theoretical — Eykholt et al. (2018) demonstrated successful physical-world attacks against traffic sign classifiers at multiple viewing angles and distances. In 2025-2026, autonomous vehicle companies reported that digital billboard advertisements could be manipulated to project adversarial patterns that confuse camera-based perception systems.

Medical imaging attacks. Finlayson et al. (2019) showed that adversarial perturbations could cause deep learning models to flip skin lesion diagnoses from malignant to benign. Ma et al. (2021) demonstrated similar vulnerabilities in lung cancer screening. A 2025 review noted that medical AI systems are especially vulnerable because the adversarial perturbation can be smaller than the resolution of the imaging modality, making detection by radiologists nearly impossible.

Diffusion-based purification attacks. In 2026, the PureProof framework demonstrated that even diffusion-based purification (DBP) — considered a state-of-the-art black-box defense — can be bypassed. PureProof uses a stochastic reverse alignment technique to craft adversarial images that survive the denoising diffusion process, achieving up to 25.6% targeted attack success rates against Qwen3-VL after DBP. This reveals that no current defense is unbreakable.

Real-World Examples: Prompt Injection in LLMs

Prompt injection has become the most practically significant adversarial attack vector, ranked LLM01 in the OWASP Top 10 for LLM Applications. In this class of attack, adversarial instructions embedded in model inputs override the model's safety alignment and system instructions, causing the model to execute attacker-desired behaviors.

Direct prompt injection places malicious instructions in the user input itself. A 2026 study across 37 real-world applications and 143,745 queries found that simple attack instructions consistently outperformed complex ones — simplicity bypasses detection mechanisms more effectively. Real-world applications were substantially more vulnerable than research benchmarks.

Indirect prompt injection is more insidious. The attacker embeds malicious instructions in external data sources — emails, documents, web content — that an LLM agent retrieves during normal operation. A 2026 study demonstrated that a single poisoned email in a multi-agent workflow could coerce GPT-4o into exfiltrating SSH keys with over 80% success rate. The attack succeeded under natural user queries ("summarize my emails") without any contrived trigger phrases. The key enabling insight was the decomposition of the attack into a trigger fragment (guaranteeing retrieval) and an attack fragment (carrying the malicious instruction).

Agentic injection. A large-scale public red-teaming competition in 2026 attracted 464 participants who submitted 272,000 attack attempts against 13 frontier models, yielding 8,648 successful attacks. All models proved vulnerable, with attack success rates ranging from 0.5% (Claude Opus 4.5) to 8.5% (Gemini 2.5 Pro). The competition's key finding: concealment matters as much as execution. Successful attacks left no trace of compromise in the final response while exfiltrating data, deleting files, or executing code behind the scenes.

Production CVEs now document active exploitation. EchoLeak (CVE-2025-32711, CVSS 9.3) enabled zero-click data exfiltration via a single crafted email sent to Microsoft 365 Copilot. GitHub Copilot RCE (CVE-2025-53773, CVSS 9.6) injected prompt payloads in public repository code comments. The Cursor IDE triple CVE chain in 2026 (max CVSS 9.8) demonstrated that AI coding assistants are the most targeted product category for prompt injection.

The NCSC 2026 AML Taxonomy

In April 2026, the UK National Cyber Security Centre (NCSC) released "Understanding Adversarial Attacks Against Machine Learning and AI," which defines seven AML attack classes that group similar attack techniques by how they interact with ML systems. This taxonomy supplements NIST AI 100-2 and MITRE ATLAS by taking an adversary-first approach:

Attack Class	Description	Example Techniques
Model Characterisation	Discovering key model information to enable later attacks	Reconnaissance, surrogate model building, identifying common hallucinations
Model Inversion	Extracting sensitive information from model outputs	Membership inference, training data reconstruction, model stealing
Training Data Poisoning	Altering training data or the training process	Malicious fine-tuning, hidden backdoor insertion, model compression alteration
Model Input Manipulation	Crafting inputs that cause misclassification	Adversarial perturbations, direct and indirect prompt injection, cross-modal attacks
Model Artefact Manipulation	Tampering with model files or metadata	Serialiser exploitation, weight modification, format deserialisation
Model Hardware Manipulation	Attacking the physical hardware running the model	Side-channel attacks on accelerators, fault injection in GPUs/TPUs
Malicious Model Training	Weaponising a model during its development	Supply chain compromise, embedding hidden capabilities during pre-training

The NCSC emphasises that standard cyber security controls provide baseline protection but are insufficient for these ML-specific vulnerabilities. The agency advocates for security measures throughout the developmental lifecycle, as described in the UK government's AI Cyber Security Code of Practice, and highlights that research gaps remain most acute in the model hardware and malicious model training classes.

The MITRE ATLAS framework provides a complementary knowledge base of adversary tactics, techniques, and mitigations for ML systems. Together, NIST AI 100-2, the NCSC AML attack classes, and MITRE ATLAS form the three-pillar reference architecture for adversarial ML threat modelling as of 2026.

Defense Methods: Adversarial Training

Adversarial training (AT) is the most empirically successful defense against adversarial examples. The fundamental idea is simple: augment the training set with adversarial examples so the model learns to be robust to them. Goodfellow et al. introduced the approach in 2015, showing that training on FGSM examples improved robustness. Madry et al. (2018) formalised AT as a minimax optimisation problem that remains the theoretical foundation:

min_θ E_(x,y)~D [ max_{||δ||p ≤ ε} L(f_θ(x+δ), y) ]

The inner maximisation finds the worst-case perturbation within the ε-ball; the outer minimisation updates model parameters to reduce loss on these worst-case examples. Madry's PGD-based AT (PGD-AT) became the benchmark, but its computational cost (10-40 attack steps per training iteration) motivated a wave of efficiency research.

Fast adversarial training. Wong et al. (2020) proposed FastAT, using single-step FGSM with random starts, achieving comparable robustness to PGD-AT at a fraction of the cost. However, FastAT suffers from catastrophic overfitting (CO), where robustness to multi-step attacks collapses while single-step accuracy remains high. The 2026 SORA method (ICLR 2026) introduced adaptive step-size distributions based on loss-surface geometry and a PertAlign metric that predicts CO onset. SORA prevents CO across datasets and architectures with a single set of hyperparameters.

Long-tail adversarial training. Most AT research assumes balanced datasets, but real-world data follows long-tail distributions. The RobustLT framework (CVPR 2026) adaptively assigns perturbation intensity by class — higher for minority classes — and stabilises adversarial distributions across training iterations. RobustLT improves both robustness and class balance on long-tail data without modifying the underlying AT algorithm.

Parameter-efficient AT. The CAAT method (2026) identifies robustness-critical parameters in Vision Transformers and applies LoRA or adapter fine-tuning to only ~6% of parameters, achieving robustness within 4.3% of full AT while reducing training cost by an order of magnitude. This is a crucial step toward making AT practical for billion-parameter models.

Nasty adversarial training. A surprising recent development: Zhou et al. (ICLR 2026) demonstrated that probability sparsity — a property originally associated with model IP protection — can improve robustness. Their NAT method regularises the output distribution's sparsity, providing interpretable robustness gains with minimal overhead.

Defense Methods: Certified Robustness and Purification

Empirical defenses like AT can be broken by stronger or adaptive attacks. Certified robustness methods provide mathematical guarantees that the model's prediction remains unchanged within a specified ε-radius, eliminating the possibility of unanticipated attack strategies.

Lipschitz-based certification bounds the model's Lipschitz constant to guarantee that small input changes produce bounded output changes. The field has struggled to scale — most methods cap at 32M-parameter VGG-style architectures on CIFAR-10. The 2026 LipNeXt architecture breaks this barrier through a convolution-free, constraint-free design that scales to billion-parameter models, achieving non-saturating gains with increasing model size. On ImageNet at ε = 1 (the standard for randomised smoothing), LipNeXt outperforms prior work by 8%.

Multi-norm certification. Traditional certified training is single-norm — it guarantees robustness under L_∞ or L₂ perturbations but not both. The 2026 CURE framework introduces the first multi-norm certified training method, achieving 25.8% better union robustness on CIFAR-10 than single-norm approaches. CURE also demonstrates that improving union-certified robustness yields better generalised robustness to patch and geometric transformations.

Diffusion-based purification offers a complementary approach: instead of making the model robust, remove adversarial perturbations from the input using generative diffusion models. DiffPure (2022) showed that diffusion can effectively "denoise" adversarial examples. However, the 2026 PureProof and VCP-Attack papers revealed that adaptive attackers with diffusion awareness can circumvent purification, achieving over 95% ASR against GPT-4o even after DBP. The arms race continues.

Runtime defenses. The Dashed Line Defense (DLD, 2026) applies a non-continuous loss mapping to disrupt black-box score-based queries without modifying model parameters. DLD provides theoretical guarantees against adaptive attacks and outperforms prior plug-and-play defenses on ImageNet. Its key insight: defenses must account for attacker adaptivity, or they provide only illusory robustness.

Comparison of Defense Approaches

Defense	Type	Provides Guarantee?	Computational Cost	Scalability	Key Limitation
PGD Adversarial Training (AT)	Empirical	No	High (10-40 steps per iteration)	Moderate	Broken by stronger attacks or higher perturbation budgets
Fast AT (FGSM-RS)	Empirical	No	Low	High	Catastrophic overfitting without stabilisation
Nasty AT (NAT)	Empirical	No	Low (regularisation only)	High	Novel approach, limited empirical validation to date
CAAT (Parameter-Efficient AT)	Empirical	No	Very low (~1% of full AT)	Very high	Slight robustness loss vs full AT
Dashed Line Defense (DLD)	Runtime	No (theoretical against SQAs)	Negligible inference-time overhead	Very high	Only protects against score-based query attacks
LipNeXt (Lipschitz Certification)	Certified	Yes (Lipschitz bound)	Moderate	High (1B+ params)	Slightly lower clean accuracy
CURE (Multi-Norm Certified)	Certified	Yes (multiple Lp norms)	High	Moderate	Trade-off between different norm guarantees
DiffPure / DBP	Purification	No	High (diffusion process)	Moderate	Bypassed by diffusion-aware attacks

No single defense is sufficient. The consensus in the 2026 literature is that layered, defense-in-depth strategies are necessary. Empirical defenses like AT handle known attack families; certified methods guarantee robustness against bounded perturbations; runtime defenses handle adaptive black-box queries; and input preprocessing mitigates certain physical-world attacks. Combining multiple approaches — while managing their interactions — remains an open research challenge.

Future Directions and Open Challenges

The adversarial ML field in 2026 faces several defining open problems that will shape the next generation of research and practice.

Closing the robustness-accuracy trade-off. Every known defense degrades clean accuracy. The GRACE method (CVPR 2026) partially addressed this for vision-language models by jointly regularising loss landscape curvature and feature-space invariance, but a fundamental trade-off appears inherent. Scaling laws from Bartoldson et al. (2024) suggest that robustness improvements may require exponentially larger models, which is impractical for many deployments.

Jailbreak and injection in LLMs. Prompt injection remains unsolved. OpenAI's February 2026 Lockdown Mode for ChatGPT included an explicit acknowledgment that injection in AI browsers "may never be fully patched." Automated injection via reinforcement learning (AutoInject 2026) and transferable suffix attacks that succeed across model families represent a rapidly escalating threat. Supply-chain attacks on AI agent marketplaces — such as the Cline/OpenClaw incident affecting 4,000 developer machines — signal that AML supply-chain attacks are industrialising.

Scaling certified robustness. LipNeXt demonstrated that billion-parameter certified models are feasible, but certification remains limited to L_p-norm perturbations. Generalised certification against semantic perturbations — translations, rotations, colour shifts — is the next frontier. The CURE framework's finding that multi-norm training improves generalised robustness is a promising step, but the gap between certified and empirical robustness on natural transformations remains wide.

Operationalising AML detection. The gap between ML research and security operations remains large. The 2026 structured-logging framework proposed by Mehta and Kaur bridges this by translating ML events into SIEM-ingestible telemetry mapped to ISO 27001 controls, enabling SOC teams to detect adversarial attacks on ML models using existing security workflows. As threat actors increasingly weaponise prompt injection against AI-powered security defenses — the 2026 Chameleon Trap phishing campaign combined hidden prompts with the Follina vulnerability to trick AI email scanners — operational integration becomes urgent.

Multi-modal and agentic attacks. VLMs and AI agents introduce new attack surfaces. Visual prompt injection hides instructions in images using steganographic embedding. The Moltbook analysis found 2.6% of AI agent posts contained hidden injection payloads — the first large-scale demonstration of bot-to-bot injection in production. Agentic systems that call tools, browse the web, and execute code amplify the blast radius of any successful injection by orders of magnitude.

The overarching challenge is that adversarial robustness is not a feature that can be added after deployment — it must be designed into the ML system from the start, alongside traditional security controls. As the NCSC 2026 paper concludes: "Standard cyber security controls offer a baseline of protection for AI/ML systems and should be considered foundational to any deployments. However, the ML-specific weaknesses outlined in these AML attack classes often require specific mitigation techniques, which are not yet mature or widely deployed." Closing this maturity gap is the defining task for adversarial ML research in the years ahead.