TL;DR: Adversarial examples are inputs perturbed (often imperceptibly) to make a model misclassify, exploiting decision boundaries that do not match human perception. They matter wherever a model gates a real decision (content moderation, fraud, malware, biometrics) because an attacker can craft inputs to evade it. No defense is complete; the strongest practical one is adversarial training, layered with input checks, ensembles, and human review for high-stakes calls.
How to approach it. Define it, then make the threat concrete (where misclassification has real consequences), then be honest that defenses are partial. The signal is treating it as an arms race to be contained, not a bug to be patched, mirroring the prompt-injection mindset.
A strong answer. What they are. An adversarial example is an input with a small, often human-imperceptible perturbation that causes a confident wrong prediction: a few pixels changed flips an image classifier, a crafted patch defeats a detector, a reworded phrase slips past a text filter. They exist because models learn boundaries from data that do not align with human-meaningful features, so there are directions in input space where tiny moves cross the boundary. Clean-test accuracy says nothing about behavior under an optimizing adversary.
Why it is a security concern. The risk is anywhere a model gates a decision an adversary wants to influence: evading content moderation or spam filters, fooling fraud or malware detection, defeating biometric or document verification, or (for LLMs) jailbreaks and prompt injection, which are the language-model analog. The attacker actively optimizes against your model, so a 99% clean accuracy can collapse to single digits under attack.
Defenses, honestly partial:
| Defense | What it buys you | Where it fails |
|---|---|---|
| Adversarial training | Strongest single defense; trains on crafted examples | Costs clean accuracy; only covers attacks you trained against |
| Input preprocessing / detection | Smooths or flags perturbations | Often itself evadable by adaptive attacks |
| Ensembles and randomization | Raises attacker cost (fool many models at once) | Beaten by transfer and query attacks given budget |
| Certified defenses (randomized smoothing) | Provable guarantee within a perturbation bound | Bound is small; limited to specific norms |
| Defense in depth plus human review | Contains blast radius of any one evasion | Adds latency and review cost |
The honest framing: this is an arms race. You raise the cost of attack and contain the consequences of a successful one; you do not solve it. Evaluate resilience explicitly, under a defined attack and perturbation budget, not just on clean data.
Key takeaways.
- Adversarial examples exploit the mismatch between learned boundaries and human perception, so clean accuracy is not a security metric.
- The threat is real only where a model gates an adversary-influenced decision; scope investment to that.
- Adversarial training is the best single defense, but every defense is partial and adaptive attacks break most layers.
- For high-stakes, irreversible calls, never let the model be the sole gate; layer rules, rate limits, and human review.
What interviewers probe next.
- "White-box vs black-box attacks?" White-box (attacker knows weights/gradients) is strongest; black-box uses queries or transferability. Defenses that only obscure gradients tend to fail against black-box and adaptive attacks.
- "Why does adversarial training hurt clean accuracy?" It enforces a smoother boundary, trading some clean-data fit for resilience under attack.
- "How is this like prompt injection?" Both exploit the gap between learned and intended behavior via crafted input; both need layered defense and assume some attacks succeed.
- "How do you measure resilience under attack?" Report accuracy under a named attack and perturbation budget, not just clean data.
Common mistakes.
- Reporting clean-data accuracy as if it implies resilience to attack.
- Claiming a single defense (or gradient obfuscation) makes the model secure, when adaptive attacks break most.
- Ignoring the realistic threat model (who attacks this and why) and over- or under-investing.
- Letting the model be the sole gate on a high-stakes, irreversible decision.