TL;DR: Data poisoning corrupts training data to degrade the model or plant a backdoor (a hidden trigger that forces a chosen output) while normal accuracy looks fine. ML supply-chain attacks tamper with the components you trust: pretrained weights, datasets, packages, or model hubs. Defend with data provenance and validation, anomaly detection on training data, vetting and pinning third-party models and dependencies, and treating training integrity like any other supply-chain security problem.
How to approach it. Distinguish training-time attacks (the focus here) from inference-time ones (adversarial examples, prompt injection). Name the two classes (poisoning/backdoors and supply-chain tampering), make the threat concrete, then frame defense as provenance and validation across the pipeline, since there is no single model-level fix.
A strong answer. Data poisoning. An attacker injects malicious examples into training data to either degrade the model broadly (an availability attack) or, more dangerously, plant a backdoor: the model behaves normally except when a specific trigger appears (a phrase, a pixel pattern), at which point it produces an attacker-chosen output. Backdoors are insidious because accuracy on clean data looks fine, so the model passes evaluation while carrying a hidden behavior. This is a real risk when training data is scraped from the open web or crowdsourced (an attacker can seed content), or when fine-tuning on user-supplied data.
Supply-chain attacks. Modern ML reuses components you did not build: pretrained weights from a hub, public datasets, packages and dependencies, and base models. Any of these can be tampered with. A backdoored pretrained model you fine-tune inherits the backdoor; a malicious package in the training environment can exfiltrate data or alter behavior; a poisoned public dataset propagates to everyone who uses it. This is the same supply-chain threat as in software, applied to model and data lineage.
The attack surface, from source to served model:
Defenses (provenance plus validation, not one model fix):
- Data provenance and integrity. Know where training data came from, version it, and checksum it; prefer trusted/curated sources over indiscriminate scraping for sensitive models. Track lineage so you can audit and, if needed, retrain without a poisoned batch.
- Training-data validation and anomaly detection. Screen for outliers, label inconsistencies, and suspicious patterns (clusters of near-duplicate trigger-bearing examples); filter or quarantine before training.
- Vet and pin third-party artifacts. Use trusted model/dataset sources, verify checksums and signatures, pin dependency versions, and scan the training environment, exactly as you would secure a software supply chain.
- Evaluate for backdoors. Beyond clean accuracy, test for anomalous behavior (trigger scanning, activation-pattern analysis) where the stakes justify it; assume clean-data metrics do not prove the absence of a backdoor.
- Limit and isolate. Sandbox training, enforce least-privilege access to training data and pipelines, and require human review for data sources feeding high-stakes models.
The honest framing: training-time integrity is a supply-chain security problem. You cannot prove a model is backdoor-free, so you control provenance, validate data and artifacts, and reduce trust in unvetted external components.
Key takeaways.
- Poisoning hits at training time; adversarial examples hit at inference time. Different stage, different defense.
- Backdoors survive standard evaluation because clean-data accuracy is untouched; only the secret trigger fires the malicious behavior.
- Treat pretrained weights, datasets, and packages as untrusted supply chain: verify signatures, pin versions, track lineage.
- You cannot prove a model is clean, so invest in provenance and validation rather than a single model-level fix.
What interviewers probe next.
- "Poisoning vs adversarial examples?" Poisoning attacks the model at training time (corrupt the data/weights); adversarial examples attack a trained model at inference time (craft inputs). Different stage, different defense.
- "Why are backdoors hard to detect?" Clean-data accuracy is unaffected, so normal evaluation passes; the malicious behavior only fires on the secret trigger.
- "Risk of using a pretrained model off a hub?" You inherit whatever is baked in, including potential backdoors; verify the source and signatures, and ideally evaluate before trusting.
- "How does this apply to RLHF/fine-tuning on user data?" User-supplied preference or fine-tuning data is an injection vector; validate and curate it, and rate-limit and attribute contributions.
Common mistakes.
- Only considering inference-time attacks and ignoring the training pipeline.
- Trusting clean-data accuracy as proof of integrity, missing backdoors.
- Pulling pretrained weights, datasets, and packages with no provenance, signature, or version pinning.
- Treating it as a model problem rather than a data and supply-chain security problem.