Explain model extraction and membership inference attacks, and how you defend against them.

TL;DR: Model extraction steals a model's functionality by querying the API heavily and training a surrogate on the input-output pairs, threatening IP and enabling offline white-box attacks. Membership inference asks whether a specific record was in the training set, exploiting that models are more confident (lower loss) on data they memorized. Defenses (rate limiting, coarse outputs, DP/regularization, query monitoring) all cost utility, so you layer them in proportion to the asset's value.

How to approach it. Separate these from the input-fooling attacks (adversarial examples, prompt injection): those manipulate behavior, these target confidentiality, the model's weights/IP and its training data. For each, name the mechanism, the asset at risk, and the defense, and be honest that every defense buys protection with accuracy or usability.

A strong answer. Model extraction (model stealing). An attacker with API access fires many queries, harvests the outputs (especially if you return probabilities or logits), and trains a surrogate model to mimic the target. Enough queries reconstruct most of the functionality, stealing IP you paid to build. Worse, the surrogate then becomes a free white-box target: craft adversarial examples against the copy offline, then transfer them to the original. The richer the output (full probability vector, logits, token explanations), the cheaper the theft. Returning only a top label forces the attacker to spend orders of magnitude more queries.

Membership inference. The attacker asks one question: was this specific record in the training set? Models tend to be more confident, lower loss, on training examples than on unseen ones, a direct symptom of memorization. So the attacker observes the model's confidence on a candidate record and thresholds it: high confidence implies a member. That is a privacy violation in itself. Knowing someone appeared in a medical or sensitive dataset can be harmful regardless of the prediction, and for LLMs it connects to verbatim memorization of training text.

The two attacks share a root cause and an attack surface, which is why one defense often dampens both:

Defenses, all with utility tradeoffs:

• Limit output granularity. Return labels or rounded confidences instead of full probability vectors or logits. This blunts both attacks at the cost of less informative responses.
• Rate limiting and query monitoring. Both attacks need volume. Per-account limits, authentication, and anomaly detection on query patterns (systematic boundary probing, high-volume scraping) raise the cost and catch campaigns.
• Reduce memorization. Regularization and especially differential privacy (DP-SGD) provably bound any single example's influence, which is exactly the signal membership inference reads, at a measurable accuracy cost.
• Watermarking / fingerprinting. Embed signals so you can later prove a suspect model derived from yours. Forensic and deterrent, not prevention.

No single measure stops a determined attacker, so scale defense in depth to the value of the model and the sensitivity of the data.

Key takeaways

• Both attacks target confidentiality and feed on informative outputs and query volume, so coarse outputs plus rate limiting attack both at once.
• DP-SGD defends membership inference by construction: it bounds per-example influence, erasing the confidence gap the attack thresholds on.
• Extraction and distillation are the same mechanism; extraction is just the unauthorized version against someone else's API.
• Invest proportionally: a high-value model on sensitive data warrants DP plus monitoring; a low-stakes classifier may need only rate limits.

What interviewers probe next.

• "How does returning probabilities help an attacker?" Full probability or logit vectors carry far more signal per query, speeding both surrogate training and confidence-based membership inference. Top-label-only is much harder to exploit.
• "Why does DP defend membership inference?" DP bounds how much any single training record shifts the model, so members are no longer noticeably more confident, which is the exact signal the attack uses.
• "Extraction vs distillation?" Same training-on-outputs mechanism; extraction is the adversarial, unauthorized form against an API you do not own.
• "How do you detect an extraction campaign?" Watch for high-volume, systematic, boundary-probing queries per account, then throttle or block.

Common mistakes.

• Lumping these with adversarial examples; those fool inputs, these steal the model or its training-data membership.
• Returning full probability vectors or logits by default, handing attackers the richest possible signal.
• Assuming one defense suffices; determined attackers route around any single control.
• Ignoring that every defense costs utility, and failing to scale it to the asset's value.