Hard Applied AI interview questions

Almost everyone can quote softmax(QKᵀ/√d_k)V. The interviewer wants the variance argument and the exact training failure the scale prevents. Here is the answer that separates memorization from understanding.

Alignment is now table stakes even outside 'safety' roles. The interviewer wants the three-stage RLHF pipeline and a crisp account of why DPO dropped the reward model entirely. Here is the answer with the mechanism, not the buzzwords.

MoE is why some frontier models have huge parameter counts but serve cheaply. The signal is the sparse-activation idea, the routing mechanism, and the operational cost (memory, load balancing) that the FLOP savings quietly hide.

Attention is permutation-invariant, so without position information a transformer cannot tell word order. The signal is knowing why, and why the field moved from absolute sinusoidal encodings to relative ones like RoPE and ALiBi that extrapolate to longer contexts.

Evaluation is the hardest, most underrated part of shipping LLMs. The signal is knowing why public benchmarks mislead, why LLM-as-judge is biased, and how to build a task-specific eval you actually trust.

PEFT is how everyone fine-tunes large models now. The signal is the low-rank insight behind LoRA, why it cuts memory so much, and what 4-bit QLoRA adds. Here is the answer that goes past 'it's efficient fine-tuning.'

RL alignment moved from PPO to leaner methods, and DeepSeek-R1 made GRPO famous. The signal is knowing what the value/critic model does in PPO and how GRPO replaces it. Here is the mechanism, not the acronyms.

Fine-tune a model on your domain and it can forget how to do everything else. The signal is explaining why shared weights cause it and naming the concrete mitigations that all reduce to one principle.

PEFT is a family, not just LoRA, and interviewers probe whether you know how each one injects trainable parameters. The signal is where each method adds capacity and the latency and quality tradeoffs that follow.

Synthetic data is how teams get training data when real data is scarce, and it has sharp failure modes. The signal is naming when it helps, how to hold the quality line, and why training on model output across generations narrows the distribution.

'Eval is the new system design' for LLM apps, and starting from zero data is the real test. The signal is bootstrapping a golden set, picking task-specific metrics, and running evals continuously as a regression gate.

Full attention is O(n^2), so long context needs cheaper attention. The signal is naming the families (sparse, sliding-window, linear), what each gives up, and why FlashAttention is not one of them.

RLAIF and Constitutional AI swap human preference labels for AI-generated ones to scale alignment. The signal is naming exactly what they substitute for human feedback, and the consistency-versus-bias tradeoff that swap creates.

Reasoning models are the 2025-2026 frontier. The signal is knowing they are trained (largely via RL on verifiable rewards) to produce long internal reasoning, and that they scale on a second axis: spending more compute at inference.

Modern LLMs dropped learned absolute positions for RoPE and ALiBi to handle long context. The signal is knowing both encode RELATIVE position, and exactly how that enables length extrapolation past the trained window.

MQA and GQA shrink the KV cache, the thing that bottlenecks LLM serving. The signal is knowing they share key/value heads across query heads to cut memory and bandwidth, with GQA as the quality-preserving middle ground. Here is the answer.

DPO simplified RLHF, and a family of variants now trade off what data and reference model they need. The signal is knowing what each variant removes or changes (reference model, paired data, separate SFT stage). Here is the answer.

Fine-tuning a 65B model used to mean a node of A100s. QLoRA collapses it onto one card with three specific tricks. The signal is knowing what gets quantized, what stays trainable, and why quality barely moves. Here is the answer.

Combine several fine-tunes into one model by doing arithmetic on their weights, with no retraining and no data. The signal is knowing why it works and which method to reach for when merges interfere.

Chain-of-thought looks like the model showing its work, but it may not reflect the real computation. The signal is knowing CoT can be a post-hoc story, with evidence, and what that breaks for safety and oversight.

Preventing hallucinations is one job; catching the ones that slip through at runtime is another. The signal is the detection toolkit, why each signal is imperfect, and how you combine them into an action.

Both are conditional compute, but one scales parameters and the other scales depth. The signal is knowing which axis each routes along and why that changes the FLOP story. Here is the sharp version.

RAG and fine-tuning get framed as either/or. RAFT trains the model to be good at RAG itself. The tell is what goes in the training set: relevant docs mixed with distractors. Here is the answer.

Distillation buys you most of a frontier model's quality at a fraction of the serving cost. The signal is naming the three LLM-specific variants and knowing which one works against a closed API. Here is the answer.

Models are trained to be helpful, which quietly trains them to never refuse. Getting honest abstention back is a system problem, not a prompt tweak. Here is the stack that actually moves the refuse-when-unsure rate.

Alignment that adds refusals and politeness often quietly degrades reasoning and coding. The alignment tax is real and measurable. Here is how to keep the safety gains without paying for them in capability.

Optimize any proxy hard enough and the model finds the exploit, not the goal. Sycophancy, padding, and fake citations are reward hacking. Here is why it happens and the controls that actually hold.

Few-shot accuracy can swing on example order alone. If your prompt is fragile to things that shouldn't matter, the fix is structural, not lucky example-hunting. Here is what actually reduces variance.

The per-token sticker price hides the real decision. Self-hosting only wins past a volume break-even most teams misjudge. Here is the back-of-envelope a staff engineer does on the whiteboard.

CoT is not a universal upgrade. On some tasks it does nothing or hurts, and the reason tells you what to try instead. Here is the escalation ladder past 'let's think step by step'.

When a drug name or a Hindi word becomes nine subword tokens, you pay in cost, context, and accuracy. Adding tokens is easy and adding them well is not. Here is the tradeoff.

For open-ended generation there's no ground-truth string to match, so accuracy is meaningless. The field uses a layered mix of automatic, model-based, and human metrics. Here is how to assemble a credible eval.

A few thousand carefully chosen examples can beat a million scraped ones. The signal is knowing which filters matter, how you measure example quality, and why diversity beats raw volume.

Push PPO hard enough and true quality peaks then falls while the reward keeps climbing. The signal is knowing the Gold-vs-proxy gap, the KL budget that bounds it, and how you actually measure when to stop.

DPO is simple and stable, but it can push down the probability of chosen responses, overfit the preference margin, and amplify length bias. The signal is naming these and the variants that fix each.

Replace a learned reward model with a checker that returns right-or-wrong, and reward hacking mostly disappears. The signal is why verifiable rewards beat learned ones for math and code, and where they break.

Distillation is not one method. The signal is knowing when to match logits versus train on generated text, why on-policy distillation beats off-policy, and how reasoning models are distilled.

A reward model is only as good as its labels, and human preference labels are noisy and inconsistent. The signal is measuring inter-annotator agreement and the concrete moves that lift label quality.

The ratio of web, code, books, and math in pretraining quietly decides downstream skills. The signal is knowing how mixtures are chosen, why upsampling helps, and how methods like DoReMi automate it.

Continued pretraining injects domain knowledge that fine-tuning cannot, but naively it wrecks general ability. The signal is the replay ratio, learning-rate rewarming, and how you measure forgetting.

If a benchmark leaked into pretraining, the score measures memorization, not ability. The signal is the detection methods (n-gram overlap, canaries, perturbation tests) and why fresh held-out evals matter.

A reward model turns pairwise preferences into a scalar signal RLHF can optimize. The signal is the Bradley-Terry loss, the base-model and head choices, and how you validate it before trusting it.

RLAIF replaces expensive human labels with an LLM's preferences, which scales but inherits the labeler model's biases. The signal is how AI feedback is collected and where it quietly fails.

Drop the oldest tokens to bound the KV cache and the model collapses. Keep just the first few tokens and it generates indefinitely. The signal is understanding why those first tokens act as an attention sink.

At long context the KV cache, not the weights, fills the GPU. The signal is naming the levers (quantization, token eviction, head sharing, low-rank) and what each one costs in quality.

Most GPU memory in naive LLM serving is wasted on KV-cache fragmentation. PagedAttention borrows virtual memory paging to reclaim it. The signal is explaining the fragmentation problem and how blocks fix it.

Both squeeze an LLM to 4-bit weights, but they decide what to protect very differently. The signal is GPTQ's error-correcting solve versus AWQ's activation-aware scaling, and the calibration each needs.

FP8 is the format behind modern H100-class training and serving. The signal is knowing the two FP8 variants, why a floating format beats INT8 for dynamic range, and where scaling still matters.

VLMs like GPT-4V and LLaVA bolt a vision encoder onto a language model through a projector. The signal is the image-tokens-as-prefix design, the alignment training, and the resolution and hallucination failure modes.

Speech LLMs either transcribe to text or model audio directly as discrete tokens. The signal is the semantic-vs-acoustic token split and why end-to-end audio models beat ASR-plus-LLM pipelines on latency and prosody.

Temperature 0 does not mean deterministic, and the reason is in the GPU kernels, not the sampler. The signal is naming the batch-invariance problem and knowing which fixes are real versus placebo.

Everyone budgets image tokens. Almost nobody budgets image latency. Images are a prefill problem, prompt caching stops paying for itself, and the cheapest answer is often not to call a VLM at all.

A VLM that reads an invoice wrong and a VLM that reads it right and reasons wrong produce the same end-to-end score, and they need opposite fixes. Here is how to decompose a multimodal eval, build the set from real traffic, and probe for the hallucinations that accuracy never catches.

Every team ships English first, then finds that the retriever, the prompt, the latency budget and the cost model each degrade differently in the next five languages. The applied playbook looks nothing like the model-level answer, and the thing that sinks launches is the eval set nobody built.

The modal Applied AI design round. Anyone can draw embed-retrieve-generate. The signal is in chunking, hybrid retrieval, the rerank/latency tradeoff, and how you prove it works. Here is the structure that scores.

Agents are over-applied. The strong answer resists the hype: most tasks want a single structured call, and agents earn their cost only under specific conditions. Here is when to reach for one and how to stop it from compounding errors.

Most RAG systems ship with no real eval, which is why most RAG systems quietly degrade. The signal is decomposing evaluation into retrieval and generation, and measuring faithfulness separately from relevance. Here is the framework that makes RAG improvements measurable.

Enterprise RAG fails on isolation, not retrieval quality. The signal is enforcing tenant and document-level permissions server-side, at retrieval time, so the model can never surface data a user cannot see. Here is the design that survives a security review.

A 2025-2026 question that catches people clinging to dogma in either direction. The signal is a cost, latency, accuracy, and scale tradeoff, plus knowing the 'lost in the middle' failure of long context. Here is the framework for choosing.

Multi-agent is the most over-applied pattern in AI right now. The signal is resisting it unless the task truly needs specialization or parallelism, knowing the supervisor and handoff patterns, and understanding why coordination amplifies failure.

Conversations and agent tasks outgrow the context window, and naive 'stuff the whole history' fails on cost, latency, and lost-in-the-middle. The signal is a tiered memory design: recent buffer, summarized mid-term, retrieved long-term.

Retrieval quality is capped by the query, and raw user queries are often bad for search. The signal is knowing the techniques that rewrite the query before retrieval and which failure each fixes.

Vector RAG quietly fails on two query shapes: multi-hop chains and global 'what are the themes' questions. The signal is knowing exactly what the graph buys you and the construction bill it charges.

Always retrieving is wasteful and sometimes harmful; never retrieving hallucinates. Self-RAG makes retrieval a decision the model controls, then critiques what it got back. The signal is the retrieve-on-demand plus self-critique loop.

Real corpora contradict themselves (old vs new policy, different teams), and naive RAG silently picks one or blends them into a wrong answer. The signal is detecting conflict and resolving it by recency, authority, and transparency.

Code-execution agents are powerful and dangerous: arbitrary model-generated code runs on your infrastructure. The signal is real isolation (containers/VMs), resource limits, and network/filesystem restrictions, not 'trust the model.' Here is the answer.

Agents fail in the middle, not just the end, so final-answer-only scoring hides the real problems and rewards lucky paths. The signal is evaluating the whole trajectory and localizing where it broke. Here is the answer.

Real documents carry images, charts, and tables, not just text, and text-only RAG silently drops them. The signal is knowing the two retrieval approaches and why the generator must change too. Here is the answer.

A computer-use agent drives a real browser or desktop through screenshots and clicks. The signal is the perceive-decide-act loop plus the containment for an agent that can click, buy, or delete anything. Here is the answer.

An agent that takes actions is far riskier than one that just talks. The signal is action-level guardrails: least privilege, argument validation, human approval for irreversible actions, and a blast-radius mindset. Here is the answer.

Single-shot retrieval fails on questions that chain facts ('who directed the highest-grossing film of 2019?'). The signal is decomposing or iterating retrieval instead of retrieving once, and guarding against errors compounding across hops. Here is the answer.

Late interaction gets cross-encoder-like quality with near-bi-encoder scalability. The signal is the per-token embeddings plus MaxSim matching that buys you the middle ground, and knowing what it costs. Here is the answer.

Standard RAG retrieves once and generates; agentic RAG turns retrieval into an iterative, reasoning-driven loop. The signal is the agent deciding whether, what, and when to retrieve, then re-retrieving until the evidence holds.

Late chunking flips the usual order of operations to fix the lost-context problem without a single extra LLM call. The whole signal is which step happens first. Here is the answer.

At a billion vectors the index choice is a memory budget problem before it's a recall problem. Flat search is out, HNSW may not fit, and PQ trades recall for RAM. Here is the decision a staff engineer makes on the whiteboard.

A model that tops MTEB can still flop on your jargon-heavy corpus, and most teams jump straight to the expensive fix. There is a ladder, and the cheap rungs are the ones people skip. Here is the order to climb it.

An agent that retries the same failed action 40 times is a runaway bill and a stuck user. Termination is something you engineer into the harness, not something the model reliably decides. Here is the control layer.

Real tools time out, rate-limit, and return malformed JSON. An agent that assumes every call succeeds is a demo, not a product. Here is the error-handling layer that keeps it alive in production.

When the right answer isn't even in the top-50, the generator can't save you. Low recall has a short list of usual suspects. Here is the order to check them so you fix the cause, not a symptom.

When a chain of LLM calls and tools produces a wrong final answer, 'the model was bad' is not a diagnosis. You need to see every step. Here is the tracing layer that turns a black box into something debuggable.

A 90% eval score and angry users at the same time means your eval set doesn't look like reality. The fix is to make evaluation track production, not the other way around. Here is how.

A live-coding round that separates people who have shipped agents from people who have read about them. The signal is a real plan-act-observe loop, tools that hit an actual SQL database, guardrails that stop the obvious failures, and a concrete evaluation plan, not a single prompt that pretends to be an agent.

Late interaction stores one vector per token, so a corpus that fit in a few GB as single vectors can balloon 100x. The signal is knowing the compression and indexing tricks (centroids, residuals, PLAID) that make multi-vector retrieval shippable.

HyDE generates a fake answer to embed, which helps on vague queries but actively hurts on factual or out-of-domain ones. The signal is naming exactly when the hypothetical document misleads retrieval and which multi-draft and hybrid variants recover.

Vector search over rows is the wrong tool when the user asks for a count or an aggregate. The signal is retrieving the right schema, generating validated SQL, and knowing when to query the database instead of embedding it.

Splitting source files every 500 characters cuts functions in half and destroys retrieval. The signal is chunking on syntax, retrieving by symbol and dependency, and combining lexical exact-match with semantic search the way code search actually needs.

Re-embedding 10M documents nightly is wasteful when only 0.5% changed. The signal is upserting by stable id, handling deletes and tombstones in an ANN index, and compacting before fragmentation tanks recall and latency.

An LLM can cite a source that does not say what the answer claims. The signal is checking claim-by-claim entailment against the cited text, not just that a citation marker exists, and knowing what to do when grounding fails.

Hybrid retrieval only beats either method if the fusion is tuned. The signal is knowing why you cannot just add BM25 and cosine scores, how RRF's k constant behaves, and when a learned weight beats rank fusion.

An off-the-shelf cross-encoder is general; your domain has jargon and relevance rules it never saw. The signal is knowing when fine-tuning pays off, how to mine hard negatives, and how to avoid training a reranker that just memorizes your retriever's mistakes.

A better embedding model is useless if old and new vectors share an index, because their spaces are incompatible. The signal is the dual-index re-embed-then-cutover plan, the cost math, and how to prove the new model is actually better before you flip.

Linear agents commit to one path and can't backtrack. Tree-of-Thoughts and LATS add search over reasoning paths. The signal is knowing what they buy, what they cost, and when a cheaper loop wins.

A context window is not memory. Real agents need a storage architecture: working buffer, episodic recall, and consolidated facts. The signal is the read/write paths and how you keep memory from rotting.

Multi-agent systems coordinate through handoffs and shared state. The signal is knowing the orchestration topologies, what a clean handoff protocol carries, and why naive multi-agent often loses to one good agent.

Most people know MCP exposes tools. The deeper signal is the full primitive set (resources, prompts, sampling, roots) and the patterns: server composition, sampling for nested LLM calls, and scoping access safely.

Final-answer accuracy hides how an agent got there. The signal is trajectory-level metrics: tool-selection and argument accuracy, step efficiency, and matching against reference paths, plus when exact-match is the wrong yardstick.

Per-step accuracy looks fine, yet a 50-step task fails. The signal is understanding compounding error and the techniques (decomposition, verification, checkpointing) that keep long-horizon agents from collapsing.

Any content an agent reads can carry instructions that hijack it. The signal is knowing why filtering can't fully solve injection and which containment controls actually bound the damage when it succeeds.

Agents that drive a browser or screen fail in ways chat agents never do: stale DOM, dynamic pages, wrong clicks. The signal is the grounding and reliability techniques that turn a flaky demo into something usable.

A VLM that reads one page perfectly can still fail an 80-page contract, and the reason is arithmetic before it is model quality. The signal is picking the right architecture and knowing what per-page processing silently loses. Here is the answer.

A 0.87 cosine score on a chunk that answers nothing is not a bug in your vector database. It is the embedding model telling you exactly what it was trained to tell you. Here is how to read that signal instead of thresholding it away.

Two ways to give an agent an action space: emit a JSON tool call per turn, or emit a program that calls the tools itself. One collapses N round-trips into one. The other is the one you can actually audit. Here is how to choose.

Single-turn evals cannot catch failures that history causes, because they have no history. The offline answer is two suites, one deterministic and one simulated, and an honest account of why the simulated user will lie to you about how good your agent is.

The most role-defining judgment call in Applied AI: a shiny benchmark number against the operational price of a system nobody can debug at 2am. Most candidates ship on the number and lose the loop there. Here is the answer that passes.

The from-scratch implementation that frontier labs and NVIDIA actually ask. Writing it proves you understand shapes, the scale, the causal mask, and how heads split and recombine. Here is a correct, readable implementation and the follow-ups.

Beam search is the decoding algorithm everyone references and few can implement correctly. The signal is keeping k hypotheses by cumulative log-prob, summing logs (not multiplying probs), and handling completed sequences and length. Here is a correct implementation and the tradeoffs.

DP feels like pattern-matching magic until you have a method. The signal is a repeatable drill: spot overlapping subproblems and optimal substructure, define the state and recurrence, then memoize or tabulate. Here is that method on a worked example.

BPE is the tokenizer behind GPT-style models, and implementing it proves you understand how subword vocabularies are built, not just that they exist. The signal is the iterative merge of the most frequent adjacent pair.

Implementing conv2d proves you understand what a CNN layer actually computes, not just that you can name it. The signal is correct output-shape math, clean stride/padding handling, and knowing the im2col trick frameworks really use.

Implementing the KV cache proves you understand why decode is efficient: you append the new token's K/V and reuse the rest instead of recomputing. The signal is the append-and-attend-over-cache logic. Here is the implementation.

A classic that rewards the two-pointer insight over brute force. The signal is realizing water at each position is bounded by the min of the max walls on each side, then computing it in one pass. Here is the answer.

Edit distance is the canonical 2D string DP, behind spell-check, diff, and fuzzy matching. The signal is defining the subproblem and the insert/delete/replace recurrence. Here is the answer.

Encoding a tree into a string and rebuilding it exactly hinges on one decision most candidates skip. Miss it and the structure becomes ambiguous. Here is the answer and the interview-grade reasoning.

LIS has an obvious O(n²) DP and a clever O(n log n) solution that surprises people. The signal is the patience-sorting/binary-search approach maintaining 'tails'. Here is both.

Minimum Window Substring is the hard sliding-window problem that tests expand-and-contract with a character-count map. The signal is the grow-to-valid then shrink-to-minimal pattern, checked in O(1) per step.

This famously-hard problem wants better than the O(m+n) merge: an O(log) binary search on the partition. The signal is binary-searching the split point so the left halves stay below the right halves. Here is the answer.

Single-head attention is common; multi-head adds the split-into-heads, attend-per-head, concatenate structure. The signal is the reshape into heads and a clear reason why multiple heads help. Here is the implementation.

Finding the max of every window of size k naively is O(n*k). The trick that drops it to O(n) is a monotonic deque of indices, and the reason it stays linear surprises most candidates.

Comparing every pair of a million documents is a trillion comparisons. MinHash estimates Jaccard similarity from a tiny signature, and LSH turns dedup into a near-linear scan. Here is the implementation.

RoPE is why modern LLMs extrapolate to longer contexts, and it's a rotation, not an addition. Coding it shows you actually understand how position enters attention. Here is the implementation.

Negative sampling in word2vec and recommendation draws millions of weighted samples; doing each in O(log n) is a bottleneck. The alias method makes each draw O(1) after an O(n) setup. Here it is.

When you shard a vector index or route requests across model replicas, naive 'hash mod N' reshuffles everything when N changes. Consistent hashing moves only a small fraction. Here is the implementation.

Negative subtrees, a path that bends through any node, and a return value that differs from the answer you track: this problem packs three traps into one DFS. Here is the clean O(n) solution and the reasoning that survives follow-ups.

The words encode a partial order between characters. Turn each adjacent pair into a directed edge, then topologically sort. The traps are the prefix edge case and detecting contradictions. Here is the full solution with cycle handling.

Words are nodes, one-letter edits are edges, and the shortest ladder is a shortest path in an unweighted graph, so BFS. The make-or-break detail is generating neighbors in O(26 * L) without scanning the whole dictionary. Here is the pattern.

Evaluating arithmetic with precedence and nested parentheses is a parsing problem, and a stack handles both cleanly. The traps are operator precedence, multi-digit numbers, and the sign of truncated division. Here is a single-pass solution.

The canonical multi-round build screen: a simple key-value store that grows new requirements every stage (TTL, transactions, scans). The signal is not stage 1, it is whether your code absorbs stage 4 without a rewrite. Here is how to design for it.

OpenAI's signature build screen: a credit ledger that grants GPU-hours, spends them, and expires unused grants in issue order. The trap is spending against the right grant first. Here is the FIFO-by-expiry design that survives every follow-up.

A classic build screen: a key-value store where writes inside a transaction can be committed or thrown away, and transactions nest. The trap is mutating the base store directly. Here is the overlay-stack design that makes rollback O(1).

Sierra's signature build screen: cells hold values or formulas referencing other cells, and editing one must recompute its dependents without infinite loops. The signal is the dependency graph plus cycle detection. Here is the topological-eval design.

A segment tree answers any associative range query (sum, min, max, gcd) with point or range updates in O(log n). The signal is the recursive split into covered, disjoint, and partial nodes, plus lazy propagation for range updates. Here is the answer.

KMP matches a pattern in linear time by precomputing a failure function that skips redundant comparisons instead of backtracking the text. The signal is what the prefix function actually stores and why the text pointer never moves backward. Here is the answer.

Matrix exponentiation evaluates linear recurrences like Fibonacci at index n in O(log n) by raising a transition matrix to the nth power via binary exponentiation. The signal is building the transition matrix and squaring it. Here is the answer.

A from-scratch test of the EM algorithm and soft clustering. The signal is the two alternating steps (responsibilities then weighted re-estimation), log-sum-exp for stability, and knowing how GMM generalizes k-means. Here is the implementation.

A from-scratch test of recurrent forward passes and backprop through time. The signal is the shared-weight recurrence, accumulating gradients across timesteps, and explaining the vanishing-gradient problem. Here is the implementation.

A from-scratch test of gated recurrence. The signal is wiring the forget/input/output gates and candidate correctly, separating cell state from hidden state, and explaining why the additive cell path fixes vanishing gradients. Here is the implementation.

A from-scratch test of how PyTorch actually works under the hood. The signal is building a computation graph during the forward pass, local derivatives per op, and a topological-order backward pass that accumulates gradients. Here is a minimal engine.

The classic recursive-descent exercise with an applied-AI twist: the JSON your model streams is truncated mid-token for the entire generation. The signal is a clean strict parser plus a small repair layer, not a second parser. Here is the implementation.

Everyone can recite the chunking tradeoff. Far fewer can write the splitter, keep the offsets a citation needs, and answer the question that kills the fixed splitter: what happens to a fact that straddles a boundary.

Every LLM product streams, and almost nobody can write the server. The interesting part is not the async generator, it is what happens when the user hits stop, the provider dies at token 90, or moderation flags text you have already put on the screen.

A sudden 22-point AUC jump is a gift and a warning. Junior candidates celebrate; strong ones get suspicious and know exactly which leakage checks to run before anything ships.

Shipping a model is an experiment, and this question separates people who run A/B tests from people who p-hack them. The signal is pre-registering the metric, sizing the test, and resisting the urge to peek.

EM is the canonical latent-variable algorithm, and a GMM is how it shows up in practice. The signal is the E-step/M-step alternation, why it is soft clustering where k-means is hard, and the honest caveat that it only finds a local optimum. Here is the answer.

Patches as tokens, global attention from layer one, and a weaker inductive bias than a CNN. The signal is naming the data regime where each architecture wins and why. Here is the answer interviewers score highest.

Forward noising is fixed, reverse denoising is learned, and the training loss is a plain noise-prediction regression. The signal is why that beats a GAN's minimax and what the VAE and U-Net each do in latent space. Here is the answer.

Senior DS loops test causal reasoning past the randomized A/B test. The signal is naming the quasi-experimental method that fits and stating the single assumption it lives or dies on. Here is the answer for when randomization is off the table.

Self-supervision is how modern models pretrain on unlabeled data, the engine behind LLMs and modern vision. The signal is the pretext-task idea and the real difference between contrastive and masked-prediction objectives.

Multimodal is now table stakes, and this checks whether you understand how images and text reach a shared model. The signal is CLIP's contrastive alignment and how modern VLMs feed image features into an LLM's token space.

GANs defined generative modeling for years, and interviewers want the minimax game, the failure modes (mode collapse, instability), and the precise reason diffusion displaced them for images while GANs still win on speed.

Autoencoders and VAEs underpin representation learning and generative modeling, including the VAE inside latent diffusion. The signal is the gap between a plain autoencoder and a variational one, and why you cannot backprop through sampling without the reparameterization trick.

Bandits optimize while they learn, shifting traffic toward winners mid-experiment instead of fixing it like an A/B test. The signal is the explore-exploit tradeoff, the three core algorithms, and knowing exactly when a bandit beats a clean A/B test.

HMMs are the classic probabilistic sequence model behind speech and tagging, and Viterbi is how you decode them. The signal is the hidden-states plus transitions/emissions structure, and that Viterbi is dynamic programming for the single best state path, not a probability.

XGBoost and LightGBM win most tabular problems, and interviewers want more than 'it's boosting.' The signal is the fit-to-residuals mechanism plus the engineering (regularization, histograms, second-order) that makes it both fast and accurate.

Contrastive learning is how modern embeddings (CLIP, sentence encoders, SimCLR) are actually trained. The signal is the pull-positives-push-negatives objective, the InfoNCE loss, and why the number and hardness of negatives makes or breaks quality.

Grid and random search ignore past results. Bayesian optimization learns from them, and the signal is whether you can explain the surrogate plus acquisition loop and name the exact condition where the sample-efficiency is worth it. Here is the answer.

Gaussian Processes give predictions with principled uncertainty, which is why they power Bayesian optimization. The signal is the distribution-over-functions intuition, the kernel's role, and naming the O(n cubed) wall. Here is the answer.

Running an A/B test is easy; running a valid one is hard. The signal is naming the traps that produce confidently wrong conclusions and the fix for each. Here is the checklist a senior experimenter carries.

GNNs power recommendations, fraud, and molecule modeling by learning over graph structure. The signal is the message-passing mechanism, why k-hop matters, and why you keep them shallow. Here is the answer.

Two cheap regularizers that fix overconfident classifiers. The signal is knowing that one softens the target and the other softens the input, and exactly why softer signals calibrate the model. Here is the answer.

Ranking is not regression: getting the absolute score wrong is fine, getting the order wrong is not. The signal is the pointwise/pairwise/listwise split and why pairwise (LambdaMART) is still the practical default. Here is the answer.

Classic bias-variance says bigger models eventually overfit, yet deep nets keep improving past the point where they memorize the data. The signal is explaining the second descent and why over-parameterized models generalize. Here is the answer.

Wrap any model and get prediction sets with a provable coverage rate, no distributional assumptions required. The signal is the calibration-set plus nonconformity-score mechanism and exactly what the guarantee does and does not promise. Here is the answer.

Time-to-event problems (churn, failure, conversion timing) hide a twist that quietly biases plain regression. The signal is naming that twist and reaching for the right framing. Here is the answer.

Many real problems give you confirmed positives but never confirmed negatives, only unlabeled data. The shortcut everyone reaches for quietly biases the model. The signal is naming the regime and its fix. Here is the answer.

A pre-existing gap between your groups means randomization or instrumentation is broken, and the whole experiment is suspect. Strong candidates name sample-ratio mismatch on sight. Here is the full diagnosis and the disciplined response.

A 92% aggregate accuracy can hide 60% on the segment that matters most. Average metrics are exactly where these failures hide. Here is how to surface them and the menu of fixes that actually map to the cause.

If labels only exist for the cases you already acted on, the model learns a distorted world: great offline, blind to everyone you never saw. Worse, its own decisions pick the next labels. Here is how to spot it and counter it.

'The model got worse' has three very different causes and three different fixes. Retraining a model that's actually broken by a pipeline bug just bakes in garbage. Here is the triage order.

Small labeled sets are the normal starting condition, not an excuse. The strong answer is a ladder of techniques that squeeze signal from unlabeled data, pretrained models, and the labeling budget. Here it is.

Most real datasets have wrong labels, and they cap your accuracy invisibly. Cleaning all of it by hand doesn't scale. Here is how to find the bad labels and train robustly around them.

Generative training fails in distinctive ways: GANs collapse to a few outputs, diffusion samples come out noisy or blurry. The symptom tells you which knob to turn. Here is the diagnosis.

Metric learning lives or dies on the pairs you feed it. Random negatives teach almost nothing, and the margin and mining strategy decide whether the embeddings are any good. Here is how the choices interact.

For search and recommendation, A/B tests can be slow and noisy because they compare different users. Interleaving compares two rankers within the same user's results and detects winners with far less traffic. Here is how.

Boxes and labels, not one tag per image. The signal is choosing two-stage vs one-stage by your latency and accuracy budget, knowing what NMS and anchors actually do, and reporting mAP correctly. Here is the answer interviewers score highest.

Semantic labels every pixel by class; instance separates each object. The signal is knowing that one cannot count overlapping objects, why Mask R-CNN adds a mask branch on RoIAlign, and what RoIAlign fixed. Here is the answer.

Most candidates can recite 'maximize the margin'. The dual is where you show you actually understand why only support vectors matter and where the kernel trick comes from. Here is the derivation an interviewer wants.

EM on a Gaussian mixture has a famous failure mode: a Gaussian collapses onto one point and likelihood shoots to infinity. Knowing why, and the three standard fixes, separates people who used sklearn from people who understand it.

Both tag sequences, but one is generative and one is discriminative, and that difference decides whether you can throw in overlapping features. Here is the comparison that shows you know the modeling tradeoff, not just the acronyms.

XGBoost made boosting practical; LightGBM made it fast. The answer is histogram binning, gradient-based sampling, and leaf-wise growth, each with a real tradeoff. Here is what they do and where they bite.

Bayesian inference needs an intractable normalizing constant, and MCMC sidesteps it. The signal is explaining the acceptance ratio, detailed balance, and why you can skip the constant entirely. Here is the answer.

Resampling, reweighting, and focal loss attack class imbalance from different angles, and each has a real downside. The signal is matching the method to the model and metric, not blindly oversampling. Here is the breakdown.

IV is the causal tool people name but can't defend. The signal is articulating the two conditions an instrument must satisfy, why one is testable and one is not, and what 2SLS actually estimates. Here is the answer.

Propensity scores promise to mimic an experiment from observational data. The signal is knowing what they can and cannot fix, the overlap trap, and why IPW blows up. Here is the answer that separates careful candidates.

A response model predicts who will convert; an uplift model predicts who converts because of the treatment. The signal is the four-quadrant intuition and how you evaluate uplift when you never observe an individual's lift. Here is the answer.

CUPED can cut the sample size an A/B test needs by half without touching validity. The signal is explaining why subtracting a pre-experiment covariate reduces variance but never biases the estimate. Here is the answer.

Checking an experiment every day and stopping when it hits significance can triple your false-positive rate. The signal is knowing why, and the family of methods that make continuous monitoring valid. Here is the answer.

Standard A/B math assumes one user's treatment doesn't affect another's outcome. In marketplaces and social networks that assumption breaks and your estimate is biased. The signal is naming SUTVA and the right randomization unit. Here is the answer.

The Kalman filter is optimal Bayesian tracking under linear-Gaussian assumptions, and it is two steps repeated forever. The signal is explaining what the gain trades off and when the assumptions break. Here is the answer.

Threshold alerts fire all weekend and miss the real outage Monday. The signal is decomposing seasonality first, choosing the right detector, and tuning for alert fatigue. Here is the answer that survives production.

There is no single fairness number, and a famous result proves you cannot satisfy the main three together. The signal is defining each metric precisely and explaining the impossibility, not picking one blindly. Here is the answer.

Raw gradient saliency maps are noisy and saturate. Integrated gradients fixes both with two axioms and a path integral, but the baseline choice quietly decides the answer. Here is what a careful candidate explains.

Pipelines fail and retry; the real question is whether a retry corrupts your data. The signal is idempotent writes (not 'exactly-once delivery') plus watermarks for late data. Here is how to make re-runs safe without double-counting.

The Databricks-flavored performance question. The signal is going straight to the usual suspects (skew, shuffle, spill) via the Spark UI, not guessing. Here is the diagnostic order and the fixes that actually move the needle.

A hard streaming-systems question: dedup at high throughput with bounded state. The signal is a tiered state design (probabilistic filter in front of durable state) plus watermark-driven eviction. Here is the architecture that does not OOM.

The canonical product-analytics SQL question. The signal is grouping users by signup cohort, computing each activity's period offset, and counting distinct returners per offset to build the retention triangle.

You cannot wait for all the data, and buffering an unbounded stream will OOM. The signal is windowed joins with watermarks for stream-stream, lookup joins for stream-table, and a clear story for late and out-of-order events.

The top cause of mysteriously slow Spark jobs: one partition does most of the work while the rest idle. The signal is reading the symptom (a few straggler tasks) and reaching for the right fix, salting, broadcast, or AQE.

A LATERAL join lets a FROM-clause subquery reference columns from the table before it, which a normal join cannot. The signal is recognizing the per-row use cases: top-N per group, table functions, unnesting arrays.

The single most common way SQL leaks the future into a training set is a careless join to a feature table. Point-in-time correctness is the fix, and it is an as-of join. Here is how to write it.

Funnels look like simple counts until you require the steps happen in order and within a window. Counting each step independently overstates conversion. Here is the ordered-funnel query.

Consecutive-run problems (active streaks, uptime windows, price-stable periods) all reduce to one trick. Once you see the row-number difference, they collapse to a window function and a group-by.

Defining the label is half the modeling problem, and the SQL hides two leakage traps: looking into the future for features, and a label window that isn't fully observed yet. Here is the correct query.

Production drift monitoring often lives in the warehouse, not a fancy tool. Comparing two distributions in SQL with a metric like PSI is a few CTEs. Here is how to compute it.

An org-chart recursion walks a tree, but a general graph has multiple paths and back-edges. The signal is accumulating the visited path to prune cycles and ranking paths by cost to get the shortest one.

Two queries that look identical except for ROWS vs RANGE return different numbers, and most candidates cannot say why. The signal is knowing RANGE groups by value (peers) while ROWS counts physical rows.

Everyone says 'add an index,' but a strong engineer reads the plan first. The signal is finding the dominant operator, comparing estimated vs actual rows, and recognizing the seq-scan, bad-join, and spill patterns.

Partition on the wrong key and you get millions of tiny files or skewed giants. The signal is partitioning on a low-cardinality filter, clustering within partitions on the next predicate, and watching file size.

All three give you ACID on object storage, but they were built for different problems. The signal is matching the format to the workload (engine neutrality, Spark-native ACID, or streaming upserts) rather than naming a favorite.

Overlapping subscription or session windows need collapsing into clean periods, and a naive self-join is quadratic. The signal is the gaps-and-islands trick: a running max end-time to mark where each new island begins.

Events show up minutes or days after they happened, and a window that already closed gives wrong counts. The signal is event-time vs processing-time, watermarks with allowed lateness, and how you correct already-emitted aggregates.

A backfill that double-counts or takes down the live pipeline is worse than the original bug. The signal is idempotent partition-scoped rewrites, isolating backfill compute from live runs, and validating before swapping.

Most candidates only know 'use Debezium.' The signal is comparing three CDC mechanisms by how they capture deletes, load the source, and handle ordering, plus the snapshot-to-stream stitching that trips up real deployments.

Anyone can say 'filter and dedup.' The signal is the funnel in cost order, the MinHash/LSH banding math, and knowing the shuffle at billions of documents is what actually costs you, plus benchmark decontamination people forget until their eval numbers get challenged.

This is two questions hiding as one: how would you have caught it before full rollout, and how do you debug it now. Answer both and you signal senior judgment. Here is the staged-rollout and root-cause playbook.

The modal ML system design round. The structure interviewers reward is the funnel: candidate generation then ranking then re-ranking, with the right model at each stage and an honest plan for cold start, freshness, and feedback loops. Here is that structure.

Extreme class imbalance plus a hard latency budget plus an adversary who adapts. The signal is handling imbalance honestly, choosing the operating point from costs, and designing for the feedback loop. Here is the end-to-end design.

Models fail silently; the question is whether you would know. The signal is monitoring the right layers (operational, data, prediction, outcome) and alerting on drift without drowning in false pages. Here is the system and the metrics that matter.

Multimodal search tests whether you understand shared embedding spaces and the retrieve-then-rank pattern under a real catalog's scale and freshness. The signal is CLIP-style joint embeddings plus hybrid retrieval and honest relevance evaluation. Here is the design.

Text-to-SQL is deceptively hard because correctness is binary and the failure mode is a confident wrong number. The signal is schema grounding, query validation, and a safety layer, not just 'prompt an LLM with the schema.' Here is the production design.

Moderation is a multi-stage classification problem with brutal tradeoffs: false negatives cause real harm, false positives censor legitimate users, and the adversary adapts. The signal is the tiered pipeline, per-severity precision/recall calibration, and human-in-the-loop. Here is the design.

As soon as a company uses LLMs in more than one place, it needs a gateway. The signal is the cross-cutting concerns (cost, reliability, observability, governance) that a gateway centralizes, not just 'proxy the API.' Here is the design.

Ads ranking is where calibrated probabilities meet hard latency and money. The signal is knowing that CTR must be calibrated (not just ranked), the feature and serving design, and the auction context. Here is the design that goes beyond 'train a classifier.'

Anomaly detection sounds easy until seasonality, cold start, and alert fatigue hit. The signal is modeling the expected baseline (including weekly and daily cycles), choosing unsupervised methods for scarce labels, and tuning to avoid drowning users in false alarms. Here is the design.

The canonical AI system-design question. The signal is covering both the model lifecycle and the serving stack under real scale, without rambling. Most candidates design only half and lose the points.

The modern agentic system-design question. The signal is the plan, search, read, synthesize, verify loop with per-claim citations, plus bounding cost and latency. The hard parts are not the writing.

Cross-session memory is what makes an assistant feel personal, and it is mostly a retrieval and state-management problem, not a bigger context window. The signal is the tiered architecture plus what to store, forget, and protect.

Customer support is the killer applied-AI use case, testing agents, RAG, tools, and the all-important human escalation. The signal is knowing when to resolve, when to act, and when to hand off, safely.

Scarce GPUs, dozens of models, every team wanting low latency at low cost. The signal is whether you can turn that into one governed serving fleet: continuous batching, KV cache, per-tenant quotas, and cost you can actually attribute.

AI code review lives or dies on precision: a few noisy comments and the team mutes the bot forever. The signal is grounding in the diff plus repo context, ruthless false-positive control, and earning developer trust one accepted comment at a time.

Summarize, draft, triage, all over the most sensitive PII a person owns. The signal is grounding in the actual thread, matching the user's voice, and a hard human-in-the-loop rule on anything that gets sent.

A GPU-heavy generative system where sampling steps dominate the bill and one bad image is a headline. The signal is the diffusion serving pipeline, the cost levers on per-image GPU time, and a mandatory two-sided safety layer.

Resume screening is high-scale and high-stakes: a legally sensitive, bias-prone decision about people. The signal is balancing throughput with fairness, human oversight, and explainability, not an LLM ranking resumes. Here is the design.

Real-time transcription at scale tests streaming ASR, latency budgets, and GPU fleet management under heavy concurrency. The signal is streaming chunked inference with partial results, not batch transcription. Here is the design.

Document extraction (IDP) is a huge enterprise use case with a hard correctness bar: a wrong extracted number is worse than none. The signal is the parse, extract, validate, human-review pipeline and confidence-based routing. Here is the design.

A voice assistant chains STT, an LLM, and TTS under a brutal latency budget where every stage adds delay. The signal is streaming and pipelining the stages plus turn-taking, not three blocking calls. Here is the design.

Healthcare is the highest-stakes AI domain: a wrong answer can harm a patient, and regulators are watching. The signal is designing decision support with grounding, human oversight, and guardrails, never an autonomous diagnoser. Here is the design.

Legal review is high-stakes and precision-critical: a missed clause or hallucinated citation has real consequences, and lawyers have been sanctioned for fabricated cites. The signal is grounding in the actual documents, citing exact passages, and lawyer-in-the-loop. Here is the design.

Dynamic pricing blends demand forecasting, optimization, and real-time serving, with fairness and trust constraints. The signal is the predict-then-optimize structure plus guardrails against perverse outcomes. Here is the design.

E-commerce search is where retrieval, ranking, and business goals collide, and it carries a hard exact-match requirement (brand, size, SKU). The signal is query understanding plus hybrid retrieval plus business-aware ranking. Here is the design.

A GPU-heavy generative-audio system with text/genre/lyric conditioning, minutes-long coherence, and copyright constraints. The signal is the generation pipeline plus the async, cost, and safety layer that makes it shippable.

Video generation is image generation plus the brutal constraint of temporal consistency, and it is the most GPU-intensive generative task there is. The signal is diffusion over spacetime plus a serving design that survives multi-minute jobs.

Code migration is a high-value agentic task with an unforgiving correctness bar: the migrated code must still work. The signal is grounding in the real repo, transforming in verifiable chunks, and gating on tests, never one giant LLM rewrite.

Notification systems fail by spamming users into muting them, and a mute is a permanently lost channel. The signal is treating it as a per-user send/hold/batch/suppress decision optimized for long-term trust, not for clicks or volume.

Two models chained (ASR then summarization), a long-context problem hiding inside, and a faithfulness bar where a fabricated action item has real consequences. The signal is the pipeline plus how you keep a 3-hour transcript honest at thousands a day.

On-device AI trades raw capability for privacy, offline use, and latency, under brutal memory and battery limits. The signal is the small-quantized-model stack plus a hybrid router that escalates the hard queries to the cloud without leaking private context.

The trap is replacing the classifier with an LLM. The real-time, imbalance, and adversarial constraints do not go away. The signal is a hybrid: a fast calibrated model scores inline, LLMs investigate the gray zone off the hot path.

How large recommenders and search pull candidates from millions of items in milliseconds. The signal is why the user and item towers are separate, how that unlocks precomputed embeddings plus an ANN index, and where ranking takes over.

Every keystroke needs ranked suggestions back in tens of milliseconds, at search-engine QPS. The interesting part is the data structure and what you precompute, not the query path. Here is the design that survives the latency budget.

ETA is spatiotemporal regression served in real time over conditions that change by the minute. The signal is feature freshness, segment or stage decomposition, and the asymmetric cost of being wrong. Here is the design.

PYMK is graph recommendation at billion-node scale. The interviewer is screening for one instinct: do you generate candidates from the social graph, or naively try to score all pairs? Here is the design that survives the follow-ups.

Feed ranking is multi-objective ML at massive scale with a brutal serving constraint. The interviewer wants to see whether you optimize a value model toward long-term satisfaction, or fall into the CTR trap that breeds clickbait. Here is the design.

The service behind every RAG stack and semantic search box. The signal is your ANN index choice and how you handle the three things that quietly break it: metadata filtering, live updates, and scale.

Predict, value, and bid on a single impression in about 10ms, billions of times a day. The signal is the pCTR-to-bid pipeline under a brutal latency budget, plus why calibration and budget pacing decide whether you make or lose money.

Image search is embedding retrieval at billion-image scale. The tell of a strong answer is knowing when a vision-only encoder suffices and when you need a CLIP-style shared space, plus how the ANN index actually serves the query. Here is the design.

Spam detection is adversarial, imbalanced classification under a low-latency bar. The interviewer is watching for whether you set the threshold by cost asymmetry and lean on signals spammers cannot fake. Here is the design.

Query understanding is the front end of search that turns a raw query into structured intent. Get it wrong and even a perfect index returns garbage. The signal is the pipeline and how you measure it. Here is the design.

Online models need features that are fresh and computed exactly the way training computed them. The classic failure is training-serving skew that silently degrades the model. The signal is the online/offline split with one shared definition. Here is the design.

New users and new items have zero interaction history, so collaborative filtering has nothing to work with. The candidates who pass treat it as a lifecycle, not a single trick. Here is how.

Rank each item by relevance alone and you ship ten near-identical results and a filter bubble. The signal interviewers want is a re-ranking stage that scores the list as a set, plus the metric that proves it works.

Some questions need precise multi-hop facts that vector retrieval can't chain. The signal is entity linking plus query translation over a graph (or GraphRAG), and knowing when the graph is worth its maintenance cost. Here is the design.

AV perception is safety-critical, real-time, multimodal ML: detect and track everything around the vehicle from multiple sensors. The signal is sensor fusion plus detection/tracking under a hard latency budget and a fail-safe bar. Here is the design.

Translation is seq2seq generation served across hundreds of language pairs under tight latency and quality budgets. The signal is dodging the quadratic pair explosion, rescuing low-resource languages, and serving it cheaply. Here is the design.

Recognizing walking, driving, or a fall from a sensor stream is windowed time-series classification, usually on a battery-bound device. The signal is windowing, temporal smoothing, and the on-device constraints most candidates skip. Here is the design.

Recommending events (or jobs) breaks normal recsys in two places: items expire and items are local. The signal is how you handle perishability, geography, and the fact that every item is a cold-start item. Here is the design.

A recommender trains on data its own past recommendations produced, so it learns to confirm its own beliefs. The signal is recognizing the loop, naming the biases it breeds, and knowing the exploration and debiasing fixes that break it.

Labeled data is the fuel for ML, and a labeling platform lives or dies on quality control. The signal is the workflow plus the quality math: consensus, gold honeypots, inter-annotator agreement, and active learning to spend the budget where it counts.

Detecting and boxing objects at scale comes down to one driving tradeoff plus a handful of CV specifics most candidates fumble: the detector family, NMS, focal loss, and why mAP not accuracy is the metric. Here is the design.

Forecasting one series is a textbook exercise; forecasting a million SKU-stores that must reconcile, with stockouts costing more than overstock, is the real interview. The strong answer goes global, hierarchical, and cost-aware.

A Devin-style agent that turns an issue into a merged PR is the hardest agent to make reliable, because every step can fail and errors compound. Here is the architecture and the loop that keeps it honest.

Exact-match caching barely helps when no two prompts are identical. Semantic caching reuses answers for similar queries, and its whole risk is returning a near-match that's subtly wrong. Here is how to build it safely.

GPUs are scarce and expensive, so multi-region HA for LLMs is not just web-app HA with bigger boxes. Capacity, routing, and failover all bend around the GPU constraint. Here is the design.

Batch scoring a billion items is a throughput-and-cost problem, the opposite of low-latency serving. Every choice that hurts latency helps you here. Here is how to design for dollars-per-million-documents.

Every team eval-ing prompts in ad-hoc notebooks is how an org ships regressions and argues about vibes. A shared eval platform makes quality measurable and comparable. Here is what it has to provide.

An agent that clicks around a real website is slow, brittle, and one wrong click from a costly mistake. The architecture is about perception, action, and guardrails around irreversible steps. Here is how to build it.

RLHF and evals are only as good as the preference data behind them, and that data is generated by humans whose quality varies wildly. The platform that produces trustworthy labels is itself a serious system. Here is its design.

'What was revenue by region last quarter?' sounds like text-to-SQL, but production analytics agents fail on ambiguity, wrong joins, and confidently wrong numbers. Here is the architecture that makes the answers trustworthy.

An AI support agent handling millions of chats will sometimes be wrong, rude, or unhelpful, and you cannot read them all. Monitoring quality at that scale is its own design problem with its own failure modes.

Classic distributed-systems tradeoffs show up in ML infra with an ML twist: stale features and eventually-consistent reads have model-accuracy consequences, not just correctness ones. Here is how to reason about it.

Anthropic's signature opener. The naive answer (one server pushes to N clients) is bottlenecked by the source uplink and scales linearly. The answer interviewers reward turns receivers into senders so the swarm's capacity grows with its size. Here is that reasoning.

The Dynamo question. Interviewers want three decisions made cleanly: how you partition keys so adding nodes does not reshuffle everything, how you replicate for durability, and where you land on the CAP spectrum. Here is how to reason about all three with concrete quorum math.

Every API platform needs one. Interviewers reward the candidate who picks the right algorithm (token bucket vs sliding window) for the burst behavior, then solves the genuinely hard part: enforcing one shared limit across many servers without a per-request round trip to a central store.

Looks simple until the follow-ups: which layer do you balance at, how do you avoid sending traffic to a dead server, and how is the load balancer itself not a single point of failure? Interviewers want the algorithm, the health-check loop, and the high-availability story.

The observability infrastructure question. Interviewers want you to separate the three telemetry types, choose storage that survives massive write volume, and design alerting that fires fast without drowning on-call in noise. Here is the pipeline with retention and cardinality math.

The distributed ordering question, the conceptual core of Kafka, replicated logs, and consensus systems. Interviewers want to see you confront the hard truth that physical clocks lie, then reach for the actual tools: a single sequencer, consensus, or logical clocks. Here is that progression.

The real-time messaging classic. Interviewers want the delivery path (persistent connections, not polling), the fan-out strategy that does not collapse on large groups, and an ordering and delivery-guarantee story. Here is that design with the fan-out tradeoff that separates seniors.

This is the question to design the scoreboard, not the player. A strong answer separates offline gates (AUC, calibration, NDCG) from the online verdict (A/B with revenue, user, and advertiser guardrails), and adds counterfactual replay so you can trust a model before it ever serves a real auction.

A news feed is not a generic recommender. The clock is a first-class signal: a story that mattered this morning is noise by tonight. The interviewer wants recency decay, an engagement-versus-quality value model, and an answer for filter bubbles, not just retrieval plus ranking.

Truth is not a label you can train on cheaply, and adversaries adapt the moment you ship. A strong answer fuses content, graph, and behavioral signals, puts humans in the loop where precision matters, and treats adversarial drift as a permanent operating condition, not a one-time training problem.

Video search is image search with a time axis, and the time axis is the whole problem. A strong answer embeds frames, pools them into scene vectors, indexes with ANN at billion scale, and explains how a multimodal query (clip, text, or both) finds the right moment, not just the right video.

Inline completion lives or dies on tail latency: a suggestion that arrives after the developer keeps typing is useless. Learn how to build the context window, hide model latency behind speculation and caching, and keep a tight feedback loop on acceptance rate.

Enterprise search lives or dies on permissions, freshness, and connecting to thirty messy SaaS sources; retrieval quality is table stakes. Learn how to fan out across connectors, enforce per-user access at query time, and blend lexical with vector search for results people trust.

Summarizing a 200-page contract is not one LLM call: it is chunking, hierarchical reduction, and a faithfulness check so you never invent facts. Learn the map-reduce pattern, when long-context beats it, and how to evaluate summaries at scale.

Moderating billions of items a day with an LLM on every one is too slow and too expensive. Learn the tiered funnel (cheap classifier then LLM then human), how to tune thresholds for precision versus recall, and how to keep up with adversaries.

Personalizing an LLM is a retrieval and memory problem, not a per-user fine-tune. Learn how to assemble user context at request time, manage long-term memory without bloating the prompt, and respect privacy and the right to be forgotten.

Auto-resolving support tickets means grounding answers in your knowledge base, safely calling real APIs (refunds, account changes), and knowing when to hand off to a human. Learn the agent loop, the guardrails, and how to measure resolution without breaking trust.

Experimenting on LLM features is hard because the outputs are open-ended and quality is fuzzy. Learn how to assign traffic, pick metrics that are not just engagement, handle variance from non-determinism, and avoid the traps that make a winning variant lose in production.

Offering RAG as a product means handling messy ingestion, multi-tenant isolation, and per-customer index freshness, all behind a simple API. Learn the ingestion and query planes, how to keep tenants isolated, and how to give customers eval and observability.

A celebrity posts and 50M followers need a notification. The hard parts are not sending one message, they are fan-out strategy, deduplication, channel routing, and not flooding users. Here is the architecture that handles both the long tail and the viral spike.

A rider taps request and within seconds a nearby driver is assigned. The interesting problems are geospatial indexing of moving drivers, the matching objective (nearest is not always best), and handling the race where two riders want the same car. Here is how to build it.

In payments, a duplicate charge or a lost credit is not a bug, it is a financial incident. The design hinges on idempotency keys, double-entry bookkeeping, and an append-only ledger that can be audited and reconciled. Here is how money systems stay correct.

A cache is easy until you scale it across many nodes. Then you face consistent hashing, eviction policy, the thundering herd on a cache miss, hot keys, and how stale you can tolerate. Here is the design that holds up under real traffic.

Metrics, traces, and IoT data are append-heavy, time-ordered, and rarely updated. A general database handles this badly. The wins come from columnar layout, compression tuned for timestamps, downsampling, and retention. Here is the time-series design interviewers want.

Raw clickstream lands in object storage and somehow becomes clean tables, dashboards, and ML features. The design questions are table format, the bronze-silver-gold layering, batch versus streaming, and how you handle schema drift and late data. Here is the lakehouse blueprint.

Crawling a few pages is trivial. Crawling the web means a URL frontier with priority, politeness per host, dedup at billions of URLs, and defenses against spider traps and infinite content. Here is the crawler architecture that scales without getting your IPs banned.

Background jobs fail, retry, and get redelivered. If a job sends an email or charges a card, running it twice is a real problem. The design is at-least-once delivery plus idempotent handlers, visibility timeouts, and dead-letter queues. Here is how to make jobs safe.

When a page loads, an ad slot is auctioned to dozens of bidders and a winner is chosen before the page finishes rendering, all in under 100ms. The design covers the auction mechanism, the brutal latency budget, budget pacing, and fraud. Here is how an exchange runs millions of auctions per second.

Taking a cloud-API POC into a bank or defense network kills your model provider, your judge API, and your vendor dashboards all at once. Learn the signed-bundle supply chain, local evaluation loop, and zero-egress observability that make an air-gapped LLM stack actually operable.

At thousand-GPU scale, hardware failure is the norm, not the exception, and a run that cannot resume wastes weeks. The signal is checkpointing strategy, deterministic resume, and minimizing lost work. Here is the fault-tolerant design.

A trustworthy experiment platform is far more than a 50/50 split. The signal is consistent assignment, exposure logging, statistical rigor, and guardrails that survive peeking and sample-ratio mismatch. Here is the design.

An ML platform is the internal system that lets many teams build and ship models reliably. The signal is naming the components (data, features, training, registry, serving, monitoring) and arguing why standardizing beats per-team reinvention, plus when not to build.

Same model, two answers: clean offline, ugly online. The culprit is almost always a feature computed differently in the two paths. Here is the diff-based hunt that localizes it to a single column.

A fraud model fed a feature an hour behind is half-blind, but recomputing everything in real time burns money you don't need to spend. Freshness is a per-feature decision on a real cost curve. Here is how to manage it.

'Why was I denied?' is a question regulators and customers will ask, and 'we retrained since then' is not an acceptable answer. Exact reproduction means versioning everything. Here is what 'everything' actually means.

If you wait for labels to measure accuracy, you find out a model broke a month after it broke. Production monitoring has to work before the truth arrives. Here is what you watch instead.

Automating retraining sounds like a convenience until an auto-retrained model silently ships worse. The gates are the whole point. Here is how to automate retraining without automating a regression into production.

Editing a prompt is a code change with no compiler and no unit test by default, so quality regressions ship invisibly. Treating prompts and models as testable artifacts is what separates a toy from a product. Here is the harness.

Every team reinvents a spreadsheet of training runs and then drowns in it. The interviewer wants the platform that ingests runs, params, metrics, and artifacts at high write volume, links them by lineage, and makes thousands of experiments comparable, which is a different system from a model registry.

A green build still means nothing for model quality. The signal is an eval harness that runs deterministically in CI, compares against a frozen baseline, and blocks the merge on regressions. Here is how to make it fast, stable, and trusted.

Promotion is the easy part. Governance is who is allowed to ship what, on whose sign-off, with what documented, and how a model gets retired. Here is the control plane an auditor actually asks for.

'Just set a seed' is the wrong answer. Reproducibility means pinning the environment, the data, the code, and the hardware-level nondeterminism, all at once. Here is the full checklist and where it leaks.

Manual rollback means minutes of damage while someone wakes up. The signal is defining the trigger signals, thresholds, and guardrails that revert automatically, without flapping on noise. Here is how to make the loop safe.

Saying 'route 5% to the canary' is the easy part. The signal is how you split traffic deterministically, gather enough signal to decide, and ramp on evidence rather than vibes. Here is the mechanics layer.

A platform that no one uses is wasted headcount; a platform built too early is premature. The signal is justifying the investment by leverage and designing a paved path teams adopt willingly. Here is the framing.

Four drift tests, four different assumptions. The weak answer lists them; the strong one knows which handles high-dimensional embeddings, which needs binning, and which gives a calibrated p-value. Here is how to choose.

Retrain too often and you burn money and risk regressions; too rarely and the model rots. The strong answer is a layered trigger policy with guardrails, not a single cron job. Here is how to set the cadence.

Classic SRE SLOs assume a request is right or wrong. ML predictions are probabilistic and labels lag, so naive uptime SLOs miss the failures that matter. Here is how to set SLOs that actually cover model quality.

Eyeballing a few outputs does not scale, and a prompt tweak that fixes one case quietly breaks ten. A real LLM eval pipeline is a versioned dataset, layered scorers, and a CI gate. Here is the architecture.

The interviewer wants real numbers, not 'use a big GPU.' Weights are fixed, the KV cache grows with load, and the lever order decides everything. Here is the back-of-envelope and the serving stack.

The interviewer wants you to know why a model that fits on no single GPU still trains, and to do the optimizer-state memory math that motivates sharding. Here is the parallelism taxonomy and the 16-bytes-per-parameter calculation.

Quantization is the first lever for fitting and speeding up models, and the interviewer wants more than 'use fewer bits.' The signal is knowing what precision buys you, why outliers break naive quantization, and how you prove quality held. Here is that answer.

A favorite at hardware-aware shops. The signal is understanding that attention's cost is memory traffic, not FLOPs, and that FlashAttention is an exact, IO-aware reordering, not an approximation. Here is the answer that shows you think about the memory hierarchy.

The KV cache is why LLM serving is hard, and the interviewer wants the mechanics: what it stores, why it limits concurrency, and the tricks that shrink it. Here is the answer that shows you understand decode-time economics.

Decode is sequential and memory-bound, so latency tricks matter. The signal is explaining speculative decoding's draft-and-verify mechanism (and why it stays exact) plus the other levers and when each applies. Here is the latency toolkit.

Mixed precision is standard at scale, and the interviewer wants the numerics: why FP16 needs loss scaling, why BF16 mostly does not, and what stays in FP32. The signal is understanding dynamic range vs precision. Here is that answer.

A hardware-literacy question, especially at NVIDIA and the labs. The signal is understanding throughput vs latency hardware, the memory hierarchy, and why matrix multiplication maps onto GPUs and TPUs. Here is the architecture-aware answer.

At NVIDIA and the labs this compute-intimacy question is real. The signal is owning the SIMT execution model and the three classic throughput killers, with the concrete fix for each. Here is the low-level answer.

FSDP and ZeRO let you train models too big for one GPU without splitting the compute. The signal is the ZeRO stages (what gets sharded at each level) and the communication you pay for it. Here is the answer.

Distributed training is bottlenecked by GPU-to-GPU communication, and these collectives are how the data moves. The signal is what each one does and which parallelism strategy depends on it. Here is the answer.

When a model is too big for one GPU you split the model itself, not just the data. The signal is distinguishing tensor parallelism (split within a layer) from pipeline parallelism (split across layers) and matching each to the interconnect. Here is the answer.

LLM inference has two phases with opposite resource profiles, and co-locating them makes a long prompt stall everyone else's tokens. The signal is knowing why prefill and decode fight, and what separating them costs. Here is the answer.

OOM is the most common wall in deep learning, and 'buy a bigger GPU' is the weakest answer. The signal is naming the memory consumers, knowing which one dominates, and matching the right lever to it.

nvidia-smi showing 100% can hide that you are using a fraction of the hardware's real compute. The signal is MFU (useful FLOPs vs peak) and the gap between 'the GPU is busy' and 'the GPU is efficient'.

Beyond FP16/BF16, FP8 is the next rung on the precision ladder, and the latest tensor cores compute on it natively. The signal is knowing the two formats, what FP8 actually buys, and why dynamic range forces careful scaling.

At long context the KV cache, not the weights, dominates GPU memory and decode bandwidth. The signal is recognizing it as the binding constraint and naming the levers that shrink it without wrecking quality.

Hosting one full fine-tuned model per customer scales linearly in GPUs and bankrupts you fast. There is a way to put hundreds of variants on one GPU without giving up batching. The interviewer wants to hear how.

CPU-based autoscaling that works fine for a web tier quietly fails on GPU inference: wrong signal, and replicas that take minutes to warm. The interviewer wants the signals you actually scale on and how you hide the cold start.

Paying for accelerators that idle half the time is the most common waste in ML training, and the instinct to add more GPUs makes it strictly worse. The interviewer wants the profiling discipline that finds what is starving them.

The counterintuitive truth of LLM serving: token generation is limited by how fast you can read weights from memory, not by math. Once you see that, the whole optimization menu falls out of one number.

Linear scaling is the marketing number; the real curve bends early for reasons that are physics, not bugs. Here is where the speedup goes and how to claw it back.

A 256-GPU job that wedges with no error at 3am is a special kind of pain. The causes are a short, recurring list. Here is the systematic way to find which one bit you.

Spot GPUs are often 60-90% cheaper, and they vanish with two minutes' warning. The savings are only real if a preemption costs you minutes, not the run. Here is how to make that true.

Naive 4-bit quantization can tank quality, and the reflex to give up and serve fp16 leaves a big speedup on the table. The accuracy is usually recoverable. Here is the ladder of fixes.

Users feel the p99, not the median, and the tail is where serving systems quietly fail. The causes are queuing and batching effects, not a slow model. Here is how to flatten it.

Dedicating a GPU per model strands most of a fleet on idle silicon. Sharing safely is a real systems problem with four distinct mechanisms, each tuned to a different traffic shape. Here is how to pick.

A serving box that's stable on short prompts crashes the moment a 30k-token request lands, because KV-cache memory scales with sequence length times batch. Here is how to bound it without crashing.

At billion-parameter scale, training can be cruising and then the loss jumps and never comes back, burning a fortune in compute. The causes and the playbook are well-known to the few who've done it. Here it is.

A single long prompt can freeze every other user's token stream for hundreds of milliseconds. Chunked prefill slices that prompt so decode keeps flowing. Here is the mechanism and the knob that controls it.

A shared system prompt gets re-prefilled on every request unless the server remembers it. Prefix caching skips that work, but only when the blocks line up exactly. Here is the hashing and eviction machinery underneath.

GPU memory runs out long before you run out of conversations to cache. Offloading KV to CPU or NVMe buys capacity, but the interconnect can become the new bottleneck. Here is the bandwidth math.

FP8 serving is fast and usually fine, until it silently degrades on the one workload your benchmark did not cover. The signal is knowing what FP8 breaks and how to validate it. Here is the recipe.

Forcing valid JSON sounds free, but the mask computation can stall every decode step. The fix is precompiling the grammar into a fast automaton. Here is where the cost hides and how to remove it.

Most models do not fill a GPU, so one model per GPU burns money. The three sharing mechanisms (MIG, MPS, swapping) have very different isolation and overhead. Here is how to pick.

Autoscaling reacts in minutes, but a burst hits in seconds, and the gap is where your tail latency dies. Absorbing bursts is about buffers and shedding, not just adding replicas. Here is the playbook.

ZeRO and FSDP both shard training state across data-parallel ranks, but the magic is in when each shard is gathered and freed. Get the memory accounting and the all-gather timing and you can size any run.

Full activation checkpointing saves memory but burns a flat 30% extra compute. Selective recomputation recovers most of that by only recomputing the cheap, memory-heavy operations. Here is how to pick what to recompute.

The collective communication in distributed training is pure overhead unless it runs while the GPU computes. Hiding it is the difference between 30% and 55% MFU. Here is how the overlap actually works and how you confirm it on a trace.

On a thousand GPUs something fails every few hours. The question is not whether you lose a node, but how many GPU-hours you lose when you do. Checkpoint frequency, sharded writes, and fast restart decide that.

When all-reduce is the bottleneck, the cause is almost always a misconfigured path: traffic on the wrong link, a downed NIC, or a topology NCCL never discovered. Here is the systematic way to find the lost bandwidth.

Data, tensor, and pipeline parallelism all leave one sequence's activations on one device, so 200k+ token training hits a wall none of them can fix. The fourth axis shards the sequence itself, and the interview lives in the communication math.

When every request thinks for 30,000 tokens, serving flips from compute-bound prefill to memory-bound decode, and the KV cache becomes the resource you actually schedule. The levers that ruled chat traffic stop being the ones that matter.

Prompt injection has no single fix, and 'sanitize the input' fails the round. The signal is defense in depth: privilege boundaries, treating retrieved content as data not instructions, and a human gate on irreversible actions. Here is the layered answer.

The CISO-facing question that sinks engineers who only think about model quality. The signal is treating governance as architecture, not a checkbox: minimization, isolation, auditability, and never training on customer data by default. Here is the framework.

Shipping an LLM feature safely is an evaluation problem before it is a model problem. The signal is a layered eval-plus-guardrail design with honest, segmented metrics, not a single 'safety classifier.' Here is how to measure and defend each failure mode.

Fairness questions trip up engineers who treat it as a vibe. The signal is knowing the formal fairness metrics conflict mathematically, that you must pick one deliberately for the context, and where in the pipeline bias enters. Here is the rigorous, honest answer.

Most ML security focuses on inference-time attacks; this asks about the training pipeline, where a poisoned dataset or a tampered dependency can plant a backdoor that clean-data evaluation never sees. The signal is knowing the attack classes and that defense is provenance, not a single model fix.

Privacy is a governance requirement that becomes a training-time technique. The signal is knowing what differential privacy actually guarantees (and its cost), and how federated learning and DP combine. Here is the rigorous, honest answer.

Two attacks that hit a deployed model's confidentiality: stealing its functionality through the API, and inferring who was in its training data. The signal is naming the exact mechanism each exploits and that every defense trades against utility.

Federated learning trains across decentralized devices without centralizing data, but a malicious participant can poison the shared model. The signal is the privacy mechanism plus the outlier-resistant aggregation defenses. Here is the answer.

GDPR erasure and copyright takedowns demand removing a user's influence from trained weights, not just a dataset row. The signal is knowing why deletion isn't enough and where retrain, SISA, and approximate scrubbing each land on the cost-versus-proof curve.

Encryption covers data at rest and in transit, but the moment you compute on it the data sits in cleartext in memory. Confidential computing closes that gap. The signal is the TEE concept, attestation, and the AI workloads that actually need it.

Privacy in ML is a toolbox, not a single switch, and each tool defends a different threat. The signal is mapping DP, federated learning, confidential computing, encryption, and minimization to what they actually protect, and knowing they compose.

Federated learning ships gradients, not raw data. The catch: a gradient is computed from the data, so it carries the data. The signal is knowing why 'we only share gradients' is not a privacy guarantee. Here is the answer.

Direct injection is the user attacking the prompt. Indirect injection hides the attack inside a page, doc, or email the model reads. The signal is realizing retrieved and tool data is untrusted and can hijack the model. Here is the answer.

An enterprise RAG that retrieves across everyone's documents is a breach waiting to happen. The fix lives in the retrieval layer, not the prompt. Here is the design that survives a security review.

Large models memorize and can be prompted to emit training examples verbatim, a real privacy and copyright liability. The defenses span data, training, and output. Here is the layered answer.

For consequential decisions, 'the model said so' is not a legal answer. Adverse-action notices and the right to explanation constrain what model you can even ship. Here is the governance view.

AI features open cross-tenant leak paths that classic SaaS isolation never had to think about: shared embeddings, shared caches, and fine-tunes that memorize. One leak ends an enterprise contract. Here is the isolation model.

Deleting a row is easy. Deleting a person's influence from embeddings, caches, derived datasets, and a trained model is not. GDPR and CCPA require it anyway. Here is the plan that survives an audit.

When an agent holds private data, reads untrusted text, and can communicate out, injected instructions can steal data. This 'lethal trifecta' is the defining agent vulnerability. The fix is architectural, not a better prompt.

Before listing defenses, a security engineer maps the attack surface systematically. LLM apps have a wider, weirder surface than classic software. Here is how to threat-model one end to end.

A request that would be refused in one message often succeeds when spread across ten. The signal is understanding that conversational state is part of the attack surface, and that turn-by-turn classifiers see only fragments.

base64, leetspeak, invisible Unicode, and foreign scripts all let a payload read as gibberish to your filter but as a clear instruction to the model. The signal is knowing why the model decodes what the classifier cannot.

Roleplay, obfuscation, optimization-based suffixes, and multi-turn escalation are different attack classes that need different defenses. The signal is organizing the space and pairing each class with a control instead of hoping one filter covers all.

A backdoored model behaves perfectly until it sees a secret trigger, then flips. The signal is explaining why clean test accuracy never reveals it, and what detection actually buys you when the trigger is unknown.

An agent that trusts its tools and its own memory can be steered by a single poisoned record that resurfaces turns or sessions later. The signal is seeing persistent state as an attack surface, not just the live prompt.

A hijacked agent does not need a 'send email' tool to leak secrets. A rendered markdown image, a URL parameter, or a DNS lookup is enough. The signal is enumerating the covert channels and locking down egress, not just tools.

Asking the model to keep secrets or enforce permissions is asking the wrong component. The signal is enforcing access control before retrieval, in code you trust, and treating the LLM as untrusted compute over already-authorized data.

Epsilon is the privacy knob, and shipping DP means choosing a number, defending it, and tracking what each query spends. The signal is treating epsilon as a finite budget with composition, not a magic constant. Here is the answer.

Federated learning keeps data on-device, but it costs you accuracy, debuggability, and engineering complexity. The signal is naming when those costs are justified and when a simpler centralized pipeline with DP wins. Here is the answer.

A fairness audit is a defined process: pick protected attributes, choose metrics that fit the harm, measure disaggregated performance, and document findings. The signal is knowing the metrics conflict and which one the use case demands. Here is the answer.

The EU AI Act assigns different duties to high-risk deployers, GPAI model providers, and limited-risk systems. The signal is naming the specific obligations per tier, not just reciting that risk tiers exist. Here is the answer.

An agent running on one over-privileged service account is a confused deputy waiting to happen. The signal is per-user delegated credentials, token exchange, per-tool scopes, and an audit trail that names the human. Here is the answer.

Every high-risk AI design ends with 'and a human reviews it.' Almost nobody measures whether that review produces any independent signal. Here is how to tell, and how to build oversight that can still disagree.

A failed prod deployment is a stress test of judgment under pressure. Interviewers want to see stabilize-first instincts, clean communication, and a root-cause fix that prevents a repeat. Here is the arc.

Ethical pushback tests integrity and judgment under pressure. Interviewers want principled, specific action, not a generic statement of values. Here is the arc that signals real backbone.

Saving a churning account is diagnosis under pressure: find the real reason, fix what you can, and earn back trust with action. Interviewers want a method, not a discount. Here is the move.