Anthropic Applied AI Engineer interview questions

Almost everyone can quote softmax(QKᵀ/√d_k)V. The interviewer wants the variance argument and the exact training failure the scale prevents. Here is the answer that separates memorization from understanding.

The single most-asked applied GenAI question, and the one most candidates answer as a definition dump. The interviewer wants a decision framework with a default and the conditions that flip it. Here is the call that scores.

Alignment is now table stakes even outside 'safety' roles. The interviewer wants the three-stage RLHF pipeline and a crisp account of why DPO dropped the reward model entirely. Here is the answer with the mechanism, not the buzzwords.

Every customer asks how to stop the model making things up. The weak answer is 'better prompts.' The signal is knowing why models hallucinate and layering grounding, abstention, and measurement into a system you can defend.

These are the knobs every LLM product tunes, and the interviewer wants to know what each does to the probability distribution and which setting fits which task. Including why factual and creative tasks want opposite settings.

Evaluation is the hardest, most underrated part of shipping LLMs. The signal is knowing why public benchmarks mislead, why LLM-as-judge is biased, and how to build a task-specific eval you actually trust.

In-context learning is the property that made prompting a paradigm; chain-of-thought is its most useful trick. The signal is knowing what 'learning' means here (no weight updates) and why making the model reason step by step actually helps. Here is the answer.

Two prompting strategies that buy accuracy with extra compute. The signal is knowing how each explores the reasoning space, that they trade tokens and latency for reliability, and how to decide when the problem actually warrants it. Here is that answer.

ReAct is the pattern behind most tool-using agents. The signal is why interleaving reasoning with actions (and observing results) beats reasoning alone or acting alone. Here is the answer that connects the prompt pattern to real agent behavior.

RL alignment moved from PPO to leaner methods, and DeepSeek-R1 made GRPO famous. The signal is knowing what the value/critic model does in PPO and how GRPO replaces it. Here is the mechanism, not the acronyms.

The difference between a demo prompt and a production prompt is structure, versioning, and defense. The signal is treating the system prompt as a contract and templates as parameterized, testable artifacts. Here is the answer.

Jailbreaking is the adversarial sport of routing a model around its own safety training. The signal is naming the technique families and treating defense as a layered, never-finished arms race, not a one-time patch.

Instruction tuning is what turns a raw next-token predictor into something that follows instructions. The signal is placing it correctly in the pretrain to SFT to alignment pipeline and knowing exactly what it does and does not fix.

Everyone quotes benchmark scores; few read them honestly. The signal is naming what each probes and why leaderboard numbers run ahead of real-world ability. The follow-up the interviewer is holding back is how you would detect contamination.

Fine-tuning is mostly a data problem, not a hyperparameter one. The signal is naming what makes a good set (quality, diversity, format, dedup) and arguing why a few thousand clean examples beat a million noisy ones.

Synthetic data is how teams get training data when real data is scarce, and it has sharp failure modes. The signal is naming when it helps, how to hold the quality line, and why training on model output across generations narrows the distribution.

'Eval is the new system design' for LLM apps, and starting from zero data is the real test. The signal is bootstrapping a golden set, picking task-specific metrics, and running evals continuously as a regression gate.

Everyone fixates on attention, but the feed-forward layers hold most of a transformer's weights and do much of the per-token 'knowledge' work. The signal is knowing what the FFN computes and why it dominates the parameter count.

Context windows are not arbitrary numbers; they fall out of attention's quadratic cost, KV-cache memory, and how the model was trained. The signal is naming all three causes and matching each to the technique that extends it.

RLAIF and Constitutional AI swap human preference labels for AI-generated ones to scale alignment. The signal is naming exactly what they substitute for human feedback, and the consistency-versus-bias tradeoff that swap creates.

Reasoning models are the 2025-2026 frontier. The signal is knowing they are trained (largely via RL on verifiable rewards) to produce long internal reasoning, and that they scale on a second axis: spending more compute at inference.

Tool calling powers every agent. The signal is knowing the model executes nothing: it emits a structured request your code runs, then you feed the result back. Here is the loop and the details that decide reliability.

Modern LLMs dropped learned absolute positions for RoPE and ALiBi to handle long context. The signal is knowing both encode RELATIVE position, and exactly how that enables length extrapolation past the trained window.

DPO simplified RLHF, and a family of variants now trade off what data and reference model they need. The signal is knowing what each variant removes or changes (reference model, paired data, separate SFT stage). Here is the answer.

Scaling laws explain why bigger models trained on more data predictably get better, and Chinchilla changed how we allocate compute. The signal is the model-size-vs-data tradeoff and that many models were under-trained. Here is the answer.

Prompting for JSON works until it doesn't, and at scale the tail breaks your parser. Constrained decoding makes malformed output literally unsamplable. The signal is knowing exactly where in the loop the constraint bites. Here is the answer.

Chain-of-thought looks like the model showing its work, but it may not reflect the real computation. The signal is knowing CoT can be a post-hoc story, with evidence, and what that breaks for safety and oversight.

Preventing hallucinations is one job; catching the ones that slip through at runtime is another. The signal is the detection toolkit, why each signal is imperfect, and how you combine them into an action.

Both are conditional compute, but one scales parameters and the other scales depth. The signal is knowing which axis each routes along and why that changes the FLOP story. Here is the sharp version.

A model advertising a million-token window may not use it. The signal is knowing needle-in-a-haystack, where it breaks, and how to test real multi-fact reasoning across the full context. Here is the answer.

The senior signal is matching the method to what is actually missing instead of defaulting to your favorite. Knowledge, behavior, or actions each point at a different tool. Here is the decision framework that scores highest.

Models are trained to be helpful, which quietly trains them to never refuse. Getting honest abstention back is a system problem, not a prompt tweak. Here is the stack that actually moves the refuse-when-unsure rate.

Alignment that adds refusals and politeness often quietly degrades reasoning and coding. The alignment tax is real and measurable. Here is how to keep the safety gains without paying for them in capability.

Optimize any proxy hard enough and the model finds the exploit, not the goal. Sycophancy, padding, and fake citations are reward hacking. Here is why it happens and the controls that actually hold.

The model's training cutoff is frozen, but your dependencies move weekly. Retraining is the wrong reflex. Here is how teams keep code suggestions current without touching the weights.

Generation has to stop somewhere, and the wrong stop rule either truncates answers or burns tokens on trailing garbage. Here is how the EOS token, stop strings, and max-token caps actually interact.

CoT is not a universal upgrade. On some tasks it does nothing or hurts, and the reason tells you what to try instead. Here is the escalation ladder past 'let's think step by step'.

A model that scores 92% on your eval can still ship a product users hate, because the model is one component in a system. Teams that only do model evals get blindsided. Here is the distinction that matters.

A few thousand carefully chosen examples can beat a million scraped ones. The signal is knowing which filters matter, how you measure example quality, and why diversity beats raw volume.

Push PPO hard enough and true quality peaks then falls while the reward keeps climbing. The signal is knowing the Gold-vs-proxy gap, the KL budget that bounds it, and how you actually measure when to stop.

DPO is simple and stable, but it can push down the probability of chosen responses, overfit the preference margin, and amplify length bias. The signal is naming these and the variants that fix each.

A reward model is only as good as its labels, and human preference labels are noisy and inconsistent. The signal is measuring inter-annotator agreement and the concrete moves that lift label quality.

A reward model turns pairwise preferences into a scalar signal RLHF can optimize. The signal is the Bradley-Terry loss, the base-model and head choices, and how you validate it before trusting it.

RLAIF replaces expensive human labels with an LLM's preferences, which scales but inherits the labeler model's biases. The signal is how AI feedback is collected and where it quietly fails.

Speculative decoding speeds up generation without changing the output distribution. The signal is knowing how draft-model, self-drafting (Medusa/EAGLE), and lookahead approaches differ in where the guesses come from.

Temperature 0 does not mean deterministic, and the reason is in the GPU kernels, not the sampler. The signal is naming the batch-invariance problem and knowing which fixes are real versus placebo.

Everyone budgets image tokens. Almost nobody budgets image latency. Images are a prefill problem, prompt caching stops paying for itself, and the cheapest answer is often not to call a VLM at all.

The modal Applied AI design round. Anyone can draw embed-retrieve-generate. The signal is in chunking, hybrid retrieval, the rerank/latency tradeoff, and how you prove it works. Here is the structure that scores.

Agents are over-applied. The strong answer resists the hype: most tasks want a single structured call, and agents earn their cost only under specific conditions. Here is when to reach for one and how to stop it from compounding errors.

A 2025-2026 question that catches people clinging to dogma in either direction. The signal is a cost, latency, accuracy, and scale tradeoff, plus knowing the 'lost in the middle' failure of long context. Here is the framework for choosing.

Agents and integrations live or die on the model returning valid, schema-conforming output. The signal is layering constrained decoding, schema validation, and retries, not hoping a prompt is enough. Here is the production-reliability answer.

Multi-agent is the most over-applied pattern in AI right now. The signal is resisting it unless the task truly needs specialization or parallelism, knowing the supervisor and handoff patterns, and understanding why coordination amplifies failure.

Conversations and agent tasks outgrow the context window, and naive 'stuff the whole history' fails on cost, latency, and lost-in-the-middle. The signal is a tiered memory design: recent buffer, summarized mid-term, retrieved long-term.

Always retrieving is wasteful and sometimes harmful; never retrieving hallucinates. Self-RAG makes retrieval a decision the model controls, then critiques what it got back. The signal is the retrieve-on-demand plus self-critique loop.

MCP standardized how agents plug into tools and data, but tool design is what makes or breaks reliability. The signal is knowing what MCP actually standardizes and the principles behind a tool the model can call correctly from its description alone.

Reflection is the pattern where an agent critiques and revises its own work. The signal is knowing when it genuinely helps (external feedback grounds the critique) versus when self-critique alone is theater.

Plan-and-Execute and ReAct are the two dominant agent control patterns. The signal is knowing the upfront-plan vs step-by-step tradeoff, why one is cheaper, and why most production agents end up blending them.

Agents need memory beyond the context window, and 'just store the history' is the wrong answer. The signal is distinguishing the memory types and mapping each to a storage and retrieval strategy. Here is the answer.

Context engineering is the discipline that replaced 'prompt engineering' for serious agent work. The signal is understanding that what you put in the context window (and what you leave out) drives behavior more than clever wording. Here is the answer.

Code-execution agents are powerful and dangerous: arbitrary model-generated code runs on your infrastructure. The signal is real isolation (containers/VMs), resource limits, and network/filesystem restrictions, not 'trust the model.' Here is the answer.

Tool selection and parameter extraction are where agents quietly fail. The strongest answer treats the tool spec as the only thing the model sees, then fixes the design, not the model. Here is how.

Agents fail in the middle, not just the end, so final-answer-only scoring hides the real problems and rewards lucky paths. The signal is evaluating the whole trajectory and localizing where it broke. Here is the answer.

Sub-agents are how you scale a complex agent task without one giant, noisy context. The signal is context isolation and clean delegation: a focused subtask in, a compact result out. Here is the answer.

Multi-agent systems live or die on how agents share information and hand off work. The signal is naming the three communication patterns and the coordination failures that sink the naive version. Here is the answer.

The same model performs wildly differently depending on the harness around it. The signal is knowing that the scaffolding often decides agent quality more than the base model. Here is the answer.

A computer-use agent drives a real browser or desktop through screenshots and clicks. The signal is the perceive-decide-act loop plus the containment for an agent that can click, buy, or delete anything. Here is the answer.

An agent that takes actions is far riskier than one that just talks. The signal is action-level guardrails: least privilege, argument validation, human approval for irreversible actions, and a blast-radius mindset. Here is the answer.

A chunk embedded in isolation forgets which document, section, and entity it came from, so it retrieves poorly. Contextual retrieval writes that context back in before indexing. The signal is fixing retrieval at index time, not query time.

Standard RAG retrieves once and generates; agentic RAG turns retrieval into an iterative, reasoning-driven loop. The signal is the agent deciding whether, what, and when to retrieve, then re-retrieving until the evidence holds.

The industry conflates 'agent' with anything that calls an LLM. The useful distinction is how much control you hand to the model, and more autonomy is not better. Here is the spectrum and the decision rule.

A live-coding round that separates people who have shipped agents from people who have read about them. The signal is a real plan-act-observe loop, tools that hit an actual SQL database, guardrails that stop the obvious failures, and a concrete evaluation plan, not a single prompt that pretends to be an agent.

A bridge between classic DSA and modern RAG. The signal is building a working inverted index and a correct BM25 score by hand, reasoning about its complexity, and then knowing exactly when you would reach for embeddings and an ANN index instead.

Splitting source files every 500 characters cuts functions in half and destroys retrieval. The signal is chunking on syntax, retrieving by symbol and dependency, and combining lexical exact-match with semantic search the way code search actually needs.

An LLM can cite a source that does not say what the answer claims. The signal is checking claim-by-claim entailment against the cited text, not just that a citation marker exists, and knowing what to do when grounding fails.

An answer can be correct and still unfaithful, stating things the retrieved context never said. The signal is scoring faithfulness against the context, not against truth, and knowing why LLM-judge faithfulness scores drift.

Linear agents commit to one path and can't backtrack. Tree-of-Thoughts and LATS add search over reasoning paths. The signal is knowing what they buy, what they cost, and when a cheaper loop wins.

A context window is not memory. Real agents need a storage architecture: working buffer, episodic recall, and consolidated facts. The signal is the read/write paths and how you keep memory from rotting.

Tools fail: timeouts, rate limits, bad arguments, garbage output. A naive agent retries blindly or gives up. The signal is classifying errors and matching each to the right recovery, with hard caps.

Multi-agent systems coordinate through handoffs and shared state. The signal is knowing the orchestration topologies, what a clean handoff protocol carries, and why naive multi-agent often loses to one good agent.

Most people know MCP exposes tools. The deeper signal is the full primitive set (resources, prompts, sampling, roots) and the patterns: server composition, sampling for nested LLM calls, and scoping access safely.

Final-answer accuracy hides how an agent got there. The signal is trajectory-level metrics: tool-selection and argument accuracy, step efficiency, and matching against reference paths, plus when exact-match is the wrong yardstick.

Per-step accuracy looks fine, yet a 50-step task fails. The signal is understanding compounding error and the techniques (decomposition, verification, checkpointing) that keep long-horizon agents from collapsing.

Agents can critique and retry their own work. The signal is knowing how Reflexion's verbal feedback loop works, when self-critique genuinely helps versus when it is theater, and what makes it real.

Autonomous agents still need humans at the right moments. The signal is designing checkpoints: where to interrupt, what to surface, how to persist and resume state, without turning the agent into a click-through rubber stamp.

Any content an agent reads can carry instructions that hijack it. The signal is knowing why filtering can't fully solve injection and which containment controls actually bound the damage when it succeeds.

Agents re-send growing context every step, so cost scales worse than linearly. The signal is knowing where the tokens go and the levers (caching, model routing, context trimming, step caps) that cut the bill.

Agents that drive a browser or screen fail in ways chat agents never do: stale DOM, dynamic pages, wrong clicks. The signal is the grounding and reliability techniques that turn a flaky demo into something usable.

Two ways to give an agent an action space: emit a JSON tool call per turn, or emit a program that calls the tools itself. One collapses N round-trips into one. The other is the one you can actually audit. Here is how to choose.

Single-turn evals cannot catch failures that history causes, because they have no history. The offline answer is two suites, one deterministic and one simulated, and an honest account of why the simulated user will lie to you about how good your agent is.

The most role-defining judgment call in Applied AI: a shiny benchmark number against the operational price of a system nobody can debug at 2am. Most candidates ship on the number and lose the loop there. Here is the answer that passes.

A favorite practical screen at the labs, because it tests concurrency, time handling, and judgment in 20 lines. The trap is the background thread that wastes CPU. Here is the lazy-refill version interviewers want, plus the follow-ups.

The from-scratch implementation that frontier labs and NVIDIA actually ask. Writing it proves you understand shapes, the scale, the causal mask, and how heads split and recombine. Here is a correct, readable implementation and the follow-ups.

Implementing the KV cache proves you understand why decode is efficient: you append the new token's K/V and reuse the rest instead of recomputing. The signal is the append-and-attend-over-cache logic. Here is the implementation.

Single-head attention is common; multi-head adds the split-into-heads, attend-per-head, concatenate structure. The signal is the reshape into heads and a clear reason why multiple heads help. Here is the implementation.

The canonical multi-round build screen: a simple key-value store that grows new requirements every stage (TTL, transactions, scans). The signal is not stage 1, it is whether your code absorbs stage 4 without a rewrite. Here is how to design for it.

OpenAI's signature build screen: a credit ledger that grants GPU-hours, spends them, and expires unused grants in issue order. The trap is spending against the right grant first. Here is the FIFO-by-expiry design that survives every follow-up.

A classic build screen: a key-value store where writes inside a transaction can be committed or thrown away, and transactions nest. The trap is mutating the base store directly. Here is the overlay-stack design that makes rollback O(1).

A practical build screen: turn a stream of sampled call stacks into a flame tree, then report self-time vs total-time per function. The trap is conflating the two times. Here is how to aggregate the tree and rank the hot functions.

Sierra's signature build screen: cells hold values or formulas referencing other cells, and editing one must recompute its dependents without infinite loops. The signal is the dependency graph plus cycle detection. Here is the topological-eval design.

Google's 'vibe coding' screen: the spec is deliberately vague and changes mid-interview. The signal isn't the algorithm, it's whether you clarify before coding, structure for change, and keep tests green while the requirements move. Here is how to run that loop.

The classic recursive-descent exercise with an applied-AI twist: the JSON your model streams is truncated mid-token for the entire generation. The signal is a clean strict parser plus a small repair layer, not a second parser. Here is the implementation.

The most job-shaped coding screen in Applied AI: fan out N LLM calls without melting the rate limit or losing the batch to one bad request. The signal is in the retry policy, not the async syntax. Here is the version that passes.

The coding round version of the RAG question every candidate can describe in prose. Embed, index, retrieve, generate, cite. What separates a pass from a fail is the branch you write for when retrieval comes back with nothing worth quoting.

Every LLM product streams, and almost nobody can write the server. The interesting part is not the async generator, it is what happens when the user hits stop, the provider dies at token 90, or moderation flags text you have already put on the screen.

Bayesian inference needs an intractable normalizing constant, and MCMC sidesteps it. The signal is explaining the acceptance ratio, detailed balance, and why you can skip the constant entirely. Here is the answer.

The applied data-wrangling screen: here is a grubby database, produce a clean analysis-ready table. The signal is profiling before transforming, doing set-based cleaning in SQL and row-level fixes in Python, joining without fanning out rows, and validating the output instead of trusting it.

Anyone can say 'filter and dedup.' The signal is the funnel in cost order, the MinHash/LSH banding math, and knowing the shuffle at billions of documents is what actually costs you, plus benchmark decontamination people forget until their eval numbers get challenged.

The canonical AI system-design question. The signal is covering both the model lifecycle and the serving stack under real scale, without rambling. Most candidates design only half and lose the points.

The modern agentic system-design question. The signal is the plan, search, read, synthesize, verify loop with per-claim citations, plus bounding cost and latency. The hard parts are not the writing.

Cross-session memory is what makes an assistant feel personal, and it is mostly a retrieval and state-management problem, not a bigger context window. The signal is the tiered architecture plus what to store, forget, and protect.

AI code review lives or dies on precision: a few noisy comments and the team mutes the bot forever. The signal is grounding in the diff plus repo context, ruthless false-positive control, and earning developer trust one accepted comment at a time.

Summarize, draft, triage, all over the most sensitive PII a person owns. The signal is grounding in the actual thread, matching the user's voice, and a hard human-in-the-loop rule on anything that gets sent.

Legal review is high-stakes and precision-critical: a missed clause or hallucinated citation has real consequences, and lawyers have been sanctioned for fabricated cites. The signal is grounding in the actual documents, citing exact passages, and lawyer-in-the-loop. Here is the design.

Code migration is a high-value agentic task with an unforgiving correctness bar: the migrated code must still work. The signal is grounding in the real repo, transforming in verifiable chunks, and gating on tests, never one giant LLM rewrite.

A Devin-style agent that turns an issue into a merged PR is the hardest agent to make reliable, because every step can fail and errors compound. Here is the architecture and the loop that keeps it honest.

Exact-match caching barely helps when no two prompts are identical. Semantic caching reuses answers for similar queries, and its whole risk is returning a near-match that's subtly wrong. Here is how to build it safely.

Every team eval-ing prompts in ad-hoc notebooks is how an org ships regressions and argues about vibes. A shared eval platform makes quality measurable and comparable. Here is what it has to provide.

An agent that clicks around a real website is slow, brittle, and one wrong click from a costly mistake. The architecture is about perception, action, and guardrails around irreversible steps. Here is how to build it.

RLHF and evals are only as good as the preference data behind them, and that data is generated by humans whose quality varies wildly. The platform that produces trustworthy labels is itself a serious system. Here is its design.

Anthropic's signature opener. The naive answer (one server pushes to N clients) is bottlenecked by the source uplink and scales linearly. The answer interviewers reward turns receivers into senders so the swarm's capacity grows with its size. Here is that reasoning.

The Dynamo question. Interviewers want three decisions made cleanly: how you partition keys so adding nodes does not reshuffle everything, how you replicate for durability, and where you land on the CAP spectrum. Here is how to reason about all three with concrete quorum math.

Every API platform needs one. Interviewers reward the candidate who picks the right algorithm (token bucket vs sliding window) for the burst behavior, then solves the genuinely hard part: enforcing one shared limit across many servers without a per-request round trip to a central store.

The distributed ordering question, the conceptual core of Kafka, replicated logs, and consensus systems. Interviewers want to see you confront the hard truth that physical clocks lie, then reach for the actual tools: a single sequencer, consensus, or logical clocks. Here is that progression.

Inline completion lives or dies on tail latency: a suggestion that arrives after the developer keeps typing is useless. Learn how to build the context window, hide model latency behind speculation and caching, and keep a tight feedback loop on acceptance rate.

Summarizing a 200-page contract is not one LLM call: it is chunking, hierarchical reduction, and a faithfulness check so you never invent facts. Learn the map-reduce pattern, when long-context beats it, and how to evaluate summaries at scale.

Moderating billions of items a day with an LLM on every one is too slow and too expensive. Learn the tiered funnel (cheap classifier then LLM then human), how to tune thresholds for precision versus recall, and how to keep up with adversaries.

Long-running agents that loop, call tools, and spawn sub-tasks need durable execution, not a single request handler. Learn how to checkpoint state, recover from failures, bound runaway loops, and observe every step a non-deterministic agent takes.

You cannot improve an LLM app you cannot see into, and LLM observability is not service monitoring. The signal is tracing multi-step chains, capturing inputs/outputs/tokens/cost, and online quality signals, not just latency and errors.

Caching is one of the biggest LLM cost levers, but 'cache the response' is naive for a non-deterministic system. The signal is distinguishing prompt (prefix) caching from semantic caching and knowing when each is safe.

Editing a prompt is a code change with no compiler and no unit test by default, so quality regressions ship invisibly. Treating prompts and models as testable artifacts is what separates a toy from a product. Here is the harness.

A green build still means nothing for model quality. The signal is an eval harness that runs deterministically in CI, compares against a frozen baseline, and blocks the merge on regressions. Here is how to make it fast, stable, and trusted.

'Just set a seed' is the wrong answer. Reproducibility means pinning the environment, the data, the code, and the hardware-level nondeterminism, all at once. Here is the full checklist and where it leaks.

Eyeballing a few outputs does not scale, and a prompt tweak that fixes one case quietly breaks ten. A real LLM eval pipeline is a versioned dataset, layered scorers, and a CI gate. Here is the architecture.

The interviewer wants real numbers, not 'use a big GPU.' Weights are fixed, the KV cache grows with load, and the lever order decides everything. Here is the back-of-envelope and the serving stack.

The interviewer wants you to know why a model that fits on no single GPU still trains, and to do the optimizer-state memory math that motivates sharding. Here is the parallelism taxonomy and the 16-bytes-per-parameter calculation.

A favorite at hardware-aware shops. The signal is understanding that attention's cost is memory traffic, not FLOPs, and that FlashAttention is an exact, IO-aware reordering, not an approximation. Here is the answer that shows you think about the memory hierarchy.

The KV cache is why LLM serving is hard, and the interviewer wants the mechanics: what it stores, why it limits concurrency, and the tricks that shrink it. Here is the answer that shows you understand decode-time economics.

Decode is sequential and memory-bound, so latency tricks matter. The signal is explaining speculative decoding's draft-and-verify mechanism (and why it stays exact) plus the other levers and when each applies. Here is the latency toolkit.

Continuous batching is the single biggest throughput lever in modern LLM serving. The signal is explaining why static batching idles the GPU on variable-length generation, and why PagedAttention is what makes the fix practical.

Users feel the p99, not the median, and the tail is where serving systems quietly fail. The causes are queuing and batching effects, not a slow model. Here is how to flatten it.

A shared system prompt gets re-prefilled on every request unless the server remembers it. Prefix caching skips that work, but only when the blocks line up exactly. Here is the hashing and eviction machinery underneath.

Round-robin routing wastes a feature that doubles throughput: cache locality. Designing the router means choosing a replica by KV state, not just least-loaded. Here is the two-layer design.

Forcing valid JSON sounds free, but the mask computation can stall every decode step. The fix is precompiling the grammar into a fast automaton. Here is where the cost hides and how to remove it.

Autoscaling reacts in minutes, but a burst hits in seconds, and the gap is where your tail latency dies. Absorbing bursts is about buffers and shedding, not just adding replicas. Here is the playbook.

RLHF with PPO is not one model training, it is four models in the loop at once, three of them on the GPU during every step. The memory math and the generation bottleneck are what trip people up.

Data, tensor, and pipeline parallelism all leave one sequence's activations on one device, so 200k+ token training hits a wall none of them can fix. The fourth axis shards the sequence itself, and the interview lives in the communication math.

When every request thinks for 30,000 tokens, serving flips from compute-bound prefill to memory-bound decode, and the KV cache becomes the resource you actually schedule. The levers that ruled chat traffic stop being the ones that matter.

Prompt injection has no single fix, and 'sanitize the input' fails the round. The signal is defense in depth: privilege boundaries, treating retrieved content as data not instructions, and a human gate on irreversible actions. Here is the layered answer.

The CISO-facing question that sinks engineers who only think about model quality. The signal is treating governance as architecture, not a checkbox: minimization, isolation, auditability, and never training on customer data by default. Here is the framework.

Shipping an LLM feature safely is an evaluation problem before it is a model problem. The signal is a layered eval-plus-guardrail design with honest, segmented metrics, not a single 'safety classifier.' Here is how to measure and defend each failure mode.

Fairness questions trip up engineers who treat it as a vibe. The signal is knowing the formal fairness metrics conflict mathematically, that you must pick one deliberately for the context, and where in the pipeline bias enters. Here is the rigorous, honest answer.

A classic ML-security question that catches people who only know clean-data accuracy. The signal is understanding why models are brittle, the realistic threat model, and that no single defense holds. Here is the honest answer.

LLM endpoints are uniquely exposed: each call can cost real money and burn scarce GPU capacity, and abuse is both a security and a financial problem. The signal is layered controls on tokens and cost, not just requests. Here is the defense.

Most ML security focuses on inference-time attacks; this asks about the training pipeline, where a poisoned dataset or a tampered dependency can plant a backdoor that clean-data evaluation never sees. The signal is knowing the attack classes and that defense is provenance, not a single model fix.

Privacy is a governance requirement that becomes a training-time technique. The signal is knowing what differential privacy actually guarantees (and its cost), and how federated learning and DP combine. Here is the rigorous, honest answer.

Two attacks that hit a deployed model's confidentiality: stealing its functionality through the API, and inferring who was in its training data. The signal is naming the exact mechanism each exploits and that every defense trades against utility.

Governance is where enterprise AI deals are won or lost, and engineers who can speak it stand out. The signal is treating it as documentation, accountability, and risk-tiered controls across the lifecycle, not a legal afterthought.

Red teaming is how you find an AI system's failures before users or attackers do. The signal is a structured program covering the real attack surface (jailbreaks, harmful content, leakage, bias) and feeding every finding back into defenses.

AI fails in ways traditional software does not: bias, hallucination, harmful output, silent quality regression. The strong answer is a real playbook (detect, contain, communicate) plus a blameless post-mortem that ships systemic fixes. Here is the answer.

Detection is imperfect, so the strong answer is layered: regex with checksums for structured PII, ML/NER for the unstructured PII regex cannot see, and a frank account of why you never rely on redaction alone. Here is the answer.

Model cards and datasheets are the standard responsible-AI documentation artifacts, increasingly required by regulation and buyers. The strong answer knows what each documents and that honest disclosure of limitations, not marketing, is the entire point. Here is the answer.

'Add guardrails' is hand-wavy. The strong answer names the concrete input and output checks, the mechanisms that enforce them, and the fail-safe behavior when one trips, plus the honesty that they are imperfect. Here is the implementation answer.

GDPR erasure and copyright takedowns demand removing a user's influence from trained weights, not just a dataset row. The signal is knowing why deletion isn't enough and where retrain, SISA, and approximate scrubbing each land on the cost-versus-proof curve.

Attackers coax models into dumping the hidden system prompt, exposing IP, business logic, or worse, embedded secrets. The signal is treating the prompt as non-secret and defending in layers, not betting on 'never reveal your instructions.'

The EU AI Act tiers obligations by risk, and that classification is a first-order design decision, not a legal footnote. The signal is knowing the four tiers and what high-risk systems must prove before they ship.

Safety is not one number. The candidates who pass name the axes, run benchmarks as a gate, and then explain why benchmarks alone certify nothing. Here is the framing interviewers score highest.

Direct injection is the user attacking the prompt. Indirect injection hides the attack inside a page, doc, or email the model reads. The signal is realizing retrieved and tool data is untrusted and can hijack the model. Here is the answer.

Every third-party model, dataset, and library is an attack surface: backdoors, poisoning, arbitrary code on load, license landmines. The signal is treating models and data as supply-chain artifacts that need provenance and vetting. Here is the answer.

Large models memorize and can be prompted to emit training examples verbatim, a real privacy and copyright liability. The defenses span data, training, and output. Here is the layered answer.

When an agent holds private data, reads untrusted text, and can communicate out, injected instructions can steal data. This 'lethal trifecta' is the defining agent vulnerability. The fix is architectural, not a better prompt.

Before listing defenses, a security engineer maps the attack surface systematically. LLM apps have a wider, weirder surface than classic software. Here is how to threat-model one end to end.

A request that would be refused in one message often succeeds when spread across ten. The signal is understanding that conversational state is part of the attack surface, and that turn-by-turn classifiers see only fragments.

base64, leetspeak, invisible Unicode, and foreign scripts all let a payload read as gibberish to your filter but as a clear instruction to the model. The signal is knowing why the model decodes what the classifier cannot.

Roleplay, obfuscation, optimization-based suffixes, and multi-turn escalation are different attack classes that need different defenses. The signal is organizing the space and pairing each class with a control instead of hoping one filter covers all.

A backdoored model behaves perfectly until it sees a secret trigger, then flips. The signal is explaining why clean test accuracy never reveals it, and what detection actually buys you when the trigger is unknown.

Gibberish-looking token strings appended to a prompt can reliably break refusals, and they transfer between models. The signal is explaining the gradient search that finds them and why output checks beat input pattern matching.

An agent that trusts its tools and its own memory can be steered by a single poisoned record that resurfaces turns or sessions later. The signal is seeing persistent state as an attack surface, not just the live prompt.

A hijacked agent does not need a 'send email' tool to leak secrets. A rendered markdown image, a URL parameter, or a DNS lookup is enough. The signal is enumerating the covert channels and locking down egress, not just tools.

Asking the model to keep secrets or enforce permissions is asking the wrong component. The signal is enforcing access control before retrieval, in code you trust, and treating the LLM as untrusted compute over already-authorized data.

An agent running on one over-privileged service account is a confused deputy waiting to happen. The signal is per-user delegated credentials, token exchange, per-tool scopes, and an audit trail that names the human. Here is the answer.

Every high-risk AI design ends with 'and a human reviews it.' Almost nobody measures whether that review produces any independent signal. Here is how to tell, and how to build oversight that can still disagree.

This question is not about whether you failed; everyone has. It is a test of ownership, debugging rigor, and honesty under pressure. Here is the structure that turns a failure story into a hire signal, and the traps that turn it into a flag.

This is a full interview round (OpenAI's project deep dive, Amazon's Tech Talk), not small talk. It is where they test whether you actually understand the work you claim. The signal is depth, your specific contribution, and the tradeoffs you can defend. Here is how to prepare and deliver it.

Applied AI work is ambiguous by nature, and interviewers (Meta's 'embracing ambiguity,' Anthropic's deliberately under-specified problems) test whether you freeze or drive. The signal is structured progress under uncertainty. Here is the arc that lands.

AI moves fast enough that learning speed is a core competency, and several companies ask it directly. The signal is a concrete method for ramping fast and a real, recent example of applying something new. Here is how to show learning agility, not just claim it.

This question screens ego and growth, not perfection. The interviewer wants to see you absorb hard feedback without defending, own a real mistake in first person, and change because of it. The signal is humility plus a durable, concrete change.

At the frontier labs this is a real gate, not a pleasantry. They reject strong engineers who cannot articulate genuine motivation. The signal is specific, honest alignment between what you want to build and what this company actually does.

Mentoring questions test whether you scale beyond your own output, a senior and leadership signal. The interviewer wants a real example with a tailored approach and a measurable outcome, not 'I helped a junior.'

A soft-sounding question that quietly screens out low-effort candidates and tests real fit. The signal is specific, researched reasons tied to your trajectory, not flattery. Here is the structure that lands.

AI moves fast, requirements shift, and tools change monthly. This probes adaptability. The signal is embracing the change, learning quickly, and adjusting course productively, not resisting it. Especially relevant for applied AI.

AI features are probabilistic, and stakeholders hear 'it works.' Setting honest expectations without killing enthusiasm is a core applied-AI skill. Here is how to frame reliability so trust survives the first mistake.

In an AI company, recommending against AI is a strong signal: it shows judgment over hype. Interviewers use this to find engineers who solve problems rather than apply a favorite hammer. Here is how to tell it.

Every AI team feels the pull between velocity and doing it responsibly. How you navigate that tension, with judgment rather than dogma in either direction, is what this question screens. Here is the answer.

Chasing every new model is as bad as ignoring them all. Interviewers want a repeatable filter for signal versus hype, and the discipline to evaluate on your own problem. Here is that filter.

Ethical pushback tests integrity and judgment under pressure. Interviewers want principled, specific action, not a generic statement of values. Here is the arc that signals real backbone.

The 2025-2026 AI-fluency screen. Interviewers want a concrete daily workflow, a real verification discipline, and a story where the AI was wrong. Vague enthusiasm fails; here is what passes.

The most common opener in an Applied AI loop, and the one candidates most often answer with a taxonomy lecture. The interviewer is screening for two things in ninety seconds, and neither of them is your definition of MLOps.

MoE is why some frontier models have huge parameter counts but serve cheaply. The signal is the sparse-activation idea, the routing mechanism, and the operational cost (memory, load balancing) that the FLOP savings quietly hide.

The two knobs that make or break a RAG system, and the ones candidates hand-wave. The signal is tuning chunking against recall and knowing exactly what dense retrieval misses that BM25 catches. Here is the reasoning, not the rules of thumb.

A classic that tests whether you reach past sorting for the right structure. The signal is the heap solution (O(n log k)), the bucket-sort O(n) trick, and how it changes for an unbounded stream. Here is the answer.

Applied AI rarely waits for certainty, and companies (Amazon's Bias for Action, NVIDIA's speed) test whether you can act decisively without freezing or being reckless. The signal is a reversible, well-reasoned call with a plan to validate. Here is the arc that lands.

Tokenization looks like plumbing until it explains your bill, your latency, and why the model fumbles numbers and rare languages. The signal is knowing how subword tokenizers work and the consequences that fall out of them.

Enterprise RAG fails on isolation, not retrieval quality. The signal is enforcing tenant and document-level permissions server-side, at retrieval time, so the model can never surface data a user cannot see. Here is the design that survives a security review.

The canonical grid DP where each cell depends on three neighbors at once. The signal interviewers want is the min-of-three recurrence, and the reason it has to be min. Here is the answer.

Moderation is a multi-stage classification problem with brutal tradeoffs: false negatives cause real harm, false positives censor legitimate users, and the adversary adapts. The signal is the tiered pipeline, per-severity precision/recall calibration, and human-in-the-loop. Here is the design.