Databricks Applied AI Engineer interview questions

Complex tasks rarely fit one prompt, and free-text LLM output rarely plugs into code cleanly. The signal is decomposing into chained steps and parsing each output. Here is the answer that connects prompting to real pipelines.

'Eval is the new system design' for LLM apps, and starting from zero data is the real test. The signal is bootstrapping a golden set, picking task-specific metrics, and running evals continuously as a regression gate.

This is an architecture and business decision, not a religious one. The signal is weighing capability, cost, privacy, control, and operational burden, then making a call instead of declaring 'open is always better.'

RAG and fine-tuning get framed as either/or. RAFT trains the model to be good at RAG itself. The tell is what goes in the training set: relevant docs mixed with distractors. Here is the answer.

Hand-tuned prompts break on the next model upgrade and don't compose across pipeline stages. Frameworks like DSPy compile prompts against a metric instead. The signal is treating prompting as an optimized program. Here is the answer.

The per-token sticker price hides the real decision. Self-hosting only wins past a volume break-even most teams misjudge. Here is the back-of-envelope a staff engineer does on the whiteboard.

Continued pretraining injects domain knowledge that fine-tuning cannot, but naively it wrecks general ability. The signal is the replay ratio, learning-rate rewarming, and how you measure forgetting.

The same model can score ten points apart on MMLU depending on prompt format and scoring method. The signal is knowing the knobs (log-prob vs generation, few-shot, normalization) that make evals reproducible.

Most GPU memory in naive LLM serving is wasted on KV-cache fragmentation. PagedAttention borrows virtual memory paging to reclaim it. The signal is explaining the fragmentation problem and how blocks fix it.

MoE saves compute but is awkward to serve: experts must be sharded, tokens routed across devices, and batches balanced. The signal is the all-to-all communication and the load-imbalance problem, not the training story.

Grammar-constrained decoding sounds like a solved problem until tokenization misaligns with the grammar. The signal is the token-vs-character boundary mismatch and how prefix automata and token healing fix it.

Temperature 0 does not mean deterministic, and the reason is in the GPU kernels, not the sampler. The signal is naming the batch-invariance problem and knowing which fixes are real versus placebo.

The modal Applied AI design round. Anyone can draw embed-retrieve-generate. The signal is in chunking, hybrid retrieval, the rerank/latency tradeoff, and how you prove it works. Here is the structure that scores.

The two knobs that make or break a RAG system, and the ones candidates hand-wave. The signal is tuning chunking against recall and knowing exactly what dense retrieval misses that BM25 catches. Here is the reasoning, not the rules of thumb.

Most RAG systems ship with no real eval, which is why most RAG systems quietly degrade. The signal is decomposing evaluation into retrieval and generation, and measuring faithfulness separately from relevance. Here is the framework that makes RAG improvements measurable.

Enterprise RAG fails on isolation, not retrieval quality. The signal is enforcing tenant and document-level permissions server-side, at retrieval time, so the model can never surface data a user cannot see. Here is the design that survives a security review.

Every RAG system rests on a vector index, and the interviewer wants to know you understand the recall/latency/memory tradeoff, not just 'use a vector DB.' The signal is why exact search does not scale and how HNSW and IVF make different bets.

Retrieval quality is capped by the query, and raw user queries are often bad for search. The signal is knowing the techniques that rewrite the query before retrieval and which failure each fixes.

Vector RAG quietly fails on two query shapes: multi-hop chains and global 'what are the themes' questions. The signal is knowing exactly what the graph buys you and the construction bill it charges.

Chunking quietly sets the ceiling on RAG quality, and 'split every 500 tokens' leaves recall on the table. The signal is knowing the strategy families and the parent-child trick that decouples retrieval size from context size.

Real corpora contradict themselves (old vs new policy, different teams), and naive RAG silently picks one or blends them into a wrong answer. The signal is detecting conflict and resolving it by recency, authority, and transparency.

The embedding model is the foundation of retrieval, and swapping it is deceptively dangerous. The signal is choosing on domain-relevant retrieval quality (not leaderboard) and knowing that a new model means re-embedding everything. Here is the answer.

Dense vs sparse is the core retrieval tradeoff, and 'just use embeddings' silently loses every exact-match query. The signal is knowing what each representation captures and why production blends them. Here is the answer.

Garbage parsing means garbage retrieval, and naive PDF text extraction destroys tables and reading order. The signal is fixing the parser, not the retriever. Here is the answer.

RAG and agents get expensive and slow fast: retrieval plus reranking plus big-model calls, multiplied across agent steps. The signal is naming the dominant cost first, then the levers that actually move it. Here is the playbook.

A RAG system is only as current as its index, and stale or duplicated content silently degrades answers. The signal is an incremental pipeline with deletes, updates, and recency signals, not a one-time bulk load. Here is the answer.

Forcing every query through the same pipeline burns latency, money, and precision. A router decides what each query actually needs. The signal is naming the four decisions a router makes and how you keep a misroute from sinking the answer.

Small chunks match precisely but starve the LLM of context; big chunks have context but match badly. Parent-child retrieval refuses to pick. The signal is decoupling what you match on from what you feed the model.

Fixed-size chunking splits a coherent idea mid-thought; semantic chunking splits on meaning. The signal is naming the methods (structure-aware, embedding-similarity, LLM-based) and the tradeoffs that decide which one to ship.

RAG failures are usually retrieval failures, yet most teams only measure the final answer. The signal is scoring retrieval on its own (recall@k, precision@k, MRR/nDCG) to isolate exactly where the system breaks.

At a billion vectors the index choice is a memory budget problem before it's a recall problem. Flat search is out, HNSW may not fit, and PQ trades recall for RAM. Here is the decision a staff engineer makes on the whiteboard.

When the right answer isn't even in the top-50, the generator can't save you. Low recall has a short list of usual suspects. Here is the order to check them so you fix the cause, not a symptom.

A 90% eval score and angry users at the same time means your eval set doesn't look like reality. The fix is to make evaluation track production, not the other way around. Here is how.

Late interaction stores one vector per token, so a corpus that fit in a few GB as single vectors can balloon 100x. The signal is knowing the compression and indexing tricks (centroids, residuals, PLAID) that make multi-vector retrieval shippable.

Vector search over rows is the wrong tool when the user asks for a count or an aggregate. The signal is retrieving the right schema, generating validated SQL, and knowing when to query the database instead of embedding it.

Re-embedding 10M documents nightly is wasteful when only 0.5% changed. The signal is upserting by stable id, handling deletes and tombstones in an ANN index, and compacting before fragmentation tanks recall and latency.

An answer can be correct and still unfaithful, stating things the retrieved context never said. The signal is scoring faithfulness against the context, not against truth, and knowing why LLM-judge faithfulness scores drift.

Cranking up top-k raises recall but drowns the answer in noise, and a tight reranker raises precision but can drop the one chunk you needed. The signal is knowing which metric caps the system and how to move both with a retrieve-wide-then-rerank shape.

An off-the-shelf cross-encoder is general; your domain has jargon and relevance rules it never saw. The signal is knowing when fine-tuning pays off, how to mine hard negatives, and how to avoid training a reranker that just memorizes your retriever's mistakes.

A better embedding model is useless if old and new vectors share an index, because their spaces are incompatible. The signal is the dual-index re-embed-then-cutover plan, the cost math, and how to prove the new model is actually better before you flip.

Agents re-send growing context every step, so cost scales worse than linearly. The signal is knowing where the tokens go and the levers (caching, model routing, context trimming, step caps) that cut the bill.

A VLM that reads one page perfectly can still fail an 80-page contract, and the reason is arithmetic before it is model quality. The signal is picking the right architecture and knowing what per-page processing silently loses. Here is the answer.

A 0.87 cosine score on a chunk that answers nothing is not a bug in your vector database. It is the embedding model telling you exactly what it was trained to tell you. Here is how to read that signal instead of thresholding it away.

The most role-defining judgment call in Applied AI: a shiny benchmark number against the operational price of a system nobody can debug at 2am. Most candidates ship on the number and lose the loop there. Here is the answer that passes.

A favorite practical screen at the labs, because it tests concurrency, time handling, and judgment in 20 lines. The trap is the background thread that wastes CPU. Here is the lazy-refill version interviewers want, plus the follow-ups.

A staple ML-coding screen. Writing the loop is easy; the signal is k-means++ init, a real convergence criterion, and knowing the failure modes. Here is the tight implementation plus the follow-ups interviewers always ask.

A from-scratch ML-coding staple that checks whether you actually know the math you use. The signal is the clean gradient (it simplifies to Xᵀ(ŷ - y)/n), numerical stability, and vectorization. Here is the implementation and the details interviewers push on.

A classic that tests whether you reach past sorting for the right structure. The signal is the heap solution (O(n log k)), the bucket-sort O(n) trick, and how it changes for an unbounded stream. Here is the answer.

Merging k sorted sources is the canonical min-heap problem and a real data-engineering pattern (merging sorted shards/streams). The signal is the heap of k heads giving O(N log k). Here is the answer.

Topological sort orders a DAG so dependencies come first, the backbone of build systems, schedulers, and ML/data pipeline DAGs. The signal is Kahn's algorithm (or DFS) plus cycle detection. Here is the answer.

Sampling uniformly from a stream you cannot fit in memory (or whose length you never learn) is a real data-engineering problem. The signal is the replace-with-decreasing-probability trick and a proof that telescopes. Here is the answer.

Comparing every pair of a million documents is a trillion comparisons. MinHash estimates Jaccard similarity from a tiny signature, and LSH turns dedup into a near-linear scan. Here is the implementation.

A wrong split quietly inflates every metric you'll ever report, and the bugs are subtle: leaked groups, shuffled time, preprocessing fit on everything. Here is the split done right.

The textbook variance formula loses catastrophic precision on streaming data and cannot update incrementally. Welford's one-pass update is the standard for online feature normalization. Here it is.

When you shard a vector index or route requests across model replicas, naive 'hash mod N' reshuffles everything when N changes. Consistent hashing moves only a small fraction. Here is the implementation.

A Bloom filter answers 'have I seen this?' using a few bits per item instead of storing the key, trading a rare false positive for a huge memory win. Here is the build, the sizing math, and where it pays off in dedup and serving.

A classic build screen: a key-value store where writes inside a transaction can be committed or thrown away, and transactions nest. The trap is mutating the base store directly. Here is the overlay-stack design that makes rollback O(1).

A segment tree answers any associative range query (sum, min, max, gcd) with point or range updates in O(log n). The signal is the recursive split into covered, disjoint, and partial nodes, plus lazy propagation for range updates. Here is the answer.

Kruskal builds a minimum spanning tree by sorting edges and adding the cheapest that does not form a cycle, using union-find for the cycle check. The signal is the greedy cut property and why disjoint-set is the right tool. Here is the answer.

A from-scratch classic that tests recursion plus the split criterion math. The signal is computing impurity correctly, choosing the best threshold by information gain, and knowing the stopping rules. Here is a clean recursive implementation.

A from-scratch favorite that tests linear algebra fluency. The signal is centering first, using SVD instead of forming the covariance matrix, and reading variance off the singular values. Here is the implementation and the details interviewers push on.

A from-scratch test of the EM algorithm and soft clustering. The signal is the two alternating steps (responsibilities then weighted re-estimation), log-sum-exp for stability, and knowing how GMM generalizes k-means. Here is the implementation.

A from-scratch test of the least-squares closed form and the numerical traps in it. The signal is deriving the normal equations, knowing why you never explicitly invert, and reaching for lstsq or the SVD instead. Here is the implementation.

The classic recursive-descent exercise with an applied-AI twist: the JSON your model streams is truncated mid-token for the entire generation. The signal is a clean strict parser plus a small repair layer, not a second parser. Here is the implementation.

The most job-shaped coding screen in Applied AI: fan out N LLM calls without melting the rate limit or losing the batch to one bad request. The signal is in the retry policy, not the async syntax. Here is the version that passes.

The coding round version of the RAG question every candidate can describe in prose. Embed, index, retrieve, generate, cite. What separates a pass from a fail is the branch you write for when retrieval comes back with nothing worth quoting.

Everyone can recite the chunking tradeoff. Far fewer can write the splitter, keep the offsets a citation needs, and answer the question that kills the fixed splitter: what happens to a fact that straddles a boundary.

A sudden 22-point AUC jump is a gift and a warning. Junior candidates celebrate; strong ones get suspicious and know exactly which leakage checks to run before anything ships.

Ensembles dominate tabular ML, and this question checks whether you know they attack different parts of the error. The signal is bagging-reduces-variance vs boosting-reduces-bias and why GBMs win on tabular data. Here is the answer.

The textbook reflex ('drop one') is usually the wrong instinct, and whether collinearity matters at all hinges on your model and what you need from it. This is the answer that separates rote from understanding.

Everyone can recite 'linearity and normality.' Few can say which assumption matters for predictions versus inference, how to spot a violation in a residual plot, and what to actually do about it. Here is that answer.

The built-in importance scores from XGBoost can rank a random ID above a vital feature, and stakeholders will make decisions on that bar chart. Here is why default importance lies and what a careful answer reports instead.

Accuracy ties are common, and accuracy is rarely the deciding factor anyway. The real tiebreakers are the ones a junior candidate forgets to mention. Here is the checklist a staff engineer runs before shipping either one.

'The model got worse' has three very different causes and three different fixes. Retraining a model that's actually broken by a pipeline bug just bakes in garbage. Here is the triage order.

EM on a Gaussian mixture has a famous failure mode: a Gaussian collapses onto one point and likelihood shoots to infinity. Knowing why, and the three standard fixes, separates people who used sklearn from people who understand it.

Anyone can say 'ensemble of trees'. The real answer is about decorrelating those trees and why that, not just averaging, is what drops variance. Here is what makes the forest more than the sum of its trees.

XGBoost made boosting practical; LightGBM made it fast. The answer is histogram binning, gradient-based sampling, and leaf-wise growth, each with a real tradeoff. Here is what they do and where they bite.

k-means assumes round, equal-size blobs and a known k. DBSCAN finds arbitrary shapes and outliers but has its own knobs. The hard part is judging clusters without labels. Here is the comparison and the evaluation toolkit.

Both explain a single prediction by attributing it to features, but only one rests on a uniqueness theorem. The signal is knowing SHAP's game-theory guarantees, LIME's instability, and when each is the right tool. Here is the answer.

SCD2 separates people who have run pipelines from people who have read about them. The whole answer hinges on one atomic MERGE that closes the old row and opens the new one, and stays correct when the job retries.

Sessionization separates people who reach for a self-join from people who know LAG plus a running sum. Here is the two-pass pattern that scales to billions of events, plus the edge cases interviewers push on.

Pipelines fail and retry; the real question is whether a retry corrupts your data. The signal is idempotent writes (not 'exactly-once delivery') plus watermarks for late data. Here is how to make re-runs safe without double-counting.

The Databricks-flavored performance question. The signal is going straight to the usual suspects (skew, shuffle, spill) via the Spark UI, not guessing. Here is the diagnostic order and the fixes that actually move the needle.

Top-N-per-group is the window-function question every data round asks, and the trap is RANK vs ROW_NUMBER vs DENSE_RANK. The signal is choosing the right ranking function for ties and knowing window frames. Here is the pattern and the tie nuance.

A hard streaming-systems question: dedup at high throughput with bounded state. The signal is a tiered state design (probabilistic filter in front of durable state) plus watermark-driven eviction. Here is the architecture that does not OOM.

A pipeline-design question that rewards matching the architecture to the freshness requirement, not chasing real-time for its own sake. The signal is the latency-vs-complexity tradeoff and knowing why Kappa emerged to kill Lambda's dual codebase. Here is the decision.

A data-warehouse fundamentals question that separates people who model for analytics from people who only know OLTP normalization. The signal is facts vs dimensions, the star schema, and why analytics denormalizes where transactional systems normalize.

The difference between a query that scans a terabyte and one that scans a gigabyte is usually layout, not the engine. The signal is partition pruning, columnar formats, and the small-files problem.

Syncing a warehouse with a live OLTP database is a CDC problem, and the naive 'full reload nightly' or 'query by updated_at' answers have real holes. The signal is log-based CDC and idempotent merges.

Bad data silently poisons everything downstream (dashboards, models, decisions), and 'it ran without error' is not 'it's correct.' The signal is the categories of checks and failing loud at the boundary.

Query tuning is a core data-engineering skill, and the signal is reading the execution plan before changing anything, then applying the right fix. Here is the diagnostic method that separates guessers from engineers.

Dedup is a daily data-engineering task and a common screen. The signal is ROW_NUMBER to keep the right record (not just DISTINCT), defining what 'duplicate' actually means, and handling near-duplicates at scale.

Hierarchies (org charts, bill-of-materials, category trees) need recursion, and a self-join only goes one level deep. The signal is the recursive CTE with its anchor plus recursive members, and knowing how it terminates.

The subtlety in rolling aggregates is the window frame clause, especially ROWS vs RANGE. The signal is the correct frame for a moving window and the tied-rows gotcha that corrupts running totals.

A new column is safe; a rename silently corrupts every dashboard downstream. The signal is knowing which changes are compatible, enforcing data contracts in CI, and running expand-contract migrations for the breaking ones.

You cannot wait for all the data, and buffering an unbounded stream will OOM. The signal is windowed joins with watermarks for stream-stream, lookup joins for stream-table, and a clear story for late and out-of-order events.

The top cause of mysteriously slow Spark jobs: one partition does most of the work while the rest idle. The signal is reading the symptom (a few straggler tasks) and reaching for the right fix, salting, broadcast, or AQE.

Materialized views trade storage and freshness for query speed by precomputing results. The signal is knowing the view-vs-materialized-view tradeoff and the refresh/staleness question.

Exact COUNT(DISTINCT) over billions of rows tracks every unique value, so it gets slow and expensive. The signal is knowing when a sketch like HyperLogLog buys huge memory and speed wins for a sub-percent error budget.

A LATERAL join lets a FROM-clause subquery reference columns from the table before it, which a normal join cannot. The signal is recognizing the per-row use cases: top-N per group, table functions, unnesting arrays.

Modern warehouses store JSON natively, and querying it well separates strong data engineers from the rest. The signal is path access plus unnesting, and a clear call on when to flatten hot fields versus keep the schema-on-read flexibility.

You cannot filter on a window function in WHERE, so people wrap queries in subqueries. QUALIFY filters window results directly. The signal is knowing why WHERE cannot do it and writing top-N-per-group cleanly.

MERGE handles insert-or-update-or-delete in one statement, the backbone of incremental loads and slowly-changing dimensions. The signal is the matched/not-matched logic and what makes a retry safe.

Modern table formats let you query a table as of a past version or timestamp, the backbone of reproducibility, audits, and recovery. The signal is the versioned-snapshot mechanism and where it breaks.

The single most common way SQL leaks the future into a training set is a careless join to a feature table. Point-in-time correctness is the fix, and it is an as-of join. Here is how to write it.

You don't always need a dedicated vector database. When embeddings live next to your relational data, nearest-neighbor in SQL makes filters and joins trivial. Here is how, and the scale where it breaks.

Building a training set from a billion-row table means sampling, and the naive 'ORDER BY random() LIMIT n' sorts the whole table. Stratified sampling needs more care still. Here are both, done efficiently.

Bucketing turns a skewed continuous feature into something a model (or a report) handles well, and the choice between equal-width and equal-count bins changes everything on skewed data. Here is both in SQL.

Production drift monitoring often lives in the warehouse, not a fancy tool. Comparing two distributions in SQL with a metric like PSI is a few CTEs. Here is how to compute it.

The applied data-wrangling screen: here is a grubby database, produce a clean analysis-ready table. The signal is profiling before transforming, doing set-based cleaning in SQL and row-level fixes in Python, joining without fanning out rows, and validating the output instead of trusting it.

An org-chart recursion walks a tree, but a general graph has multiple paths and back-edges. The signal is accumulating the visited path to prune cycles and ranking paths by cost to get the shortest one.

Wide tables with a column per period are painless to read and painful to query. The signal is unpivoting to long format with UNPIVOT or a UNION/cross-join trick, and knowing how NULLs and types bite you.

Two queries that look identical except for ROWS vs RANGE return different numbers, and most candidates cannot say why. The signal is knowing RANGE groups by value (peers) while ROWS counts physical rows.

Everyone says 'add an index,' but a strong engineer reads the plan first. The signal is finding the dominant operator, comparing estimated vs actual rows, and recognizing the seq-scan, bad-join, and spill patterns.

Partition on the wrong key and you get millions of tiny files or skewed giants. The signal is partitioning on a low-cardinality filter, clustering within partitions on the next predicate, and watching file size.

At-least-once duplicates and at-most-once loses data; everyone wants exactly-once and few can explain how Kafka delivers it. The signal is idempotent producers plus transactional read-process-write plus the read-committed isolation that ties them together.

All three give you ACID on object storage, but they were built for different problems. The signal is matching the format to the workload (engine neutrality, Spark-native ACID, or streaming upserts) rather than naming a favorite.

Downstream pipelines break when an upstream team renames a column at 2am. The signal is a data contract: an enforced, versioned agreement on schema and semantics, checked at the producer before bad data spreads.

When a metric looks wrong, lineage tells you which upstream table to blame. The signal is distinguishing table vs column lineage and knowing the three ways to capture it: SQL parsing, runtime hooks, and metadata APIs.

Overlapping subscription or session windows need collapsing into clean periods, and a naive self-join is quadratic. The signal is the gaps-and-islands trick: a running max end-time to mark where each new island begins.

Events show up minutes or days after they happened, and a window that already closed gives wrong counts. The signal is event-time vs processing-time, watermarks with allowed lateness, and how you correct already-emitted aggregates.

A backfill that double-counts or takes down the live pipeline is worse than the original bug. The signal is idempotent partition-scoped rewrites, isolating backfill compute from live runs, and validating before swapping.

A flat path extract is easy; an array of objects three levels deep is where people freeze. The signal is LATERAL FLATTEN / UNNEST to explode arrays into rows and a clear rule for when to flatten vs query in place.

Most candidates only know 'use Debezium.' The signal is comparing three CDC mechanisms by how they capture deletes, load the source, and handle ordering, plus the snapshot-to-stream stitching that trips up real deployments.

Anyone can say 'filter and dedup.' The signal is the funnel in cost order, the MinHash/LSH banding math, and knowing the shuffle at billions of documents is what actually costs you, plus benchmark decontamination people forget until their eval numbers get challenged.

Models fail silently; the question is whether you would know. The signal is monitoring the right layers (operational, data, prediction, outcome) and alerting on drift without drowning in false pages. Here is the system and the metrics that matter.

Text-to-SQL is deceptively hard because correctness is binary and the failure mode is a confident wrong number. The signal is schema grounding, query validation, and a safety layer, not just 'prompt an LLM with the schema.' Here is the production design.

As soon as a company uses LLMs in more than one place, it needs a gateway. The signal is the cross-cutting concerns (cost, reliability, observability, governance) that a gateway centralizes, not just 'proxy the API.' Here is the design.

Anomaly detection sounds easy until seasonality, cold start, and alert fatigue hit. The signal is modeling the expected baseline (including weekly and daily cycles), choosing unsupervised methods for scarce labels, and tuning to avoid drowning users in false alarms. Here is the design.

Scarce GPUs, dozens of models, every team wanting low latency at low cost. The signal is whether you can turn that into one governed serving fleet: continuous batching, KV cache, per-tenant quotas, and cost you can actually attribute.

Document extraction (IDP) is a huge enterprise use case with a hard correctness bar: a wrong extracted number is worse than none. The signal is the parse, extract, validate, human-review pipeline and confidence-based routing. Here is the design.

The service behind every RAG stack and semantic search box. The signal is your ANN index choice and how you handle the three things that quietly break it: metadata filtering, live updates, and scale.

Online models need features that are fresh and computed exactly the way training computed them. The classic failure is training-serving skew that silently degrades the model. The signal is the online/offline split with one shared definition. Here is the design.

Exact-match caching barely helps when no two prompts are identical. Semantic caching reuses answers for similar queries, and its whole risk is returning a near-match that's subtly wrong. Here is how to build it safely.

Batch scoring a billion items is a throughput-and-cost problem, the opposite of low-latency serving. Every choice that hurts latency helps you here. Here is how to design for dollars-per-million-documents.

Every team eval-ing prompts in ad-hoc notebooks is how an org ships regressions and argues about vibes. A shared eval platform makes quality measurable and comparable. Here is what it has to provide.

Prompts buried in source code mean every wording tweak is a full deploy, and nobody knows which prompt is live. Treating prompts as managed, versioned configuration is the fix. Here is the platform.

'What was revenue by region last quarter?' sounds like text-to-SQL, but production analytics agents fail on ambiguity, wrong joins, and confidently wrong numbers. Here is the architecture that makes the answers trustworthy.

Give every team an LLM API key and your bill becomes a mystery with a six-figure surprise at month end. Attributing, limiting, and optimizing spend is a platform feature. Here is how to build it.

Classic distributed-systems tradeoffs show up in ML infra with an ML twist: stale features and eventually-consistent reads have model-accuracy consequences, not just correctness ones. Here is how to reason about it.

Enterprise search lives or dies on permissions, freshness, and connecting to thirty messy SaaS sources; retrieval quality is table stakes. Learn how to fan out across connectors, enforce per-user access at query time, and blend lexical with vector search for results people trust.

Summarizing a 200-page contract is not one LLM call: it is chunking, hierarchical reduction, and a faithfulness check so you never invent facts. Learn the map-reduce pattern, when long-context beats it, and how to evaluate summaries at scale.

Experimenting on LLM features is hard because the outputs are open-ended and quality is fuzzy. Learn how to assign traffic, pick metrics that are not just engagement, handle variance from non-determinism, and avoid the traps that make a winning variant lose in production.

Offering RAG as a product means handling messy ingestion, multi-tenant isolation, and per-customer index freshness, all behind a simple API. Learn the ingestion and query planes, how to keep tenants isolated, and how to give customers eval and observability.

Metrics, traces, and IoT data are append-heavy, time-ordered, and rarely updated. A general database handles this badly. The wins come from columnar layout, compression tuned for timestamps, downsampling, and retention. Here is the time-series design interviewers want.

Raw clickstream lands in object storage and somehow becomes clean tables, dashboards, and ML features. The design questions are table format, the bronze-silver-gold layering, batch versus streaming, and how you handle schema drift and late data. Here is the lakehouse blueprint.

Taking a cloud-API POC into a bank or defense network kills your model provider, your judge API, and your vendor dashboards all at once. Learn the signed-bundle supply chain, local evaluation loop, and zero-egress observability that make an air-gapped LLM stack actually operable.

The naive answer is 'retrain weekly.' The senior answer is a hybrid: drift-triggered retraining with a max-staleness fallback, plus a gate that a fresh model must beat the incumbent. Here is how to reason about it.

A feature store is easy to define and easy to get wrong. The signal is explaining the exact bug it prevents (training-serving skew) and the point-in-time correctness that stops label leakage. Here is the full picture.

The trap is jumping straight to 'add more GPUs.' The signal is profiling first, then applying the cheap, no-retrain levers in the right order. Here is the diagnosis-then-optimize playbook for p99.

The trap is describing plain software CI/CD. The signal is the three things ML adds: data validation, a model-quality gate against a baseline, and versioning data plus model plus code together. Here is what is genuinely different.

Reproducibility is what separates an ML platform from a pile of notebooks. The signal is tracking the full provenance (data, code, config, metrics) and a registry that governs promotion. Here is what to track and why each piece matters.

Deploying a model is not flipping a switch. The signal is knowing what each rollout strategy validates, why shadow is uniquely useful for ML, and keeping rollback one step away. Here is the comparison and when to use each.

Grid search is the wrong answer at scale. The signal is knowing why random beats grid, how Bayesian optimization and early-stopping schemes (Hyperband/ASHA) spend compute smartly, and how to parallelize without stragglers.

GPU spend is usually the biggest line item in an AI org, and this question checks whether you think in utilization, not just capacity. The signal is squeezing per-unit cost (utilization, right-sizing, spot, quantization) before scaling out.

The most common ML production failure is not a code bug, it is a quiet upstream data change. The signal is validating at ingestion (schema plus distribution), data contracts with producers, and failing loud instead of training on garbage.

You cannot improve an LLM app you cannot see into, and LLM observability is not service monitoring. The signal is tracing multi-step chains, capturing inputs/outputs/tokens/cost, and online quality signals, not just latency and errors.

Caching is one of the biggest LLM cost levers, but 'cache the response' is naive for a non-deterministic system. The signal is distinguishing prompt (prefix) caching from semantic caching and knowing when each is safe.

LLMOps is not MLOps with bigger models. The signal is the genuinely new surfaces (prompts as deployable artifacts, eval-driven development, often no training step) and treating prompts with the same version discipline as code.

Routing each request to the right model is one of the biggest LLM cost/latency levers in production. The signal is matching query difficulty to model capability and knowing when the cascade pattern beats a classifier.

Models decay silently as the world shifts, and 'monitor for drift' is too vague. The signal is the actual statistical methods and distinguishing data drift you can detect without labels from concept drift you often cannot.

A high accuracy number hides slice failures, brittleness, and silent data bugs. The signal is naming the test layers that catch what the headline metric can't. Here is the framework that gets scored highest.

When a model regresses or an auditor asks 'how was this built,' guesswork is a failing answer. The signal is tracking the whole chain so any model is reproducible and any past decision is explainable. Here is the chain.

Not every prediction needs a low-latency endpoint, and defaulting to one burns money. The signal is matching the serving pattern to the latency and freshness requirement. Here is the decision and when you'd reverse it.

A bad model in production needs a fast, safe rollback, but ML rollback is trickier than code: the model is data and the truth signal lags. The signal is pre-defined criteria plus a previous version kept warm. Here is the answer.

AI systems get expensive fast, and 'the GPU bill is huge' is not actionable without attribution. The signal is breaking spend down by model, feature, team, and request, then naming the levers. Here is the answer.

Shadow deployment validates a new model on real traffic without exposing users to it. The signal is knowing outputs are served in parallel then discarded, and how it complements canary and A/B. Here is the answer.

ML workflows are multi-step DAGs (ingest, feature, train, eval, deploy), and orchestrators run them reliably. The signal is the DAG/scheduling model plus what's actually ML-specific: data deps, versioned artifacts, drift triggers, eval gates.

An ML platform is the internal system that lets many teams build and ship models reliably. The signal is naming the components (data, features, training, registry, serving, monitoring) and arguing why standardizing beats per-team reinvention, plus when not to build.

Champion-challenger is how you improve a production model without betting the business on a hunch: the live model versus candidates competing on real metrics. The signal is the ongoing-competition framing and disciplined, pre-committed promotion.

Monitoring an ML system is more than CPU and latency; the model can silently rot while the dashboard stays green. The signal is the four-layer taxonomy (operational, data, prediction, outcome) and using inputs as leading indicators because labels lag.

Same model, two answers: clean offline, ugly online. The culprit is almost always a feature computed differently in the two paths. Here is the diff-based hunt that localizes it to a single column.

'Which model is in prod right now, and how did it get there?' should be a one-second lookup with an audit trail. If it isn't, you have a rollback and compliance problem already. Here is the promotion pipeline.

A fraud model fed a feature an hour behind is half-blind, but recomputing everything in real time burns money you don't need to spend. Freshness is a per-feature decision on a real cost curve. Here is how to manage it.

'Why was I denied?' is a question regulators and customers will ask, and 'we retrained since then' is not an acceptable answer. Exact reproduction means versioning everything. Here is what 'everything' actually means.

Automating retraining sounds like a convenience until an auto-retrained model silently ships worse. The gates are the whole point. Here is how to automate retraining without automating a regression into production.

A data engineer renames a column and three production models silently start scoring on garbage. Features are a shared contract, and changing them needs the same care as a breaking API change. Here is the discipline.

An alert that fires constantly gets muted, and a model that fails with no alert is worse. Good ML alerting is a design problem with the same principles as SRE, plus ML-specific twists. Here is how to get it right.

'Works on my machine' is a dependency and environment problem, and for ML it includes CUDA versions and exact library pins that change numerical results. Here is how to make the production environment match training.

Every team reinvents a spreadsheet of training runs and then drowns in it. The interviewer wants the platform that ingests runs, params, metrics, and artifacts at high write volume, links them by lineage, and makes thousands of experiments comparable, which is a different system from a model registry.

A green build still means nothing for model quality. The signal is an eval harness that runs deterministically in CI, compares against a frozen baseline, and blocks the merge on regressions. Here is how to make it fast, stable, and trusted.

'Just set a seed' is the wrong answer. Reproducibility means pinning the environment, the data, the code, and the hardware-level nondeterminism, all at once. Here is the full checklist and where it leaks.

You cannot put a 2 TB dataset in git, and copying it per experiment bankrupts you. The signal is knowing how content-addressed versioning works and when file-level (DVC) versus branch-level (lakeFS) fits. Here is the decision.

Anyone can wire tasks into a DAG. The signal is the patterns that keep it correct under retries and backfills: idempotency, data-aware triggering, and the asset model. Here is what separates a flaky pipeline from a trustworthy one.

A platform that no one uses is wasted headcount; a platform built too early is premature. The signal is justifying the investment by leverage and designing a paved path teams adopt willingly. Here is the framing.

Four drift tests, four different assumptions. The weak answer lists them; the strong one knows which handles high-dimensional embeddings, which needs binning, and which gives a calibrated p-value. Here is how to choose.

Three drifts get casually lumped together, but they have different detectability and different fixes. The interview signal is knowing which you can catch without labels and which one genuinely demands a new model.

Retrain too often and you burn money and risk regressions; too rarely and the model rots. The strong answer is a layered trigger policy with guardrails, not a single cron job. Here is how to set the cadence.

Classic SRE SLOs assume a request is right or wrong. ML predictions are probabilistic and labels lag, so naive uptime SLOs miss the failures that matter. Here is how to set SLOs that actually cover model quality.

A Deployment and a Service will serve a model, but GPUs break every Kubernetes default: scheduling, probes, autoscaling signals, and rollouts. Here is the setup that survives production, and when KServe earns its complexity.

Knowing the algorithms is half the job. The other half is the serving stack that actually delivers throughput inside a latency budget. The signal is matching the server to the workload and naming the four knobs that move the needle.

Picking GPUs is a real applied decision, and 'get the biggest one' misses the point. The signal is knowing which spec binds your workload (VRAM, bandwidth, interconnect) and the train-versus-serve difference that flips the answer.

Autoscaling GPUs is not autoscaling web servers: GPUs are scarce and expensive, model loading is slow, and the right signal is not CPU. The signal is scaling on queue and GPU metrics, taming cold starts, and the scale-to-zero economics.

Under load, a shared inference service must decide whose request runs now. The signal is queuing with priorities, backpressure, and fairness wired into batching, not first-come-first-served until the service falls over.

LLM serving has its own metrics, and one latency number hides the real bottleneck. The signal is splitting prefill from decode and reading GPU utilization as a clue, not a verdict. Here is the diagnostic toolkit.

LLM inference has two phases with opposite resource profiles, and co-locating them makes a long prompt stall everyone else's tokens. The signal is knowing why prefill and decode fight, and what separating them costs. Here is the answer.

These four get conflated constantly, but they live at different layers: data processing, distributed training, and general orchestration. The signal is matching the tool to the workload instead of reaching for the one you know. Here is the answer.

You rarely write a serving stack from scratch. The signal is knowing what frameworks hand you (batching, multi-model, GPU scheduling) and why LLM-specific servers exist at all when general ones already batch.

At long context the KV cache, not the weights, dominates GPU memory and decode bandwidth. The signal is recognizing it as the binding constraint and naming the levers that shrink it without wrecking quality.

A freshly started replica is slow on its first requests while it loads weights and compiles kernels, and that clashes with reactive autoscaling. The signal is naming the cold-start sources and the warmup and provisioning fixes.

Hosting one full fine-tuned model per customer scales linearly in GPUs and bankrupts you fast. There is a way to put hundreds of variants on one GPU without giving up batching. The interviewer wants to hear how.

CPU-based autoscaling that works fine for a web tier quietly fails on GPU inference: wrong signal, and replicas that take minutes to warm. The interviewer wants the signals you actually scale on and how you hide the cold start.

The counterintuitive truth of LLM serving: token generation is limited by how fast you can read weights from memory, not by math. Once you see that, the whole optimization menu falls out of one number.

Spot GPUs are often 60-90% cheaper, and they vanish with two minutes' warning. The savings are only real if a preemption costs you minutes, not the run. Here is how to make that true.

Naive 4-bit quantization can tank quality, and the reflex to give up and serve fp16 leaves a big speedup on the table. The accuracy is usually recoverable. Here is the ladder of fixes.

Dedicating a GPU per model strands most of a fleet on idle silicon. Sharing safely is a real systems problem with four distinct mechanisms, each tuned to a different traffic shape. Here is how to pick.

Launching an LLM service on a vibes-based GPU count is how you get paged on day one. Capacity planning for LLMs differs from web services because tokens, not requests, are the unit. Here is how to size it.

A single long prompt can freeze every other user's token stream for hundreds of milliseconds. Chunked prefill slices that prompt so decode keeps flowing. Here is the mechanism and the knob that controls it.

A shared system prompt gets re-prefilled on every request unless the server remembers it. Prefix caching skips that work, but only when the blocks line up exactly. Here is the hashing and eviction machinery underneath.

Round-robin routing wastes a feature that doubles throughput: cache locality. Designing the router means choosing a replica by KV state, not just least-loaded. Here is the two-layer design.

Three batching strategies, three very different latency profiles. Picking wrong leaves throughput or tail latency on the floor. Here is what each one costs and when to use it.

GPU memory runs out long before you run out of conversations to cache. Offloading KV to CPU or NVMe buys capacity, but the interconnect can become the new bottleneck. Here is the bandwidth math.

FP8 serving is fast and usually fine, until it silently degrades on the one workload your benchmark did not cover. The signal is knowing what FP8 breaks and how to validate it. Here is the recipe.

Forcing valid JSON sounds free, but the mask computation can stall every decode step. The fix is precompiling the grammar into a fast automaton. Here is where the cost hides and how to remove it.

Most models do not fill a GPU, so one model per GPU burns money. The three sharing mechanisms (MIG, MPS, swapping) have very different isolation and overhead. Here is how to pick.

When every request thinks for 30,000 tokens, serving flips from compute-bound prefill to memory-bound decode, and the KV cache becomes the resource you actually schedule. The levers that ruled chat traffic stop being the ones that matter.

The CISO-facing question that sinks engineers who only think about model quality. The signal is treating governance as architecture, not a checkbox: minimization, isolation, auditability, and never training on customer data by default. Here is the framework.

LLM apps leak secrets in three quiet places: the prompt, the logs, and the next training set. Each needs a different control. Here is how to close all three.

AI features open cross-tenant leak paths that classic SaaS isolation never had to think about: shared embeddings, shared caches, and fine-tunes that memorize. One leak ends an enterprise contract. Here is the isolation model.

Asking the model to keep secrets or enforce permissions is asking the wrong component. The signal is enforcing access control before retrieval, in code you trust, and treating the LLM as untrusted compute over already-authorized data.

In an AI company, recommending against AI is a strong signal: it shows judgment over hype. Interviewers use this to find engineers who solve problems rather than apply a favorite hammer. Here is how to tell it.

Most AI pilots fail not on the model but on scoping: vague success criteria, the wrong use case, or data that was never ready. For forward-deployed and applied roles, running a pilot well is the job. Here is the playbook.

Chasing every new model is as bad as ignoring them all. Interviewers want a repeatable filter for signal versus hype, and the discipline to evaluate on your own problem. Here is that filter.

Scope creep quietly kills deployments and burns trust on both sides. Interviewers want to see you protect the timeline without becoming the team that says no to everything. Here is the move.

A failed prod deployment is a stress test of judgment under pressure. Interviewers want to see stabilize-first instincts, clean communication, and a root-cause fix that prevents a repeat. Here is the arc.

Executives buy outcomes and care about risk, not your architecture. Interviewers want to see you lead with the decision, quantify in their terms, and surface risk honestly. Here is the pattern.

Vague asks are the norm in forward-deployed work. Interviewers want to see structured discovery that finds the real problem and a measurable first win, not a rush to build. Here is the method.

Building it is not adopting it. Interviewers want to see you win a beachhead, prove value, and let advocates spread it, not mandate usage from the top. Here is the playbook.

Saving a churning account is diagnosis under pressure: find the real reason, fix what you can, and earn back trust with action. Interviewers want a method, not a discount. Here is the move.

A forward-deployed engagement succeeds only if the customer can run it without you. Interviewers want enablement and a real handoff plan, not a dependency. Here is how to build self-sufficiency.

Saying no to a customer is a relationship skill: protect them from a bad outcome while keeping their trust. Interviewers want the reason, the alternative, and the framing. Here is the move.

The most common opener in an Applied AI loop, and the one candidates most often answer with a taxonomy lecture. The interviewer is screening for two things in ninety seconds, and neither of them is your definition of MLOps.

Applied AI rarely waits for certainty, and companies (Amazon's Bias for Action, NVIDIA's speed) test whether you can act decisively without freezing or being reckless. The signal is a reversible, well-reasoned call with a plan to validate. Here is the arc that lands.

Moderation is a multi-stage classification problem with brutal tradeoffs: false negatives cause real harm, false positives censor legitimate users, and the adversary adapts. The signal is the tiered pipeline, per-severity precision/recall calibration, and human-in-the-loop. Here is the design.

At thousand-GPU scale, hardware failure is the norm, not the exception, and a run that cannot resume wastes weeks. The signal is checkpointing strategy, deterministic resume, and minimizing lost work. Here is the fault-tolerant design.

Applied AI is half translation. Interviewers ask this to see whether you can drop the jargon, frame tradeoffs in business terms, and earn trust with people who control the roadmap. Here is how to show that skill in the answer itself.

This is two questions hiding as one: how would you have caught it before full rollout, and how do you debug it now. Answer both and you signal senior judgment. Here is the staged-rollout and root-cause playbook.

Extreme class imbalance plus a hard latency budget plus an adversary who adapts. The signal is handling imbalance honestly, choosing the operating point from costs, and designing for the feedback loop. Here is the end-to-end design.

This question is not about whether you failed; everyone has. It is a test of ownership, debugging rigor, and honesty under pressure. Here is the structure that turns a failure story into a hire signal, and the traps that turn it into a flag.

This is a full interview round (OpenAI's project deep dive, Amazon's Tech Talk), not small talk. It is where they test whether you actually understand the work you claim. The signal is depth, your specific contribution, and the tradeoffs you can defend. Here is how to prepare and deliver it.