rag

The single most-asked applied GenAI question, and the one most candidates answer as a definition dump. The interviewer wants a decision framework with a default and the conditions that flip it. Here is the call that scores.

Every customer asks how to stop the model making things up. The weak answer is 'better prompts.' The signal is knowing why models hallucinate and layering grounding, abstention, and measurement into a system you can defend.

RAG and fine-tuning get framed as either/or. RAFT trains the model to be good at RAG itself. The tell is what goes in the training set: relevant docs mixed with distractors. Here is the answer.

The senior signal is matching the method to what is actually missing instead of defaulting to your favorite. Knowledge, behavior, or actions each point at a different tool. Here is the decision framework that scores highest.

Models are trained to be helpful, which quietly trains them to never refuse. Getting honest abstention back is a system problem, not a prompt tweak. Here is the stack that actually moves the refuse-when-unsure rate.

The model's training cutoff is frozen, but your dependencies move weekly. Retraining is the wrong reflex. Here is how teams keep code suggestions current without touching the weights.

Every team ships English first, then finds that the retriever, the prompt, the latency budget and the cost model each degrade differently in the next five languages. The applied playbook looks nothing like the model-level answer, and the thing that sinks launches is the eval set nobody built.

The modal Applied AI design round. Anyone can draw embed-retrieve-generate. The signal is in chunking, hybrid retrieval, the rerank/latency tradeoff, and how you prove it works. Here is the structure that scores.

The two knobs that make or break a RAG system, and the ones candidates hand-wave. The signal is tuning chunking against recall and knowing exactly what dense retrieval misses that BM25 catches. Here is the reasoning, not the rules of thumb.

Most RAG systems ship with no real eval, which is why most RAG systems quietly degrade. The signal is decomposing evaluation into retrieval and generation, and measuring faithfulness separately from relevance. Here is the framework that makes RAG improvements measurable.

Enterprise RAG fails on isolation, not retrieval quality. The signal is enforcing tenant and document-level permissions server-side, at retrieval time, so the model can never surface data a user cannot see. Here is the design that survives a security review.

A 2025-2026 question that catches people clinging to dogma in either direction. The signal is a cost, latency, accuracy, and scale tradeoff, plus knowing the 'lost in the middle' failure of long context. Here is the framework for choosing.

Retrieval quality is capped by the query, and raw user queries are often bad for search. The signal is knowing the techniques that rewrite the query before retrieval and which failure each fixes.

Vector RAG quietly fails on two query shapes: multi-hop chains and global 'what are the themes' questions. The signal is knowing exactly what the graph buys you and the construction bill it charges.

Always retrieving is wasteful and sometimes harmful; never retrieving hallucinates. Self-RAG makes retrieval a decision the model controls, then critiques what it got back. The signal is the retrieve-on-demand plus self-critique loop.

Chunking quietly sets the ceiling on RAG quality, and 'split every 500 tokens' leaves recall on the table. The signal is knowing the strategy families and the parent-child trick that decouples retrieval size from context size.

Citations turn a RAG answer from 'trust me' into something verifiable, and they are a real engineering problem. The signal is grounding each claim to a specific passage and verifying the attribution, not appending a bag of links at the end.

Real corpora contradict themselves (old vs new policy, different teams), and naive RAG silently picks one or blends them into a wrong answer. The signal is detecting conflict and resolving it by recency, authority, and transparency.

Garbage parsing means garbage retrieval, and naive PDF text extraction destroys tables and reading order. The signal is fixing the parser, not the retriever. Here is the answer.

RAG and agents get expensive and slow fast: retrieval plus reranking plus big-model calls, multiplied across agent steps. The signal is naming the dominant cost first, then the levers that actually move it. Here is the playbook.

Single-shot retrieval fails on questions that chain facts ('who directed the highest-grossing film of 2019?'). The signal is decomposing or iterating retrieval instead of retrieving once, and guarding against errors compounding across hops. Here is the answer.

A RAG system is only as current as its index, and stale or duplicated content silently degrades answers. The signal is an incremental pipeline with deletes, updates, and recency signals, not a one-time bulk load. Here is the answer.

Forcing every query through the same pipeline burns latency, money, and precision. A router decides what each query actually needs. The signal is naming the four decisions a router makes and how you keep a misroute from sinking the answer.

A short question and the passage that answers it live in different neighborhoods of embedding space. HyDE closes that gap by embedding a fake answer instead of the question. The signal is knowing why a wrong hypothetical still retrieves the right docs.

Small chunks match precisely but starve the LLM of context; big chunks have context but match badly. Parent-child retrieval refuses to pick. The signal is decoupling what you match on from what you feed the model.

A chunk embedded in isolation forgets which document, section, and entity it came from, so it retrieves poorly. Contextual retrieval writes that context back in before indexing. The signal is fixing retrieval at index time, not query time.

Fixed-size chunking splits a coherent idea mid-thought; semantic chunking splits on meaning. The signal is naming the methods (structure-aware, embedding-similarity, LLM-based) and the tradeoffs that decide which one to ship.

Standard RAG retrieves once and generates; agentic RAG turns retrieval into an iterative, reasoning-driven loop. The signal is the agent deciding whether, what, and when to retrieve, then re-retrieving until the evidence holds.

RAG failures are usually retrieval failures, yet most teams only measure the final answer. The signal is scoring retrieval on its own (recall@k, precision@k, MRR/nDCG) to isolate exactly where the system breaks.

Real users type 'what about the new policy?' with no context. Retrieving on those four words returns noise. Here is how strong systems disambiguate before they ever hit the retriever.

A 90% eval score and angry users at the same time means your eval set doesn't look like reality. The fix is to make evaluation track production, not the other way around. Here is how.

A single embedding can't represent a question with three independent parts. The signal is knowing when to split a query, how to retrieve and recombine per sub-query, and the latency and drift costs that make decomposition a net loss on simple questions.

HyDE generates a fake answer to embed, which helps on vague queries but actively hurts on factual or out-of-domain ones. The signal is naming exactly when the hypothetical document misleads retrieval and which multi-draft and hybrid variants recover.

Vector search over rows is the wrong tool when the user asks for a count or an aggregate. The signal is retrieving the right schema, generating validated SQL, and knowing when to query the database instead of embedding it.

Splitting source files every 500 characters cuts functions in half and destroys retrieval. The signal is chunking on syntax, retrieving by symbol and dependency, and combining lexical exact-match with semantic search the way code search actually needs.

Re-embedding 10M documents nightly is wasteful when only 0.5% changed. The signal is upserting by stable id, handling deletes and tombstones in an ANN index, and compacting before fragmentation tanks recall and latency.

An LLM can cite a source that does not say what the answer claims. The signal is checking claim-by-claim entailment against the cited text, not just that a citation marker exists, and knowing what to do when grounding fails.

A better embedding model is useless if old and new vectors share an index, because their spaces are incompatible. The signal is the dual-index re-embed-then-cutover plan, the cost math, and how to prove the new model is actually better before you flip.

The most role-defining judgment call in Applied AI: a shiny benchmark number against the operational price of a system nobody can debug at 2am. Most candidates ship on the number and lose the loop there. Here is the answer that passes.

The coding round version of the RAG question every candidate can describe in prose. Embed, index, retrieve, generate, cite. What separates a pass from a fail is the branch you write for when retrieval comes back with nothing worth quoting.

Everyone can recite the chunking tradeoff. Far fewer can write the splitter, keep the offsets a citation needs, and answer the question that kills the fixed splitter: what happens to a fact that straddles a boundary.

Text-to-SQL is deceptively hard because correctness is binary and the failure mode is a confident wrong number. The signal is schema grounding, query validation, and a safety layer, not just 'prompt an LLM with the schema.' Here is the production design.

Customer support is the killer applied-AI use case, testing agents, RAG, tools, and the all-important human escalation. The signal is knowing when to resolve, when to act, and when to hand off, safely.

AI code review lives or dies on precision: a few noisy comments and the team mutes the bot forever. The signal is grounding in the diff plus repo context, ruthless false-positive control, and earning developer trust one accepted comment at a time.

Legal review is high-stakes and precision-critical: a missed clause or hallucinated citation has real consequences, and lawyers have been sanctioned for fabricated cites. The signal is grounding in the actual documents, citing exact passages, and lawyer-in-the-loop. Here is the design.

Enterprise search lives or dies on permissions, freshness, and connecting to thirty messy SaaS sources; retrieval quality is table stakes. Learn how to fan out across connectors, enforce per-user access at query time, and blend lexical with vector search for results people trust.

Auto-resolving support tickets means grounding answers in your knowledge base, safely calling real APIs (refunds, account changes), and knowing when to hand off to a human. Learn the agent loop, the guardrails, and how to measure resolution without breaking trust.

Offering RAG as a product means handling messy ingestion, multi-tenant isolation, and per-customer index freshness, all behind a simple API. Learn the ingestion and query planes, how to keep tenants isolated, and how to give customers eval and observability.

Direct injection is the user attacking the prompt. Indirect injection hides the attack inside a page, doc, or email the model reads. The signal is realizing retrieved and tool data is untrusted and can hijack the model. Here is the answer.

An enterprise RAG that retrieves across everyone's documents is a breach waiting to happen. The fix lives in the retrieval layer, not the prompt. Here is the design that survives a security review.