retrieval

RAG and fine-tuning get framed as either/or. RAFT trains the model to be good at RAG itself. The tell is what goes in the training set: relevant docs mixed with distractors. Here is the answer.

A model advertising a million-token window may not use it. The signal is knowing needle-in-a-haystack, where it breaks, and how to test real multi-fact reasoning across the full context. Here is the answer.

One trained embedding you can truncate to any length and still use. The signal is the nested-prefix training objective and the coarse-to-fine retrieval win it unlocks at scale. Here is the answer.

Modern embedding models want a task instruction prepended to the text, and getting the prefix wrong quietly wrecks retrieval. The signal is why instructions help and the asymmetric query/passage convention.

Every team ships English first, then finds that the retriever, the prompt, the latency budget and the cost model each degrade differently in the next five languages. The applied playbook looks nothing like the model-level answer, and the thing that sinks launches is the eval set nobody built.

The modal Applied AI design round. Anyone can draw embed-retrieve-generate. The signal is in chunking, hybrid retrieval, the rerank/latency tradeoff, and how you prove it works. Here is the structure that scores.

A 2025-2026 question that catches people clinging to dogma in either direction. The signal is a cost, latency, accuracy, and scale tradeoff, plus knowing the 'lost in the middle' failure of long context. Here is the framework for choosing.

Conversations and agent tasks outgrow the context window, and naive 'stuff the whole history' fails on cost, latency, and lost-in-the-middle. The signal is a tiered memory design: recent buffer, summarized mid-term, retrieved long-term.

Retrieval quality is capped by the query, and raw user queries are often bad for search. The signal is knowing the techniques that rewrite the query before retrieval and which failure each fixes.

Vector RAG quietly fails on two query shapes: multi-hop chains and global 'what are the themes' questions. The signal is knowing exactly what the graph buys you and the construction bill it charges.

Always retrieving is wasteful and sometimes harmful; never retrieving hallucinates. Self-RAG makes retrieval a decision the model controls, then critiques what it got back. The signal is the retrieve-on-demand plus self-critique loop.

Chunking quietly sets the ceiling on RAG quality, and 'split every 500 tokens' leaves recall on the table. The signal is knowing the strategy families and the parent-child trick that decouples retrieval size from context size.

Context engineering is the discipline that replaced 'prompt engineering' for serious agent work. The signal is understanding that what you put in the context window (and what you leave out) drives behavior more than clever wording. Here is the answer.

The embedding model is the foundation of retrieval, and swapping it is deceptively dangerous. The signal is choosing on domain-relevant retrieval quality (not leaderboard) and knowing that a new model means re-embedding everything. Here is the answer.

Real documents carry images, charts, and tables, not just text, and text-only RAG silently drops them. The signal is knowing the two retrieval approaches and why the generator must change too. Here is the answer.

Reranking is the cheapest large win in RAG quality: retrieve broadly, then reorder precisely. The signal is the bi-encoder-versus-cross-encoder distinction and the retrieve-then-rerank two-stage design. Here is the answer.

Pure vector search misses exact terms; pure keyword search misses meaning. The signal is knowing why each fails alone and how to merge two rankings on incomparable scales without hand-tuning weights. Here is the answer.

Late interaction gets cross-encoder-like quality with near-bi-encoder scalability. The signal is the per-token embeddings plus MaxSim matching that buys you the middle ground, and knowing what it costs. Here is the answer.

Forcing every query through the same pipeline burns latency, money, and precision. A router decides what each query actually needs. The signal is naming the four decisions a router makes and how you keep a misroute from sinking the answer.

A short question and the passage that answers it live in different neighborhoods of embedding space. HyDE closes that gap by embedding a fake answer instead of the question. The signal is knowing why a wrong hypothetical still retrieves the right docs.

Small chunks match precisely but starve the LLM of context; big chunks have context but match badly. Parent-child retrieval refuses to pick. The signal is decoupling what you match on from what you feed the model.

Fixed-size chunking splits a coherent idea mid-thought; semantic chunking splits on meaning. The signal is naming the methods (structure-aware, embedding-similarity, LLM-based) and the tradeoffs that decide which one to ship.

Late chunking flips the usual order of operations to fix the lost-context problem without a single extra LLM call. The whole signal is which step happens first. Here is the answer.

A model that tops MTEB can still flop on your jargon-heavy corpus, and most teams jump straight to the expensive fix. There is a ladder, and the cheap rungs are the ones people skip. Here is the order to climb it.

Top-k by pure similarity often hands the model five paraphrases of the same paragraph, starving it of the other facts the question needs. The fix is ranking for relevance and novelty at once. Here is how.

Real users type 'what about the new policy?' with no context. Retrieving on those four words returns noise. Here is how strong systems disambiguate before they ever hit the retriever.

When the right answer isn't even in the top-50, the generator can't save you. Low recall has a short list of usual suspects. Here is the order to check them so you fix the cause, not a symptom.

More chunks means higher recall but more noise, more cost, and more lost-in-the-middle. The right k is an empirical tradeoff, not a default of 5. Here is how to find it.

A bridge between classic DSA and modern RAG. The signal is building a working inverted index and a correct BM25 score by hand, reasoning about its complexity, and then knowing exactly when you would reach for embeddings and an ANN index instead.

Late interaction stores one vector per token, so a corpus that fit in a few GB as single vectors can balloon 100x. The signal is knowing the compression and indexing tricks (centroids, residuals, PLAID) that make multi-vector retrieval shippable.

A single embedding can't represent a question with three independent parts. The signal is knowing when to split a query, how to retrieve and recombine per sub-query, and the latency and drift costs that make decomposition a net loss on simple questions.

HyDE generates a fake answer to embed, which helps on vague queries but actively hurts on factual or out-of-domain ones. The signal is naming exactly when the hypothetical document misleads retrieval and which multi-draft and hybrid variants recover.

Splitting source files every 500 characters cuts functions in half and destroys retrieval. The signal is chunking on syntax, retrieving by symbol and dependency, and combining lexical exact-match with semantic search the way code search actually needs.

Hybrid retrieval only beats either method if the fusion is tuned. The signal is knowing why you cannot just add BM25 and cosine scores, how RRF's k constant behaves, and when a learned weight beats rank fusion.

An off-the-shelf cross-encoder is general; your domain has jargon and relevance rules it never saw. The signal is knowing when fine-tuning pays off, how to mine hard negatives, and how to avoid training a reranker that just memorizes your retriever's mistakes.

A better embedding model is useless if old and new vectors share an index, because their spaces are incompatible. The signal is the dual-index re-embed-then-cutover plan, the cost math, and how to prove the new model is actually better before you flip.

A context window is not memory. Real agents need a storage architecture: working buffer, episodic recall, and consolidated facts. The signal is the read/write paths and how you keep memory from rotting.

A 0.87 cosine score on a chunk that answers nothing is not a bug in your vector database. It is the embedding model telling you exactly what it was trained to tell you. Here is how to read that signal instead of thresholding it away.

The coding round version of the RAG question every candidate can describe in prose. Embed, index, retrieve, generate, cite. What separates a pass from a fail is the branch you write for when retrieval comes back with nothing worth quoting.

Everyone can recite the chunking tradeoff. Far fewer can write the splitter, keep the offsets a citation needs, and answer the question that kills the fixed splitter: what happens to a fact that straddles a boundary.

The modal ML system design round. The structure interviewers reward is the funnel: candidate generation then ranking then re-ranking, with the right model at each stage and an honest plan for cold start, freshness, and feedback loops. Here is that structure.

Multimodal search tests whether you understand shared embedding spaces and the retrieve-then-rank pattern under a real catalog's scale and freshness. The signal is CLIP-style joint embeddings plus hybrid retrieval and honest relevance evaluation. Here is the design.

Cross-session memory is what makes an assistant feel personal, and it is mostly a retrieval and state-management problem, not a bigger context window. The signal is the tiered architecture plus what to store, forget, and protect.

How large recommenders and search pull candidates from millions of items in milliseconds. The signal is why the user and item towers are separate, how that unlocks precomputed embeddings plus an ANN index, and where ranking takes over.

The service behind every RAG stack and semantic search box. The signal is your ANN index choice and how you handle the three things that quietly break it: metadata filtering, live updates, and scale.

Personalizing an LLM is a retrieval and memory problem, not a per-user fine-tune. Learn how to assemble user context at request time, manage long-term memory without bloating the prompt, and respect privacy and the right to be forgotten.

Offering RAG as a product means handling messy ingestion, multi-tenant isolation, and per-customer index freshness, all behind a simple API. Learn the ingestion and query planes, how to keep tenants isolated, and how to give customers eval and observability.