Core

Retrieval vs Long Context

When you can fit a whole document in a model's large context window, should you, or should you retrieve only the relevant chunks? Long context is simpler but expensive (quadratic attention), slower, and unevenly used (lost in the middle); retrieval is cheaper, faster, updates without retraining, and surfaces only what matters. The answer is usually retrieval for large, changing, or partially-relevant corpora, and long context for small, cohesive inputs. Applied-AI interviews probe it because 'just use the big context window' is a common, costly oversimplification.

a free account unlocks the core curriculum tier · no card

RELATED CONCEPTS

The Context Window The RAG Pipeline Vector Search and ANN Indexes LLM Cost Optimization

PRACTICE THIS IN REAL QUESTIONS

RAG & Agent System DesignContext windows are now huge. When do you just stuff everything in context instead of building RAG?→RAG & Agent System DesignHow do you choose top-k and the context budget for RAG, given recall, noise, and cost all pull against each other?→RAG & Agent System DesignDesign a production RAG system over 10M documents serving ~1,000 QPS at sub-second latency.→System Design for AI in ProductionDesign a text-to-SQL feature: let users ask questions in natural language over a real database.→RAG & Agent System DesignWhat is query transformation in RAG (HyDE, decomposition, step-back), and when does each help?→RAG & Agent System DesignWhat is GraphRAG, and when does it beat traditional vector RAG?→

COMPANIES THAT ASSUME THIS

OpenAI Anthropic Google Microsoft Scale AI Cognition