context window
Applied AI interview questions tagged context window, across every topic.
6 questions · 1 unlocked for you
Concepts behind "context window"
The curriculum that explains the ideas these questions test.
Foundational
TokenizationModels do not read characters or words; they read tokens, subword chunks produced by an algorithm like BPE that maps text to integer IDs. Tokenization decides how many tokens a piece of text costs (driving price, latency, and context usage), why models miscount letters or fumble rare words, and why non-English text is more expensive. Applied-AI interviews probe it because token accounting is the first thing that bites a production LLM bill.🧠 Foundations of LLMs & GenAI
Foundational
The Context WindowThe context window is the maximum number of tokens a model can attend to at once, prompt plus generation. It is bounded by attention's quadratic cost, the KV cache's linear memory growth, and the length the model was trained on. A bigger window is not free or uniformly useful (models lose information in the middle), which is why retrieval often beats stuffing everything into context. Applied-AI interviews probe it because it shapes cost, latency, and the RAG-vs-long-context decision.🧠 Foundations of LLMs & GenAI
Core
Positional Encodings (RoPE and ALiBi)Attention is order-blind, so models inject token position separately. Modern LLMs use relative schemes: RoPE rotates query/key vectors by an angle proportional to position so the attention score depends only on the offset between tokens, and ALiBi adds a distance penalty to attention scores. Both extrapolate to longer sequences far better than learned absolute positions, which is why RoPE-with-scaling is how context windows get extended. Applied-AI interviews probe it because it explains how long-context models are built.🧠 Foundations of LLMs & GenAISign in
Core
Retrieval vs Long ContextWhen you can fit a whole document in a model's large context window, should you, or should you retrieve only the relevant chunks? Long context is simpler but expensive (quadratic attention), slower, and unevenly used (lost in the middle); retrieval is cheaper, faster, updates without retraining, and surfaces only what matters. The answer is usually retrieval for large, changing, or partially-relevant corpora, and long context for small, cohesive inputs. Applied-AI interviews probe it because 'just use the big context window' is a common, costly oversimplification.🤖 Retrieval & AgentsSign in