Why is an LLM's context window limited, and how do models extend it?

Context windows are not arbitrary numbers; they fall out of attention's quadratic cost, KV-cache memory, and how the model was trained. The signal is naming all three causes and matching each to the technique that extends it.

Updated Aug 2026 · Grounded in real Applied AI Engineer interview loops and written to a senior-engineer editorial bar.