lost in the middle

A model advertising a million-token window may not use it. The signal is knowing needle-in-a-haystack, where it breaks, and how to test real multi-fact reasoning across the full context. Here is the answer.

A 2025-2026 question that catches people clinging to dogma in either direction. The signal is a cost, latency, accuracy, and scale tradeoff, plus knowing the 'lost in the middle' failure of long context. Here is the framework for choosing.

More chunks means higher recall but more noise, more cost, and more lost-in-the-middle. The right k is an empirical tradeoff, not a default of 5. Here is how to find it.