sliding window

Full attention is O(n^2), so long context needs cheaper attention. The signal is naming the families (sparse, sliding-window, linear), what each gives up, and why FlashAttention is not one of them.

Drop the oldest tokens to bound the KV cache and the model collapses. Keep just the first few tokens and it generates indefinitely. The signal is understanding why those first tokens act as an attention sink.

Sliding window is one of the highest-yield coding patterns, and this is its canonical problem. The real signal is recognizing when a window collapses an O(n squared) scan into one linear pass, and maintaining the invariant cleanly.

Minimum Window Substring is the hard sliding-window problem that tests expand-and-contract with a character-count map. The signal is the grow-to-valid then shrink-to-minimal pattern, checked in O(1) per step.

Finding the max of every window of size k naively is O(n*k). The trick that drops it to O(n) is a monotonic deque of indices, and the reason it stays linear surprises most candidates.