How do you compress the KV cache at inference, and what does each method trade off?

At long context the KV cache, not the weights, fills the GPU. The signal is naming the levers (quantization, token eviction, head sharing, low-rank) and what each one costs in quality.

Updated Aug 2026 · Grounded in real Applied AI Engineer interview loops and written to a senior-engineer editorial bar.