AppliedAIPrep logoAppliedAI/Prep
← LLM & GenAI Fundamentals / 87
expertDeepSeekNVIDIAMistral

What is Multi-head Latent Attention (MLA), and how does it differ from MQA and GQA?

MQA and GQA shrink the KV cache by sharing key/value heads. MLA takes a different route: compress K and V into a low-rank latent and cache that. The signal is knowing it is a cache trick, not a head-sharing trick, and why it keeps quality.

Updated Aug 2026 · Grounded in real Applied AI Engineer interview loops and written to a senior-engineer editorial bar.

MQA and GQA shrink the KV cache by sharing key/value heads. MLA takes a different route: compress K and V into a low-rank latent and cache that. The signal is knowing it is a cache trick, not a head-sharing trick, and why it keeps quality.

Unlock the other 750 answers · ₹2,000 / $25Your progress and mastery stay saved · 6 months · one payment · no auto-renew
UP NEXT ON YOUR JOURNEY
Next in this trackWhat is an attention sink, and how does StreamingLLM use it for endless generation?Next in this trackHow do RoPE scaling methods (position interpolation, NTK, YaRN) extend a model's context window?Next in this trackHow do you compress the KV cache at inference, and what does each method trade off?
DISCUSSION · 0

No comments yet — be the first to share your approach.