AppliedAIPrep logoAppliedAI/Prep
← LLM & GenAI Fundamentals / 41
hardGoogleMetaMistral

What are Multi-Query (MQA) and Grouped-Query Attention (GQA), and why do they exist?

MQA and GQA shrink the KV cache, the thing that bottlenecks LLM serving. The signal is knowing they share key/value heads across query heads to cut memory and bandwidth, with GQA as the quality-preserving middle ground. Here is the answer.

Updated Aug 2026 · Grounded in real Applied AI Engineer interview loops and written to a senior-engineer editorial bar.

MQA and GQA shrink the KV cache, the thing that bottlenecks LLM serving. The signal is knowing they share key/value heads across query heads to cut memory and bandwidth, with GQA as the quality-preserving middle ground. Here is the answer.

Unlock the other 750 answers · ₹2,000 / $25Your progress and mastery stay saved · 6 months · one payment · no auto-renew
UP NEXT ON YOUR JOURNEY
Next in this trackBeyond DPO: what are SimPO, KTO, and ORPO, and why do these alignment variants exist?Next in this trackWhat are neural scaling laws and the Chinchilla compute-optimal result?Next in this trackCompare LLM decoding strategies: greedy, beam search, temperature, top-k, top-p, and repetition penalties.
DISCUSSION · 0

No comments yet — be the first to share your approach.