TL;DR: Encoder-only (BERT) uses bidirectional attention, best for understanding tasks (classification, retrieval embeddings). Decoder-only (GPT) uses causal (left-to-right) attention, best for generation. Encoder-decoder (T5) pairs a bidirectional encoder with a causal decoder, natural for sequence-to-sequence (translation, summarization). Decoder-only now dominates because a single causal-LM objective scales, generalizes across tasks via prompting, and serves efficiently with a KV cache.
How to approach it. Anchor the distinction in the attention pattern (bidirectional vs causal), then map each to the task type it suits, then explain the consolidation onto decoder-only as a scaling/simplicity story, not a quality-on-every-task story.
A strong answer. The three families differ in what each token can attend to:
- Encoder-only (BERT-style): every token attends to all tokens (bidirectional), so it builds rich representations using both left and right context. Trained with masked language modeling. Best for understanding: classification, NER, and producing embeddings for retrieval/similarity. It cannot generate autoregressively well because it was not trained to predict the next token.
- Decoder-only (GPT-style): each token attends only to itself and earlier tokens (causal mask), trained to predict the next token. Built for generation, and the causal structure is what makes a KV cache possible (past tokens' keys/values are fixed and reusable).
- Encoder-decoder (T5, original Transformer): a bidirectional encoder reads the full input, a causal decoder generates the output while cross-attending to the encoder. Natural for sequence-to-sequence where input and output differ (translation, summarization).
| Family | Attention | Objective | Sweet spot | Example |
|---|---|---|---|---|
| Encoder-only | Bidirectional | Masked LM | Understanding, embeddings | BERT |
| Decoder-only | Causal | Next-token | Generation, generalist via prompting | GPT |
| Encoder-decoder | Encoder bidir + decoder causal + cross-attn | Span/seq2seq | Translation, summarization | T5 |
Why decoder-only won. A single next-token objective is simple and scales cleanly with data and parameters; in-context learning lets one model do classification, extraction, translation, and Q&A by prompting, collapsing the need for task-specific architectures; and causal attention serves efficiently via KV caching. So even tasks an encoder would theoretically suit (classification) are routinely done by prompting a large decoder-only model, because the generalist scales and is operationally simpler. Encoder-only models persist where you specifically need cheap, high-quality embeddings; encoder-decoder persists in some seq2seq niches.
The defensible nuance: decoder-only is not architecturally better at understanding token-for-token (bidirectional context is genuinely richer); it won on scaling, generality, and serving economics.
Key takeaways
- The attention pattern is the real distinction: bidirectional for understanding, causal for generation, both stitched together for seq2seq.
- Decoder-only dominance is a scaling/serving story, not a per-task quality story; bidirectional context is genuinely richer token-for-token.
- Causal attention is precisely what enables the KV cache: earlier tokens' keys/values never change, so you reuse instead of recompute.
- Encoder-only still earns its place for cheap, high-quality embeddings; encoder-decoder still fits genuine seq2seq tasks.
What interviewers probe next.
- "Why can't BERT generate text?" It is trained on masked prediction with bidirectional attention, not next-token prediction; it has no causal generation objective.
- "If bidirectional context is richer, why not use it for generation?" Generation is inherently left-to-right at inference; you cannot attend to tokens you have not produced yet, so causal attention matches the task.
- "Where do embeddings come from in a decoder-only world?" Either dedicated encoder-style embedding models or pooled hidden states / specialized fine-tunes of decoder models.
- "What enables the KV cache?" Causal attention: earlier tokens' keys/values never change, so you cache and reuse them instead of recomputing each step.
Common mistakes.
- Assuming "transformer" means GPT and forgetting bidirectional encoders exist.
- Claiming decoder-only is strictly better at everything, rather than winning on scaling/generality/serving.
- Confusing the training objective (MLM vs next-token) with the attention pattern.
- Saying encoder-decoder is obsolete; it still fits genuine seq2seq tasks.