bert

A clean fundamentals question that catches people who only know GPT. The signal is matching the architecture to the task by its attention pattern, and explaining why the field consolidated on decoder-only even for non-generation tasks. Here is that answer.

BERT and GPT split on one decision made before a single weight is trained: what does each token get to see? That choice cascades into attention, use case, and why one family now dominates. Here is the answer.