knowledge distillation

Distillation buys you most of a frontier model's quality at a fraction of the serving cost. The signal is naming the three LLM-specific variants and knowing which one works against a closed API. Here is the answer.

Distillation is not one method. The signal is knowing when to match logits versus train on generated text, why on-policy distillation beats off-policy, and how reasoning models are distilled.