cross entropy

Perplexity is the classic LM metric. The signal is knowing it is just exponentiated cross-entropy, what it intuitively measures, and why a low perplexity does not mean a good product.

A deceptively simple ML-coding ask. Anyone can write exp/sum; the signal is the max-subtraction trick and the log-sum-exp form that keep it from overflowing. Here is the stable implementation and why the naive one breaks.

A from-scratch ML-coding staple that checks whether you actually know the math you use. The signal is the clean gradient (it simplifies to Xᵀ(ŷ - y)/n), numerical stability, and vectorization. Here is the implementation and the details interviewers push on.

The loss defines what the model optimizes, and picking the wrong one quietly dooms it. The signal is matching the loss to the task and data (outliers, imbalance), not defaulting to MSE or cross-entropy on reflex.

These four quantities underlie cross-entropy loss, decision-tree splits, distillation, and the KL penalty in RLHF. The signal is deriving them from one another and pointing to exactly where each shows up in a real training loop.

Cross-entropy is the start, not the answer. The signal is matching the loss to the task structure: focal for detection's background flood, IoU-family for box overlap, Dice for imbalanced masks. Here is how to reason about it.