learning rate

Almost every modern training run uses warmup-then-cosine, and getting it wrong destabilizes early training or wastes the tail. It's a closed-form function of the step. Here is the implementation and the why.

The learning rate is the one hyperparameter that can NaN your run on step 50 or stall it forever. Modern training schedules it instead of fixing it. The signal is knowing exactly what warmup and decay each buy you. Here is the answer.

'The model won't train' has a short list of usual suspects, each with a distinctive loss-curve signature. The shape of the curve names the bug before you touch a single hyperparameter. Here is how to read it.

Batch size is a training knob with non-obvious effects on speed, memory, and generalization. The signal is the large-batch tradeoffs and the learning-rate-scaling and gradient-accumulation tricks that make scaling actually work.