momentum

Implementing Adam proves you understand what's happening under .step(): momentum plus per-parameter adaptive rates with bias correction. The signal is the two moments and the bias-correction step.

Everyone says 'gradient descent,' but the batch-size choice and momentum are the real interview content. The signal is the noise-vs-cost tradeoff across batch/SGD/mini-batch and why momentum earns its keep.