Implement learning-rate schedulers from scratch: step decay, exponential decay, and cosine annealing.

A from-scratch test of how learning-rate schedules are actually computed. The signal is correct closed-form formulas, knowing when each schedule shines, and why decaying the rate helps convergence. Here is the implementation of the common ones.

Updated Aug 2026 · Grounded in real Applied AI Engineer interview loops and written to a senior-engineer editorial bar.