Implement the AdamW optimizer from scratch and explain how decoupled weight decay differs from Adam.

A from-scratch test of modern optimizers. The signal is the moment estimates, bias correction, and the one change that matters: AdamW decouples weight decay from the gradient instead of folding it into the loss. Here is the implementation and why it wins.

Updated Aug 2026 · Grounded in real Applied AI Engineer interview loops and written to a senior-engineer editorial bar.