optimization

DP feels like pattern-matching magic until you have a method. The signal is a repeatable drill: spot overlapping subproblems and optimal substructure, define the state and recurrence, then memoize or tabulate. Here is that method on a worked example.

Many state-of-the-art training runs keep a shadow copy of the weights that's a moving average of the trajectory, and serve that instead of the final step. It's a few lines and a real accuracy boost. Here it is.

Almost every modern training run uses warmup-then-cosine, and getting it wrong destabilizes early training or wastes the tail. It's a closed-form function of the step. Here is the implementation and the why.

Binary search on the answer solves minimize-the-maximum and capacity problems by guessing the result and checking feasibility in a monotone predicate. The signal is spotting monotonicity and writing a clean feasibility function. Here is the answer.

A from-scratch test of how learning-rate schedules are actually computed. The signal is correct closed-form formulas, knowing when each schedule shines, and why decaying the rate helps convergence. Here is the implementation of the common ones.

Optimizer questions test whether you understand what each one adapts and the subtle AdamW fix that the whole field now uses. The signal is the per-parameter adaptivity story plus why coupling weight decay to Adam was a bug. Here is that answer.

Bandits optimize while they learn, shifting traffic toward winners mid-experiment instead of fixing it like an A/B test. The signal is the explore-exploit tradeoff, the three core algorithms, and knowing exactly when a bandit beats a clean A/B test.

The learning rate is the one hyperparameter that can NaN your run on step 50 or stall it forever. Modern training schedules it instead of fixing it. The signal is knowing exactly what warmup and decay each buy you. Here is the answer.

Everyone says 'gradient descent,' but the batch-size choice and momentum are the real interview content. The signal is the noise-vs-cost tradeoff across batch/SGD/mini-batch and why momentum earns its keep.

The loss defines what the model optimizes, and picking the wrong one quietly dooms it. The signal is matching the loss to the task and data (outliers, imbalance), not defaulting to MSE or cross-entropy on reflex.

'The model won't train' has a short list of usual suspects, each with a distinctive loss-curve signature. The shape of the curve names the bug before you touch a single hyperparameter. Here is how to read it.

The idea that models, like students, learn better from easy examples first is intuitive, and sometimes it works and sometimes it does nothing. Knowing when it pays off is the real signal. Here is the honest answer.

Everyone says 'add an index,' but a strong engineer reads the plan first. The signal is finding the dominant operator, comparing estimated vs actual rows, and recognizing the seq-scan, bad-join, and spill patterns.

Dynamic pricing blends demand forecasting, optimization, and real-time serving, with fairness and trust constraints. The signal is the predict-then-optimize structure plus guardrails against perverse outcomes. Here is the design.

The trap is jumping straight to 'add more GPUs.' The signal is profiling first, then applying the cheap, no-retrain levers in the right order. Here is the diagnosis-then-optimize playbook for p99.

OOM is the most common wall in deep learning, and 'buy a bigger GPU' is the weakest answer. The signal is naming the memory consumers, knowing which one dominates, and matching the right lever to it.