ml coding

A staple ML-coding screen. Writing the loop is easy; the signal is k-means++ init, a real convergence criterion, and knowing the failure modes. Here is the tight implementation plus the follow-ups interviewers always ask.

A deceptively simple ML-coding ask. Anyone can write exp/sum; the signal is the max-subtraction trick and the log-sum-exp form that keep it from overflowing. Here is the stable implementation and why the naive one breaks.

The from-scratch implementation that frontier labs and NVIDIA actually ask. Writing it proves you understand shapes, the scale, the causal mask, and how heads split and recombine. Here is a correct, readable implementation and the follow-ups.

Beam search is the decoding algorithm everyone references and few can implement correctly. The signal is keeping k hypotheses by cumulative log-prob, summing logs (not multiplying probs), and handling completed sequences and length. Here is a correct implementation and the tradeoffs.

A from-scratch ML-coding staple that checks whether you actually know the math you use. The signal is the clean gradient (it simplifies to Xᵀ(ŷ - y)/n), numerical stability, and vectorization. Here is the implementation and the details interviewers push on.

The core operation under every embedding/RAG retrieval, asked as a coding exercise. The signal is vectorizing the similarity, normalizing correctly, and using a partial selection (argpartition) instead of a full sort. Here is the efficient implementation and the scaling follow-up.

A common ML-coding check that verifies you actually understand the metrics you quote. The signal is getting TP/FP/FN/TN right and knowing macro vs micro averaging for multiclass, not reaching for sklearn.

Data loading is where training pipelines silently bottleneck. The signal is a correct shuffle-then-batch iterator plus knowing why prefetching and parallel loading keep the GPU fed.

BPE is the tokenizer behind GPT-style models, and implementing it proves you understand how subword vocabularies are built, not just that they exist. The signal is the iterative merge of the most frequent adjacent pair.

k-NN is simple to write and a good test of vectorization and the lazy-learner tradeoff. The signal is a clean vectorized distance computation and knowing why naive prediction is O(n) and how to speed it. Here is the implementation.

Implementing conv2d proves you understand what a CNN layer actually computes, not just that you can name it. The signal is correct output-shape math, clean stride/padding handling, and knowing the im2col trick frameworks really use.

LayerNorm sits in every transformer, and implementing it checks that you know what it normalizes over and why. The signal is normalizing across the feature dimension per token, the learnable scale/shift, and the RMSNorm simplification modern LLMs adopted.

Implementing the KV cache proves you understand why decode is efficient: you append the new token's K/V and reuse the rest instead of recomputing. The signal is the append-and-attend-over-cache logic. Here is the implementation.

Implementing BatchNorm checks that you know the train/inference difference, the part everyone forgets. The signal is normalizing over the batch with running stats maintained for inference. Here is the implementation.

TF-IDF is the classic text-vectorization scheme and a fast check of whether you actually understand term-frequency times inverse-document-frequency, or just recite it. The signal is explaining why IDF kills common words. Here is the implementation.

Building a PR/ROC curve from scratch proves you understand thresholds and the precision/recall tradeoff. The real signal is sweeping the threshold in one sorted pass and integrating the area. Here is the implementation.

Implementing Adam proves you understand what's happening under .step(): momentum plus per-parameter adaptive rates with bias correction. The signal is the two moments and the bias-correction step.

Single-head attention is common; multi-head adds the split-into-heads, attend-per-head, concatenate structure. The signal is the reshape into heads and a clear reason why multiple heads help. Here is the implementation.

Dropout is five lines, yet most candidates botch two of them: disabling it at inference and scaling the survivors so the expected activation stays constant. Inverted dropout is the convention they want to see.

A from-scratch classic that tests recursion plus the split criterion math. The signal is computing impurity correctly, choosing the best threshold by information gain, and knowing the stopping rules. Here is a clean recursive implementation.

A from-scratch staple that checks whether you understand the conditional-independence assumption and why you work in log space. The signal is fitting per-class means and variances, summing log-likelihoods, and adding the log prior. Here is the implementation.

A from-scratch favorite that tests linear algebra fluency. The signal is centering first, using SVD instead of forming the covariance matrix, and reading variance off the singular values. Here is the implementation and the details interviewers push on.

A from-scratch test of the EM algorithm and soft clustering. The signal is the two alternating steps (responsibilities then weighted re-estimation), log-sum-exp for stability, and knowing how GMM generalizes k-means. Here is the implementation.

A from-scratch test of the least-squares closed form and the numerical traps in it. The signal is deriving the normal equations, knowing why you never explicitly invert, and reaching for lstsq or the SVD instead. Here is the implementation.

A from-scratch test of recurrent forward passes and backprop through time. The signal is the shared-weight recurrence, accumulating gradients across timesteps, and explaining the vanishing-gradient problem. Here is the implementation.

A from-scratch test of gated recurrence. The signal is wiring the forget/input/output gates and candidate correctly, separating cell state from hidden state, and explaining why the additive cell path fixes vanishing gradients. Here is the implementation.

A from-scratch test of modern optimizers. The signal is the moment estimates, bias correction, and the one change that matters: AdamW decouples weight decay from the gradient instead of folding it into the loss. Here is the implementation and why it wins.

A from-scratch test of how learning-rate schedules are actually computed. The signal is correct closed-form formulas, knowing when each schedule shines, and why decaying the rate helps convergence. Here is the implementation of the common ones.

A from-scratch test of how PyTorch actually works under the hood. The signal is building a computation graph during the forward pass, local derivatives per op, and a topological-order backward pass that accumulates gradients. Here is a minimal engine.

A from-scratch test of subword tokenization. The signal is greedy longest-match encoding against a fixed vocabulary, the continuation-prefix convention, and how WordPiece differs from BPE. Here is the implementation.

A from-scratch test of distributed input pipelines. The signal is partitioning data across workers with no overlap and no gaps, epoch-consistent shuffling with a shared seed, and handling the uneven-last-batch problem. Here is the implementation.

The classic recursive-descent exercise with an applied-AI twist: the JSON your model streams is truncated mid-token for the entire generation. The signal is a clean strict parser plus a small repair layer, not a second parser. Here is the implementation.