drift detection

Models fail silently; the question is whether you would know. The signal is monitoring the right layers (operational, data, prediction, outcome) and alerting on drift without drowning in false pages. Here is the system and the metrics that matter.

The naive answer is 'retrain weekly.' The senior answer is a hybrid: drift-triggered retraining with a max-staleness fallback, plus a gate that a fresh model must beat the incumbent. Here is how to reason about it.

Models decay silently as the world shifts, and 'monitor for drift' is too vague. The signal is the actual statistical methods and distinguishing data drift you can detect without labels from concept drift you often cannot.

Four drift tests, four different assumptions. The weak answer lists them; the strong one knows which handles high-dimensional embeddings, which needs binning, and which gives a calibrated p-value. Here is how to choose.

Three drifts get casually lumped together, but they have different detectability and different fixes. The interview signal is knowing which you can catch without labels and which one genuinely demands a new model.

You cannot wait for ground truth to tell you the model broke. Prediction-side monitoring catches failures in minutes, not weeks, if you know which output signals actually move first.