TL;DR: Model the expected value of the metric (including seasonality such as weekday vs weekend and time-of-day) and flag significant deviations from it. Use largely unsupervised methods (labels are rare): seasonal decomposition or forecasting (Prophet-style, SARIMA, or learned) for the baseline, and statistical or ML detectors on the residual. Handle cold start with population and peer baselines, tune thresholds to control false-positive rate, and add severity plus grouping so you alert on real incidents without alert fatigue.
How to approach it
Clarify what anomalous means here: a spike, a drop, a level shift, or a trend break. Pin down the cost of false positives vs misses (for billing, a missed overspend is costly, but daily false alarms get the alert muted). Then design around the two hard parts, seasonality (the baseline is not constant) and the scarcity of labels (mostly unsupervised), plus alerting hygiene. The interviewer wants to see you treat alert fatigue as a first-class design constraint, not an afterthought.
A strong answer
Frame as expected vs observed. An anomaly is a significant deviation from what the metric should be, so the core is a good model of the expected value, and the hard part is that expected is not a flat line.
- Seasonality and trend. Billing, traffic, and most metrics have weekly and daily cycles (low on weekends, peaks at business hours) and trends (growth). A naive static threshold fires every Monday morning. So decompose the series into trend, seasonality, and residual (STL, SARIMA, Prophet-style additive models, or a learned forecaster), predict the expected value with its uncertainty band, and flag points whose residual is large relative to normal variation. This handles the seasonality the question is testing.
- Unsupervised by necessity. You rarely have labeled anomalies, so lean on unsupervised and statistical methods: deviation beyond a dynamic band (forecast plus or minus k sigma), robust z-scores on residuals, or models like isolation forest for multivariate cases. Move to supervised classification only once you accumulate labeled incidents.
- Detection types. Point anomalies (a single spike), contextual (a normal value in the wrong context, e.g. high at 3am), and change-points or level shifts (a sustained jump, common in billing when a new resource turns on). Cover the ones that matter; level-shift detection is key for billing.
Cold start. A brand-new account or metric has no history to learn seasonality from. Bootstrap with a peer or population baseline (similar accounts or services), use broad priors initially, and tighten the per-entity model as data accrues. Do not silence detection for new entities; use the cohort.
Alerting hygiene (where these systems fail in practice). Tune the threshold to a target false-positive rate, require sustained deviation rather than a single noisy point, assign severity (magnitude times confidence), group related anomalies into one incident (a billing spike across services is one alert, not fifty), and provide context (expected vs actual, which dimension drove it). Otherwise users mute the system and miss the real one.
Operations. Stream metrics in, score against the periodically retrained baseline, persist anomalies, and route by severity. Capture user feedback (was this a real issue?) as labels to improve over time.
The defensible framing: model expected value with seasonality and uncertainty, detect on residuals (unsupervised, since labels are scarce), bootstrap cold start from peers, and engineer alerting (severity, grouping, FP control) so it surfaces real incidents without fatigue.
Key takeaways
- The hard part is a moving baseline: decompose trend and seasonality, then detect on the residual, never on a static threshold.
- Labels are scarce, so default to unsupervised detection on dynamic bands and graduate to supervised only as incidents accumulate.
- Cold start is solved by peer and population baselines, not by leaving new entities unmonitored.
- Alert fatigue is a design constraint: FP-rate tuning, sustained-deviation gates, severity, and grouping are what keep the system trusted.
What interviewers probe next
- "How do you handle weekly and daily seasonality?" Seasonal decomposition or a forecaster that models the cycles, then detect on the residual; never a static threshold.
- "Labels are scarce, supervised or unsupervised?" Mostly unsupervised and statistical; move toward supervised only as labeled incidents accumulate, and use feedback as labels.
- "Cold start for a new metric?" Peer or population baseline and broad priors, refined as history grows.
- "How do you avoid alert fatigue?" FP-rate-tuned thresholds, sustained-deviation requirements, severity scoring, and grouping correlated anomalies into one incident.
Common mistakes
- Static thresholds that ignore seasonality and fire on every normal weekly cycle.
- Assuming labeled anomalies exist, failing to design for unsupervised detection.
- Ignoring cold start, so new accounts or metrics are unmonitored or noisy.
- No alert hygiene (severity, grouping, FP control), so users mute the system and miss real incidents.