118How do you build anomaly detection for a streaming time series, and how do you handle seasonality and concept drift?▼hardNetflixUberMicrosoft2 replies◆ premiumThreshold alerts fire all weekend and miss the real outage Monday. The signal is decomposing seasonality first, choosing the right detector, and tuning for alert fatigue. Here is the answer that survives production.Open full answer →
04Design a monitoring system for a fleet of 100+ production ML models.▼hardMetaMicrosoftDatabricks1 repliesunlockedModels fail silently; the question is whether you would know. The signal is monitoring the right layers (operational, data, prediction, outcome) and alerting on drift without drowning in false pages. Here is the system and the metrics that matter.Open full answer →
10Design an anomaly detection system for a metric (e.g. cloud billing) with seasonality and cold start.▼hardAmazonMicrosoftGoogle1 repliesunlockedAnomaly detection sounds easy until seasonality, cold start, and alert fatigue hit. The signal is modeling the expected baseline (including weekly and daily cycles), choosing unsupervised methods for scarce labels, and tuning to avoid drowning users in false alarms. Here is the design.Open full answer →
35Your ML monitoring is either too noisy to read or too quiet to trust. How do you design good alerts?▼mediumMetaGoogleStripe1 replies◆ premiumAn alert that fires constantly gets muted, and a model that fails with no alert is worse. Good ML alerting is a design problem with the same principles as SRE, plus ML-specific twists. Here is how to get it right.Open full answer →
