metrics
Applied AI interview questions tagged metrics, across every topic.
14 questions · 3 unlocked for you
Concepts behind "metrics"
The curriculum that explains the ideas these questions test.
Foundational
The Bias-Variance TradeoffA model's error decomposes into bias (error from being too simple to capture the pattern, underfitting) and variance (error from being too sensitive to the training sample, overfitting). Reducing one often raises the other, so generalization is about finding the balance. It is the lens behind regularization, model-complexity choices, and ensembling. Applied-AI interviews probe it because diagnosing whether a model underfits or overfits, and acting on it, is the core debugging skill of ML.📊 Evaluation & ML Foundations
Foundational
Cross-Validation (Done Right)Cross-validation estimates how a model generalizes by training and testing on rotating folds, giving a more reliable estimate than a single split. The traps are what make it an interview topic: use stratified folds for imbalanced classes, grouped folds when records share an entity, and time-ordered splits for temporal data (never random), and fit all preprocessing inside each fold to avoid leakage. Applied-AI interviews probe it because the wrong scheme produces optimistic estimates that fall apart in production.📊 Evaluation & ML Foundations
Foundational
Ensembling: Bagging, Boosting, StackingEnsembles combine multiple models to beat any single one, because if their errors are decorrelated, combining cancels mistakes. Bagging trains parallel models on bootstrap samples and averages (reducing variance, e.g. random forest); boosting trains models sequentially to fix prior errors (reducing bias, e.g. XGBoost); stacking trains a meta-model to combine base models. Diversity among models is the requirement. Applied-AI interviews probe it because gradient boosting dominates tabular ML and the bias/variance framing connects to everything.📊 Evaluation & ML Foundations
Foundational
Precision, Recall, and F1Precision is how many of your positive predictions were right; recall is how many of the actual positives you caught. They trade off as you move the decision threshold, and which matters depends on the cost of false positives vs false negatives. F1 is their harmonic mean. On imbalanced data, accuracy lies and these metrics (with PR-AUC) tell the truth. Applied-AI interviews probe them because choosing and tuning the threshold by business cost is a core, constantly-tested skill.📊 Evaluation & ML Foundations
Foundational
Eval-Driven Development and Golden DatasetsYou cannot improve an LLM system you cannot measure, so the first thing to build is an evaluation: a golden dataset of representative inputs with expected behavior, plus metrics, that you run on every change. This turns 'it feels better' into a number, catches regressions before users do, and lets you iterate quickly. Applied-AI interviews probe it because teams that ship reliable LLM features evaluate continuously, and 'we tried some prompts and it looked good' is the anti-pattern.📊 Evaluation & ML Foundations
Foundational
Offline vs Online EvaluationOffline evaluation scores a model on held-out data; online evaluation measures its impact on real users (via an A/B test). They often disagree: an offline win frequently fails to move the online metric, because offline data is a static proxy and the real world has feedback loops, distribution shift, and second-order effects. The discipline is to gate with offline evals (fast, cheap) and confirm with online tests (the truth). Applied-AI interviews probe it because shipping on offline metrics alone is a classic, costly mistake.📊 Evaluation & ML Foundations