metrics
Applied AI interview questions tagged metrics, across every topic.
14 questions · 3 unlocked for you
Concepts behind "metrics"
The curriculum that explains the ideas these questions test.
Foundational
The Bias-Variance TradeoffA model's error decomposes into bias (error from being too simple to capture the pattern, underfitting) and variance (error from being too sensitive to the training sample, overfitting). Reducing one often raises the other, so generalization is about finding the balance. It is the lens behind regularization, model-complexity choices, and ensembling. Applied-AI interviews probe it because diagnosing whether a model underfits or overfits, and acting on it, is the core debugging skill of ML.📊 Evaluation & ML Foundations
Foundational
Cross-Validation (Done Right)Cross-validation estimates how a model generalizes by training and testing on rotating folds, giving a more reliable estimate than a single split. The traps are what make it an interview topic: use stratified folds for imbalanced classes, grouped folds when records share an entity, and time-ordered splits for temporal data (never random), and fit all preprocessing inside each fold to avoid leakage. Applied-AI interviews probe it because the wrong scheme produces optimistic estimates that fall apart in production.📊 Evaluation & ML Foundations
Foundational
Ensembling: Bagging, Boosting, StackingEnsembles combine multiple models to beat any single one, because if their errors are decorrelated, combining cancels mistakes. Bagging trains parallel models on bootstrap samples and averages (reducing variance, e.g. random forest); boosting trains models sequentially to fix prior errors (reducing bias, e.g. XGBoost); stacking trains a meta-model to combine base models. Diversity among models is the requirement. Applied-AI interviews probe it because gradient boosting dominates tabular ML and the bias/variance framing connects to everything.📊 Evaluation & ML Foundations
Foundational
Precision, Recall, and F1: Thresholds and Imbalanced DataPrecision is how many flagged items were right, recall is how many real positives you caught. How they trade off with the threshold, and why accuracy lies.📊 Evaluation & ML Foundations
Foundational
Eval-Driven Development and Golden DatasetsYou cannot improve an LLM system you cannot measure, so the first thing to build is an evaluation: a golden dataset of representative inputs with expected behavior, plus metrics, that you run on every change. This turns 'it feels better' into a number, catches regressions before users do, and lets you iterate quickly. Applied-AI interviews probe it because teams that ship reliable LLM features evaluate continuously, and 'we tried some prompts and it looked good' is the anti-pattern.📊 Evaluation & ML Foundations
Foundational
Offline vs Online Evaluation: Why Offline Wins Fail to HoldWhat held-out metrics measure, what an A/B test measures, and why they disagree: static data, feedback loops, and shift. Gate offline, confirm online.📊 Evaluation & ML Foundations