TL;DR: Engineer features from domain knowledge and the model's blind spots (interactions, ratios, time-based aggregates), encode categoricals by cardinality (one-hot for low, target/embedding for high, with leakage-safe target encoding), and handle missing data by first asking why it is missing (sometimes missingness is signal: add an indicator) before imputing. The cardinal rule: fit every transform on training data only, inside the CV fold, or you leak.
How to approach it. Lead with the discipline that prevents the most common bug: fitting preprocessing on the full dataset leaks test information. Then cover the three areas (engineering, encoding, missingness) with their specific traps. The interviewer is checking for leakage-awareness as much as technique breadth, so name the fit/transform boundary early.
A strong answer. Feature engineering. Good features encode domain knowledge the model cannot easily learn: interactions and ratios (price per square foot), time-based aggregates (purchases in the last 7 days, with point-in-time correctness), binning, and decompositions (date into day-of-week, is-holiday). For tabular problems this often matters more than model choice. Tree models handle raw scales and monotone transforms fine; linear and distance/NN models need scaling (standardize) and care with skew.
Encoding categoricals, by cardinality:
| Cardinality | Encoding | Watch out for |
|---|---|---|
| Low (color, state) | one-hot (column per category) | dimensionality if it creeps up |
| High (zip, user id) | target/mean encoding or learned embeddings | label leakage; needs out-of-fold or smoothing |
| Ordinal (size S/M/L) | integer encode preserving order | do not one-hot away the order |
Target encoding (replace the category with the mean target for that category) is powerful but a notorious leakage source: computing the mean over the whole dataset including the row's own target leaks the label. Use out-of-fold encoding (compute the mean from other folds) or smoothing toward the global mean, never naively on all data.
Missing data, ask why first. Before imputing, determine the mechanism: missing-at-random vs missing-not-at-random. Missingness is often informative (a blank income field may correlate with the target), so adding a "was-missing" indicator column frequently helps more than the imputed value. Then impute: simple (mean/median/mode) as a baseline, or model-based (KNN, iterative) when worth it. Gradient-boosted trees (XGBoost, LightGBM) handle missing values natively by learning the best split direction, which is often the simplest good answer.
The leakage rule that ties it together:
Fit scalers, imputers, and target encoders on training data only, inside each CV fold, and apply to validation/test. Fitting on the full dataset (computing the mean or scaler over everything before splitting) leaks information and inflates offline metrics, the same class of bug as temporal leakage. Build it as a pipeline so the fit/transform boundary is enforced automatically.
Key takeaways
- Fit every transform on the training fold only; fitting before the split is leakage and inflates offline metrics.
- Encode by cardinality: one-hot for low, target encoding or embeddings for high, integer for ordinal.
- Target encoding leaks the row's own label unless you use out-of-fold or smoothing.
- Treat missingness as a possible signal: add a was-missing indicator before assuming imputation is enough.
What interviewers probe next.
- "Why is target encoding risky?" It uses the label, so naive computation leaks each row's own target; use out-of-fold or smoothed encoding.
- "How do you decide whether to impute or flag missing?" Investigate the mechanism; if missingness correlates with the target (MNAR), an indicator captures real signal; pure imputation would discard it.
- "Scaling: when does it matter?" For distance-based (KNN, k-means, SVM) and gradient-based linear/NN models; tree models are scale-invariant.
- "How do you avoid leakage operationally?" Wrap preprocessing in a pipeline fit only on the training fold; never fit on data the model will be evaluated against.
Common mistakes.
- Fitting scalers/encoders/imputers on the full dataset before the split, leaking into validation.
- Naive target encoding that leaks the row's own label.
- Blindly imputing without checking whether missingness is itself informative.
- One-hot encoding a very high-cardinality feature, exploding dimensionality.