TL;DR: Use a multi-stage funnel: cheap candidate generation narrows millions of items to a few hundred (two-tower retrieval plus heuristic sources), then an expensive ranker (a DLRM/DCN-style model) scores those hundreds on rich features, then a re-rank pass applies business rules, diversity, and freshness. Train on logged interactions, evaluate offline (AUC/nDCG) and online (A/B on engagement), and design explicitly for cold start and feedback loops.
How to approach it. Clarify the objective metric (watch time, clicks, long-term retention) and the constraints (latency budget, catalog size, freshness) before drawing anything. Then lay out the funnel and justify why you cannot just score everything: at 100M users and millions of items, ranking every item per request is computationally impossible, so you stage the work and spend compute only where it changes the answer.
A strong answer.
millions of items
| CANDIDATE GENERATION (~1-10ms): two-tower ANN (user emb · item emb),
| plus recency/popularity/follow-graph sources, unioned
v ~hundreds of candidates
| RANKING (~10-50ms): heavy model (DLRM/DCN/DeepFM) on rich
| user x item x context features -> predicted engagement
v ranked list
| RE-RANK: diversity, freshness, business rules, dedupe, fatigue
v final feed
- Candidate generation must be cheap and high-recall. A two-tower model embeds users and items independently so item vectors are precomputed and served via ANN (HNSW); blend with non-personalized sources (trending, recently published, social graph) so new users and new items still get coverage.
- Ranking is where personalization quality lives. A two-tower bi-encoder is too coarse here; use a feature-rich model over user history, item, and context (time, device), trained on logged clicks and watches. This stage sees only the few hundred candidates, so it can afford a heavy cross-feature model inside the latency budget.
- Re-ranking enforces what the ranker ignores: diversity (no ten near-identical items), freshness, frequency capping, and policy rules.
Cross-cutting concerns that signal seniority:
- Cold start. New users get popularity and context-based candidates; new items get content-feature embeddings until they accrue interactions.
- Feedback loops and position bias. The model is trained on items the old model surfaced; correct with logged propensities or an exploration slice so the system does not ossify around its own past picks.
- Freshness. Stream features into an online store; periodically refresh item embeddings.
- Evaluation. Offline AUC/nDCG to gate, then an online A/B on the real objective; offline wins do not always survive online (calibration, latency, novelty effects).
| Stage | Candidates in | Model | Latency | Optimizes for |
|---|---|---|---|---|
| Retrieval | Millions | Two-tower + ANN, heuristics | ~1-10ms | Recall |
| Ranking | ~Hundreds | DLRM/DCN cross-feature | ~10-50ms | Precision of engagement |
| Re-rank | Top ranked | Rules, diversity, freshness | ~1ms | Feed quality, policy |
Key takeaways
- Stage the funnel so you spend heavy compute only on the few hundred candidates that survive retrieval.
- Two towers for retrieval (precompute item vectors for ANN), cross-feature model for ranking (accuracy on a small set).
- Correct the feedback loop with propensities or exploration, or the model ossifies around its own past picks.
- Gate on offline AUC/nDCG but decide on an online A/B against the true objective, not the offline proxy.
What interviewers probe next.
- "Why two-tower for retrieval but not ranking?" Independent towers let you precompute item vectors for ANN at scale; ranking can afford a cross-feature model on the small candidate set for accuracy.
- "How do you serve features at <50ms p99?" Online feature store (Redis/Cassandra) with training-serving parity; precompute what you can.
- "Optimizing engagement can harm long-term retention. What do you do?" Add long-term and quality objectives plus guardrail metrics; do not optimize a single short-term proxy blindly.
- "Drift and retraining?" Monitor feature and prediction distributions; retrain on a cadence plus drift triggers.
Common mistakes.
- Trying to rank the full catalog per request instead of staging retrieval then ranking.
- Ignoring cold start, so new users and items get nothing.
- Optimizing a single short-term metric and creating a degenerate feedback loop.
- Quoting offline metrics as proof of success without an online A/B.