TL;DR — Every ML-system-design interview is the same seven-step walk applied to a different product. This article gives you a reusable FRAMEWORK and then works five full problems end-to-end — recommendations, search ranking, news feed, fraud detection, and ad CTR — each with a diagram, the one metric that matters, and the one tradeoff the interviewer is really probing. Read this and you can walk into any "Design X's ML system" prompt and drive the conversation instead of freezing.
0. The ML System Design interview FRAMEWORK
Interviewers score you on a consistent rubric — problem formulation, architecture, ML depth, tradeoff reasoning, and evaluation/monitoring. Drive the conversation through seven steps and you hit all of them. Budget your time roughly 10% clarify → 20% high-level → 50% deep-dive (data/model/serving) → 20% eval + tradeoffs.
1. REQUIREMENTS & SCALE What are we optimising? Functional + non-functional.
- Clarify the business goal → the ML objective (a proxy you can train on).
- Scale: #users, #items, QPS, latency SLA, freshness. Do the napkin math.
2. DATA & FEATURES Where do labels come from? What features, from where?
- Label source (implicit clicks? explicit? delayed labels?). Positives/negatives.
- Feature families: user, item, context, cross, historical/aggregate, embeddings.
- Feature store: offline (train) + online (serve), SAME transform (no skew).
3. CANDIDATE GENERATION Cut millions of items → hundreds. (retrieval)
- Cheap recall: ANN over embeddings, inverted index, rules, co-visitation.
- Optimise RECALL here, not precision. Multiple sources, then merge.
4. MODEL & RANKING Score the shortlist precisely. (precision)
- Feature-rich model (GBDT / two-tower + DNN / wide&deep). Loss = the objective.
- Re-ranking layer: diversity, freshness, business rules, calibration.
5. SERVING / INFRA Make it real-time, reliable, cheap.
- Two-stage online; precompute embeddings offline; feature store; caching.
- Latency budget split across stages; batching; autoscaling; model registry.
6. EVALUATION Prove it works — offline THEN online.
- Offline: replay held-out data → ranking/classification metrics.
- Online: A/B test on a BUSINESS metric with significance. Shadow → canary → A/B.
7. SCALING & FAILURE MODES What breaks, and how you defend.
- Cold start, feedback loops, drift, stale features, hot keys, cascading fallback,
fairness/bias, adversaries (fraud/ads), retraining cadence.
Two ideas thread through every problem below:
- The two-stage funnel — candidate generation (cheap, high recall) → ranking (expensive, high precision). This is how you serve billions of items under a tens-of-ms budget.
- Offline metric ≠ online metric. You train/validate on an offline proxy (NDCG, AUC) but you ship on an online business metric (watch-time, conversion, fraud-loss). State both, every time.
1. Design a recommendation system (e.g. Spotify / Netflix / YouTube)
1. Requirements & scale. Goal: increase engagement (watch-time/retention). ML objective: predict P(user enjoys item), proxy = completed play / long watch. Scale: ~100M users × ~10M items, home screen must render in <200 ms, recs refresh per session. Personalised, diverse, not stale.
2. Data & features. Labels = implicit feedback (plays, skips, completes, likes) — abundant but biased (you only see what was shown → position/exposure bias). Features: user (history, demographics, taste embedding), item (genre, metadata, item embedding, popularity), context (time, device), cross (user×genre affinity). Negatives = shown-but-skipped + random sampled un-shown items.
3. Candidate generation. Two-tower model: a user tower and item tower each produce embeddings; item embeddings are precomputed offline into an ANN index (FAISS/ScaNN/Pinecone). At request time embed the user, ANN-retrieve ~500 nearest items. Merge with other sources (trending, followed-artists, collaborative co-visitation).
4. Model & ranking. Rank the ~500 with a heavier DNN / gradient-boosted model on rich features, predicting P(complete). Then re-rank for diversity (don't show 10 songs by one artist), freshness, and exploration.
5. Serving. Offline: nightly retrain embeddings + candidate index. Online: user-embed → ANN → rank → re-rank, all under budget; cache per-user candidate lists briefly; feature store for online features.
user ──► [user tower] ──embed──► ANN over item index (10M→500) ──► RANKER (DNN) ──► re-rank ──► top-k
▲ precomputed offline │ P(complete) diversity/
item embeddings (batch) rich features freshness
6. Evaluation. Offline: NDCG@k, Recall@k, MAP on held-out interactions. Online: A/B on watch-time / retention / CTR with significance.
- Key metric: NDCG@k offline; long-term retention online.
- The tradeoff — accuracy vs diversity/exploration. Pure
P(complete)maximisation creates a filter bubble and feedback loop (popular gets more popular). Inject exploration (bandits) and diversity constraints, accepting slightly lower short-term CTR for healthier long-term engagement.
(Deep dive: Recommendation Systems — CF, matrix factorisation, group recs & fairness.)
2. Design search ranking (e.g. Google / Amazon / e-commerce search)
1. Requirements & scale. Goal: return the most relevant results for a query, ranked, in <300 ms. Unlike recs, there's an explicit query → strong intent signal. Scale: billions of docs, tens of thousands of QPS.
2. Data & features. Labels from click logs + dwell time, ideally human relevance judgments (graded 0–4). Beware position bias (rank-1 gets clicked because it's rank-1) → correct with inverse-propensity weighting. Features: query (terms, embedding, intent class), document (BM25 score, freshness, quality/PageRank-like, embedding), query-doc (text match, semantic similarity, historical CTR for this pair), user/context.
3. Candidate generation (retrieval). Two lanes merged: lexical (inverted index + BM25) for exact-term recall and semantic (dense embedding ANN) for meaning. Retrieve a few thousand candidates — optimise recall.
4. Model & ranking — Learning to Rank (LTR). Rank candidates with an LTR model (LambdaMART/GBDT or a neural ranker) trained with a pairwise/listwise loss that directly optimises ordering (it's about relative order, not absolute scores). Re-rank for freshness, diversity, and spam demotion.
query ─► retrieval ─┬─ BM25 inverted index (lexical) ─┐
└─ dense ANN (semantic) ──────────┴─► merge (few thousand)
│
LTR RANKER (LambdaMART / neural)
pairwise/listwise loss → order
│ re-rank: fresh/diverse/spam
▼ top-10
6. Evaluation. Offline: NDCG (position-aware, graded relevance), MRR, MAP. Online: A/B on click-through, successful-session rate, query reformulation rate (fewer reformulations = better).
- Key metric: NDCG — it rewards putting graded-relevant docs higher, which is exactly what ranking is.
- The tradeoff — relevance vs latency (retrieval depth). Retrieving and scoring more candidates raises recall but costs latency and compute. Split the budget: cheap retrieval prunes hard, the expensive ranker only touches the shortlist. Hybrid lexical+semantic beats either alone but doubles retrieval cost.
(Deep dive: Search Systems.)
3. Design a news feed (e.g. Facebook / LinkedIn / Twitter feed ranking)
1. Requirements & scale. Goal: rank a user's eligible posts (from friends/follows/groups) to maximise meaningful engagement, refreshed in near-real-time, rendered in <200 ms. Scale: billions of users, high write throughput (new posts constantly).
2. Data & features. Labels = multi-action implicit feedback: like, comment, share, dwell, hide, "see less". Feed ranking predicts several actions, not one. Features: author-viewer affinity (interaction history), post (type, media, age, engagement velocity), viewer, context. Content freshness matters a lot — a great post from last week is worthless now.
3. Candidate generation. Retrieve eligible inventory: posts from your graph within a recency window + injected sources (groups, recommended). Hundreds to low-thousands. This is a graph/recency retrieval, not ANN-first.
4. Model & ranking — multi-task. A multi-task DNN predicts P(like), P(comment), P(share), P(dwell), P(hide) in one model (shared bottom, per-task heads). Combine into a single score with business-tuned weights:
score = w_like·P(like) + w_comment·P(comment) + w_share·P(share)
+ w_dwell·E[dwell] − w_hide·P(hide) ← negative signals subtract
The weights encode product policy (comments/shares often weighted above passive likes). Re-rank for diversity (don't show 5 posts from one author) and integrity (demote misinformation/clickbait).
viewer ─► eligible posts (graph + recency) ─► MULTI-TASK DNN ─► weighted blend ─► integrity/
(hundreds–thousands) P(like/comment/ diversity
share/dwell/hide) re-rank ─► feed
6. Evaluation. Offline: per-task AUC/calibration. Online: A/B on meaningful engagement, session time, and guardrail metrics (report rate, "see less", next-day return).
- Key metric: a composite meaningful-engagement score, guarded by integrity metrics.
- The tradeoff — engagement vs integrity/well-being. Naively maximising engagement amplifies outrage/clickbait and creates feedback loops. Feed ranking deliberately adds negative-signal and integrity terms that lower raw engagement to protect long-term trust. This value-laden weighting is the crux of the problem.
(Deep dive: Feed Ranking.)
4. Design fraud detection (e.g. payments / account-takeover)
1. Requirements & scale. Goal: block fraudulent transactions while barely touching legitimate ones, decision in <100–300 ms at checkout. Extreme class imbalance (fraud « 1% of transactions) and an adversary who adapts. Cost of errors is asymmetric and quantifiable (a false negative = chargeback loss; a false positive = a blocked good customer).
2. Data & features. Labels = confirmed fraud/chargebacks — delayed (you learn weeks later) and noisy. Features: transaction (amount, merchant, currency), velocity/aggregates (txns per card in last 1m/1h/24h — the money features), user history, device/IP/geo, cross (amount vs user's typical). Real-time aggregate features are essential and must be computed with low latency (streaming feature store + cached counters).
3. Candidate generation. Not a retrieval problem — instead a rules layer + model layer. Cheap deterministic rules catch known patterns and block obvious fraud instantly; everything else goes to the model. Think funnel: rules → model score → (optional) manual-review queue for the uncertain middle.
4. Model & ranking. A gradient-boosted tree (XGBoost/LightGBM) on tabular features is the workhorse (handles mixed features, robust, interpretable-ish); graph/GNN features for rings. Handle imbalance with class weighting / focal loss / careful sampling — and evaluate on the right metric (below). Output a calibrated P(fraud); threshold by expected-cost.
txn ─► RULES (instant block/allow) ─► real-time features (velocity/aggregates from stream)
│ uncertain │
▼ ▼
MODEL P(fraud) (GBDT) ─► threshold by EXPECTED COST ─► allow / block / review-queue
(labels return later → retrain)
6. Evaluation. Accuracy is useless here — predict "all legit" and you're 99.5% accurate and worthless. Use precision, recall, and PR-AUC (precision-recall curve emphasises the minority class), and ultimately money saved vs friction caused (cost-weighted). Online: champion/challenger, watch fraud-loss rate and false-positive/decline rate.
- Key metric: PR-AUC / precision-recall at the operating threshold, translated to $ fraud loss vs false-decline cost.
- The tradeoff — recall (catch fraud) vs precision (don't annoy good users). Move the threshold: higher recall blocks more fraud but declines more good customers (lost revenue + churn). The threshold is a business cost decision, not a modelling one — set it by expected cost, and note labels are delayed so you monitor prediction drift as an early warning of evolving fraud (concept drift from adversaries).
(Deep dive: Fraud Detection.)
5. Design ad click-through-rate (CTR) prediction
1. Requirements & scale. Goal: for each candidate ad, estimate P(click) (and downstream P(conversion)) to rank ads and price them. This drives a multi-billion-dollar auction, so predictions must be well-calibrated (a predicted 0.02 must mean a true 2% click rate — the number feeds the bid). Massive scale: millions of ads, huge sparse feature space, <50 ms per request, retrained very frequently (freshness matters hugely).
2. Data & features. Labels = clicks (abundant, real-time, but heavily imbalanced — most impressions aren't clicked). Features: user, ad/advertiser, context (page, placement, device, time), and — crucially — feature crosses (user_country × ad_category) that capture interactions. Features are massive, sparse, categorical (billions of one-hot dimensions) → hashed and embedded.
3. Candidate generation. Ad targeting/retrieval: match the request against eligible ads by targeting rules + budget/pacing constraints → a few hundred/thousand eligible ads.
4. Model & ranking. Classic baseline: logistic regression on hand-crafted feature crosses — simple, fast, calibrated, but relies on manual cross engineering. Modern: Wide & Deep / DeepFM / DCN (Deep & Cross Network) — the "wide" part memorises explicit crosses, the "deep" part generalises via embeddings and learns high-order crosses automatically. Rank ads by expected value = P(click) × bid (× quality), run the auction, and calibrate the probabilities (Platt/isotonic) because the raw number sets the price.
request ─► targeting/retrieval (eligible ads + budget) ─► CTR MODEL (Wide&Deep / DCN)
│ calibrated P(click)
▼
rank by E[value] = P(click) × bid × quality ─► auction ─► ad
6. Evaluation. Offline: AUC (ranking quality) and log-loss / calibration (are the probabilities honest?). Online: A/B on actual CTR, revenue, and advertiser ROI.
- Key metric: AUC + calibration (log-loss) offline; revenue / effective CTR online. Calibration is non-negotiable because the probability is a price input, not just a ranking.
- The tradeoff — model complexity/freshness vs latency & calibration. Deep cross networks lift AUC but cost latency and can be worse-calibrated than logistic regression; and CTR data drifts fast, so retraining cadence (hourly/continuous) often matters more than a fancier architecture. You balance a richer model against a strict per-request budget and the need for trustworthy, frequently-refreshed probabilities.
6. Cross-cutting patterns (say these and you sound senior)
TWO-STAGE FUNNEL candidate gen (recall, cheap) → ranking (precision, expensive).
Recs, search, feed, ads all share it; fraud swaps retrieval for rules.
OFFLINE ≠ ONLINE validate on NDCG/AUC/PR-AUC; SHIP on watch-time/revenue/fraud-$.
IMPLICIT LABELS clicks/plays are cheap but biased (position/exposure) → debias.
FEEDBACK LOOPS the model shapes the data it's later trained on → exploration + de-bias.
COLD START new user/item/ad has no history → content features, popularity, exploration.
DRIFT & RETRAINING world moves (fraud adversaries, ad trends) → monitor drift, retrain on cadence.
CALIBRATION needed when the probability is used as a NUMBER (ads pricing, fraud cost).
ROLL-OUT SAFETY shadow → canary → A/B; never flip 100% traffic to a new model.
7. Interview questions companies actually ask
Q [easy] "Walk me through your framework for any ML system design question."
A Requirements/scale → data & features → candidate generation → model & ranking →
serving/infra → evaluation (offline then online) → scaling & failure modes. Clarify the
business goal, translate it to an ML objective, and state offline AND online metrics.
Q [easy] "Why is accuracy a bad metric for fraud detection?"
A Extreme imbalance: predict 'all legitimate' and you're ~99.5% accurate but catch zero
fraud. Use precision, recall, PR-AUC, and ultimately cost-weighted $ fraud loss vs
false-decline cost.
Q [medium] "Why two stages (candidate generation + ranking) instead of one big model?"
A You can't run an expensive feature-rich model over millions of items in tens of ms.
Cheap retrieval (ANN/index/rules) cuts millions → hundreds optimising RECALL; the heavy
ranker scores only the shortlist optimising PRECISION. It's the latency/quality tradeoff.
Q [medium] "How do you evaluate a recommender or ranker offline vs online?"
A Offline: replay held-out interactions → NDCG@k, Recall@k, MAP (position-aware for ranking).
Online: A/B test on a business metric (watch-time, conversion, revenue) with statistical
significance, rolled out via shadow → canary → A/B. Offline is a proxy; online decides.
Q [medium] "Feed ranking: how do you combine multiple engagement signals?"
A Multi-task model predicts P(like), P(comment), P(share), P(dwell), P(hide); blend with
business-tuned weights, subtracting negative signals (hide/see-less), then re-rank for
diversity and integrity. Weights encode product policy, not just accuracy.
Q [medium] "Why does calibration matter for ad CTR but less for a movie recommender?"
A In ads the predicted probability is a PRICE input (bid × P(click)) and feeds an auction,
so a 0.02 must mean a true 2%. A recommender only needs the right ORDER, so ranking
metrics (NDCG) suffice and raw scores can be uncalibrated.
Q [hard] "What are feedback loops and how do you mitigate them?"
A The model chooses what users see, so it shapes the very data it's next trained on —
popular items get more exposure, get recommended more, look even more popular. Mitigate
with exploration (bandits), inverse-propensity/de-biasing, and diversity constraints.
Q [hard] "How do delayed labels change your fraud/CTR design?"
A You can't compute live accuracy, so monitor PREDICTION and INPUT drift as early warnings,
use champion/challenger, and design a label pipeline (chargebacks, delayed conversions)
with attribution windows. Retrain on a cadence tuned to how fast the world (adversary) moves.
Q [hard] "How would you handle cold start across these systems?"
A New user/item/ad has no interaction history: fall back to content/metadata features,
popularity/trending, onboarding signals, and exploration to gather data fast; switch to
the collaborative/interaction-based model once enough history accrues (a hybrid switch).
Q [medium] "Position bias in click logs — what is it and how do you correct it?"
A Items shown higher get clicked partly BECAUSE they're higher, not because they're better,
so naive training rewards whatever ranked high. Correct with inverse-propensity weighting,
randomisation/exploration data, or modelling examination probability separately.
Sources: Meta ML System Design Guide (2026) · ML System Design Interview Guide — Interview Kickstart · ML System Design Interview — Exponent · Evaluation — Hello Interview · Fraud Detection System Design (2026) · Precision vs Recall in Fraud (TDS) · Deep & Cross Network for Ad Click Predictions · Google ML System Design Interview
8. When to use / tradeoffs (the metric + tradeoff cheat sheet)
| Problem | Key offline metric | Ship on (online) | The one tradeoff |
|---|---|---|---|
| Recommendation | NDCG@k | Watch-time / retention | Accuracy vs diversity/exploration (filter bubble) |
| Search ranking | NDCG / MRR | Successful-session rate | Relevance vs latency (retrieval depth) |
| News feed | Per-task AUC | Meaningful engagement + guardrails | Engagement vs integrity/well-being |
| Fraud detection | PR-AUC (precision/recall) | $ fraud loss vs false-decline | Recall vs precision (cost-set threshold) |
| Ad CTR | AUC + calibration/log-loss | Revenue / effective CTR | Model complexity/freshness vs latency & calibration |
9. Summary + related articles
- Every prompt is the same seven-step framework: requirements → data/features → candidate generation → model/ranking → serving → evaluation → scaling/failure modes.
- The two-stage funnel (cheap high-recall retrieval → expensive high-precision ranking) is the universal architecture; fraud swaps retrieval for a rules layer.
- Always state offline metric (NDCG/AUC/PR-AUC) AND the online business metric — they differ.
- Match the metric to the problem: NDCG for ranking, PR-AUC for imbalanced fraud, AUC+calibration for priced ads.
- Name the failure modes: cold start, feedback loops, drift, delayed labels, position bias, and roll out via shadow → canary → A/B.
Related: Recommendation Systems · Search Systems · Feed Ranking · Fraud Detection · ML Inference Systems · (scalability: ../11-1/scalability-basics.md) · (distributed systems: ../11-1/distributed-systems.md)
Resources
- ML System Design Interview Guide (2026) — https://interviewkickstart.com/blogs/articles/machine-learning-system-design-interview-guide
- ML System Design framework & evaluation — https://www.hellointerview.com/learn/ml-system-design/core-concepts/evaluation
- Fraud Detection System Design — https://www.systemdesignhandbook.com/guides/fraud-detection-system-design/
- Deep & Cross Network for Ad Click Predictions — https://arxiv.org/pdf/1708.05123
- System Design Primer — https://github.com/donnemartin/system-design-primer