TL;DR — A recommender predicts what a user will like and ranks items by that predicted preference. The three classic engines are collaborative filtering (people like you liked X), content-based (items similar to what you liked), and hybrid (both). This article also covers group recommendation (one pick for many people) and fairness — a heavily-interviewed, less-taught corner — using a fairness-aware group recommender as the running example.
1. Simple explanation
A recommender answers: "Given everything I know about this user (and everyone else), which items should I show, in what order?"
Analogy: a great travel agent. A content-based agent says "you loved that quiet beach trip, here's another quiet beach trip." A collaborative agent says "travelers who booked what you did also loved this city break — try it." A group agent, planning for 4 friends, must pick one trip everyone's reasonably happy with — that's where fairness enters.
2. The landscape (diagram)
RECOMMENDER SYSTEMS
│
├── Collaborative Filtering ─ "people like you liked X"
│ ├─ user-based CF · item-based CF
│ └─ matrix factorization (latent factors)
├── Content-Based ─ "items similar to what you liked" (features + cosine similarity)
├── Hybrid ─ combine both (weighted / switching / cascade)
│
├── GROUP recommendation ─ one pick for MANY users (aggregation strategies)
└── FAIRNESS ─ don't leave a minority member miserable (MinSat, Jain, variance)
Every engine ends the same way: score every item → RANK → return top-k.
3. Collaborative Filtering (CF)
Idea: use the user–item interaction matrix (who rated/clicked what) — no item content needed.
- User-based CF: find users similar to you, recommend what they liked.
- Item-based CF: find items similar to ones you liked (by who co-liked them).
- Matrix Factorization (MF): the scalable workhorse — factor the sparse rating matrix
Rinto two small matrices of latent factors.
The math (matrix factorization):
R ≈ U · Vᵀ
R = users × items rating matrix (mostly empty)
U = users × k (each user = k hidden "taste factors")
V = items × k (each item = k hidden factors)
predicted rating: r̂_ui = u_u · v_i (dot product)
train by minimizing: Σ_(u,i observed) (r_ui − u_u·v_i)² + λ(‖u_u‖² + ‖v_i‖²)
└── fit the known ratings ──┘ └── regularization ──┘
In plain words: learn a short vector for every user and every item so their dot product reproduces the known ratings; then predict the blanks. λ prevents overfitting.
# tiny user-based CF: recommend items liked by similar users
import numpy as np
def cosine(a, b): return a @ b / (np.linalg.norm(a) * np.linalg.norm(b) + 1e-9)
def user_based_recommend(user_row, R, k=3):
sims = [cosine(user_row, other) for other in R] # similarity to every user
top = np.argsort(sims)[-k-1:-1] # k most similar (excl self)
scores = R[top].mean(axis=0) # avg their ratings
scores[user_row > 0] = -1 # hide already-seen items
return np.argsort(scores)[::-1] # ranked recommendations
Challenges: the cold-start problem (new user/item has no interactions), sparsity, and popularity bias.
4. Content-Based Filtering
Idea: describe each item by features (a vector), describe the user by the items they liked, recommend items whose vectors are closest to the user's taste.
The math (cosine similarity — the core of most recsys):
sim(a, b) = cos(a, b) = (a · b) / (‖a‖ · ‖b‖) → −1 … 1 (1 = same direction/taste)
No interaction data needed → solves cold-start for new items (you have their features immediately). Limitation: it only recommends things similar to your history (a "filter bubble"), and needs good features.
5. Hybrid
Combine CF + content to cover each other's weaknesses:
Weighted → score = α·CF + (1−α)·content
Switching → use content for cold-start users, CF once they have history
Cascade → content narrows the candidates, CF re-ranks them
Almost every production system (Netflix, Spotify, YouTube) is a hybrid + a learned ranker on top.
6. ★ GROUP RECOMMENDATION (one pick for many people)
Now the harder, less-taught problem: n people, one choice. Each person i has a preference vector tᵢ; each item x has a feature profile τ_x. Their satisfaction:
sᵢ(x) = cos(tᵢ, τ_x) → how happy person i would be with item x (0…1)
Build the score table (people × safe items), then a selector picks one item. The four selectors:
satisfaction sᵢ(x):
R1 R2 R3
Asha 0.9 0.6 0.5
Ben 0.2 0.7 0.5
Cara 0.4 0.5 0.5
| Selector | Formula | Meaning | Picks (example) |
|---|---|---|---|
| Average (utilitarian) | argmax (1/n) Σ sᵢ | highest average happiness | R2 (0.60) |
| Least-Misery (egalitarian) | argmax min_i sᵢ | protect the worst-off | R2/R3 (0.50) |
| Most-Pleasure | argmax max_i sᵢ | make the happiest happier | R1 (0.90, but Ben=0.2 ❌) |
| Fairness-aware ★ | argmax (1/n)Σ sᵢ − λ·Var(sᵢ) | high average but penalize disagreement | R2 (0.593) |
The fairness-aware objective (worked with λ = 1):
Var(x) = (1/n) Σ (sᵢ − avg)² (how spread out the scores are)
R1: avg 0.50, Var 0.087 → 0.50 − 0.087 = 0.413
R2: avg 0.60, Var 0.007 → 0.60 − 0.007 = 0.593 ← WINNER (high avg AND fair)
R3: avg 0.50, Var 0.000 → 0.50 − 0.000 = 0.500
λ (lambda) is the fairness dial: λ=0 → pure Average; larger λ → punish choices where some people are left out. Most-Pleasure would pick R1 and ignore Ben — the fairness-aware selector avoids exactly that.
def satisfaction(person_pref, item_pref):
a, b = np.array(person_pref), np.array(item_pref)
return float(a @ b / (np.linalg.norm(a) * np.linalg.norm(b) + 1e-9)) # cosine 0..1
def average(s): return sum(s) / len(s)
def least_misery(s): return min(s)
def most_pleasure(s): return max(s)
def fairness_aware(s, lam=1.0):
avg = sum(s) / len(s)
var = sum((x - avg) ** 2 for x in s) / len(s)
return avg - lam * var
def pick(members, safe_items, selector="fairness_aware", lam=1.0):
fns = {"average": average, "least_misery": least_misery, "most_pleasure": most_pleasure}
best, best_val = None, -1e9
for item in safe_items: # only SAFE options
col = [satisfaction(m.pref, item.pref) for m in members] # each member's score
val = fairness_aware(col, lam) if selector == "fairness_aware" else fns[selector](col)
if val > best_val:
best, best_val = item, val
return best
6b. Interactive & Conversational Recommendation (closing the loop)
A one-shot recommender guesses once and hopes. An interactive/conversational recommender learns during the interaction — it recommends, watches the reaction, and updates. This is a general and increasingly-asked pattern (conversational recsys, LLM shopping assistants, group mediators).
THE LOOP (general):
recommend → get FEEDBACK (accept / reject / "more like this" / "cheaper")
→ UPDATE the model/weights → re-recommend → until the user is satisfied
For a GROUP, a mediator closes it by re-weighting members from feedback, then re-running the fair selector. A clean update rule is multiplicative weights / exponentiated-gradient:
wᵢ ← wᵢ · exp(η · gᵢ) / Z
wᵢ = member i's current weight (how much their happiness counts right now)
gᵢ = feedback signal for member i (e.g. +1 if they objected/are under-served, 0 otherwise)
η = learning rate (how fast we react) Z = normaliser so weights sum to 1
→ then re-pick with the WEIGHTED fair score: argmax Σ wᵢ·sᵢ(x) − λ·Var(sᵢ)
→ loop until consensus (no more objections / a round cap).
In plain words: whoever is being left out gets more weight next round, so the next pick leans their way — the system negotiates instead of guessing once.
Related general tools: this is the recsys cousin of online learning and multi-armed bandits (explore vs exploit — try uncertain items to learn preferences, exploit known-good ones). An LLM can drive the loop in natural language (elicit "why not?", update, explain the new pick).
7. ★ FAIRNESS in recommendations (metrics)
How do you measure whether a recommendation was fair to the group?
MinSat = min_i sᵢ → the least-happy member's score (higher = fairer)
Variance = (1/n)Σ(sᵢ−avg)² → spread; lower = more equal
Jain's index = (Σ sᵢ)² / (n · Σ sᵢ²) → ranges 1/n … 1 (1 = perfectly equal)
Max-Envy = max_i,j (sⱼ − sᵢ) → biggest happiness gap between two members
There's a fairness–utility tradeoff: pushing fairness (higher MinSat) usually lowers the average. The λ dial makes that tradeoff explicit and tunable.
8. Evaluation (offline + online)
Offline (ranking quality):
Precision@k = (relevant in top-k) / k Recall@k = (relevant in top-k) / (all relevant)
MAP = mean average precision Hit-rate@k
NDCG@k = DCG@k / IDCG@k where DCG = Σ rel_i / log2(i+1)
→ rewards putting relevant items HIGHER (position matters)
Online (does it move the business): A/B test → CTR, conversion, watch-time, retention — with statistical significance (don't ship on noise). Watch for feedback loops (recommending popular items makes them more popular).
9. Cold start + real-time (production)
- Cold start: new user → content-based / popularity / onboarding questions; new item → content features until interactions accrue.
- Real-time: precompute candidate generation (ANN over embeddings), then a fast learned ranker at request time; stream interactions back for online updates. Two-stage candidate-generation → ranking is the standard large-scale architecture (YouTube/Meta).
10. Real-world example (a group trip-planning app)
Consider a fairness-aware group trip recommender:
- Each traveler + each trip → a preference vector (e.g. adventure / relaxation / culture / nightlife / pace / budget-sensitivity).
- Satisfaction
sᵢ(x) = cosine(tᵢ, τ_x); retrieval is a vector search over trip embeddings (content-based) with a keyword fallback. - A group's pick uses the four aggregation selectors above; a fairness-aware setup adds the fairness-aware selector (avg − λ·Var) plus an LLM mediator that re-weights members from feedback.
- Evaluated with MinSat / Jain / Max-Envy / NDCG / consensus rounds vs the Average and Least-Misery baselines.
- Safe set
F(hard-constraint-filtered options — budget, dates, accessibility) is treated as given — you only ever recommend feasible trips.
11. Interview questions companies actually ask
Q [easy] "How does a recommendation system work?"
A Score every item for the user (via CF, content, or hybrid), then rank and return top-k.
Q [easy] "Collaborative vs content-based filtering?"
A CF uses interaction patterns ("people like you liked X"), no item features; content-based
uses item features + your history ("similar to what you liked"). CF has cold-start; content solves it.
Q [medium] "What's the cold-start problem and how do you handle it?"
A A new user/item has no interactions, so CF can't score it. Fixes: content-based/popularity/
onboarding for new users; item features for new items; a hybrid switch once data accrues.
Q [medium] "Design YouTube/Netflix recommendations."
A Two stages: candidate generation (ANN over user/item embeddings → thousands of candidates)
→ ranking (a learned model scores them with rich features) → business re-ranking (diversity,
freshness). Log interactions → retrain. Evaluate with NDCG offline + A/B (watch-time) online.
Q [medium] "Which metric — and why not accuracy?"
A Ranking metrics: Precision@k/Recall@k, NDCG (position-aware), MAP. Accuracy ignores ranking
ORDER and lies under popularity bias; NDCG rewards putting relevant items at the top.
Q [hard] "Recommend ONE item for a GROUP — how?"
A Score each member's satisfaction per item, then aggregate: Average (utilitarian),
Least-Misery (protect worst-off), or fairness-aware (avg − λ·Var) to balance happiness vs
leaving someone out. Optionally a mediation loop that re-weights members from feedback.
Q [hard] "How do you make recommendations FAIR?"
A Define fairness explicitly (MinSat, Jain index, low variance, low max-envy), optimize a
utility−λ·fairness objective, and measure the utility–fairness tradeoff — don't just maximize CTR.
Q [hard] "Feedback loops / popularity bias — what and how to fix?"
A The system recommends popular items → they get more interactions → get recommended more.
Fix with exploration (bandits), de-biasing/propensity weighting, and diversity constraints.
12. When to use / tradeoffs
CF → lots of interaction data, no good item features. Cold-start weak.
Content-based → good item features, sparse interactions, new items. Filter-bubble risk.
Hybrid → almost always in production (covers both weaknesses).
Group + fairness → any "decide for a group" product (travel, playlists, meetings, events);
pick Average for max total happiness, fairness-aware to protect minorities.
13. Summary + related articles
- Recommenders score → rank → top-k; the three engines are CF, content-based, hybrid.
- CF math = matrix factorization (
R ≈ U·Vᵀ); content math = cosine similarity. - Group recommendation aggregates per-member satisfaction; the fairness-aware selector (
avg − λ·Var) balances utility vs leaving someone out. - Fairness metrics: MinSat, Jain index, variance, max-envy; there's a real utility–fairness tradeoff.
- Eval: NDCG/Precision@k offline, A/B (CTR/retention) online; beware feedback loops.
Related: Feed Ranking · Search Systems · ML Inference Systems · Common ML System Design Interview Questions · (embeddings: ../../nlp/3-6/) · (fairness: ../../ai-safety/7-5/) · (agent mediator: ../../ai-agents/6-4/)
Resources
- Recommendation Systems — https://www.coursera.org/learn/recommender-systems
- Matrix Factorization (Koren et al.) — https://datajobs.com/data-science-repo/Recommender-Systems-[Netflix].pdf
- Group Recommendation & fairness — search: "aggregation strategies group recommender systems"