TL;DR — Retrieval is a two-stage cascade: a cheap scorer sweeps the whole corpus and returns
kcandidates, then an expensive scorer reorders just thosekinto the final few. Stage 1 optimises recall, stage 2 optimises precision, and the split exists because you cannot afford to run the good scorer over a million documents. Three things fall out of the simulation below, all measured. Depth has sharply diminishing returns: going fromk=5tok=20lifts nDCG@5 from 0.780 to 0.968 (+24%) for 4× the reranker calls, butk=20tok=60buys only +2% for 3× more. The ceiling is set by stage 1 — atk=20the best any reranker could achieve is 0.979, so 0.021 of the loss is unrecoverable recall against 0.011 of reranker error. And a weak reranker is worse than none: a reranker with no signal drops nDCG@5 to 0.259, far below the 0.706 you get by not reranking at all. Reranking is not free accuracy; it is a bet that your second scorer is genuinely better than your first.
1. Simple explanation
A vector search compares a question against a million documents. To make that affordable, the comparison has to be cheap: embed the question once, embed every document once (offline), compare with a dot product. That is a bi-encoder — question and document never meet, they are each squashed into a vector independently and compared at the end.
Squashing a document into one vector loses information. A 400-word passage becomes 1024 numbers, and whether it answers your specific question is not necessarily what survived the squash.
A cross-encoder does the thing the bi-encoder cannot: it reads the question and the document together, with attention flowing between them, and outputs a single relevance score. Far more accurate, and completely unaffordable at corpus scale — you would run a full transformer forward pass per document, per query.
So you cascade. Cheap scorer narrows a million to twenty. Expensive scorer picks the best five of those twenty.
Analogy — hiring. You cannot interview 2,000 applicants. So you screen résumés — fast, shallow, keyword-ish, and it makes mistakes in both directions. That gets you to 20 people you actually interview properly. The interview is your cross-encoder: expensive, high-signal, and applied only to a shortlist. Two consequences follow immediately, and they are the whole article. A brilliant candidate rejected at the résumé screen is never interviewed — no amount of interviewing skill recovers them. And if your interviewers are no better than a coin flip, interviewing actively destroys the ordering the résumé screen gave you.
2. Diagram
1,000,000 documents
│
▼
┌────────────────────────┐
│ STAGE 1 — bi-encoder │ cheap: one dot product per doc
│ or BM25 │ optimises RECALL: "don't lose the good ones"
└───────────┬────────────┘
│ top k candidates k is the ONE knob that matters
▼
┌────────────────────────┐
│ STAGE 2 — cross- │ expensive: one forward pass per CANDIDATE
│ encoder / LLM judge │ optimises PRECISION: "order these correctly"
└───────────┬────────────┘
│ top 5
▼
into the prompt
MEASURED (200 queries, 60 docs each, simulation in §5):
k stage1 reranked ceiling cost
───────────────────────────────────────────────
5 0.706 0.780 0.783 5 calls
10 0.706 0.916 0.922 10
20 0.706 0.968 0.979 20 <- knee of the curve
30 0.706 0.981 0.992 30
60 0.706 0.989 1.000 60 <- 3x cost, +2% quality
stage1 is FLAT: reranking never changes what stage 1 found.
`ceiling` is what a PERFECT reranker would score on that candidate set.
0.979 ceiling at k=20
├────────────────────────────────────┤ 0.021 lost to stage-1 recall
├────────┤ 0.011 lost to reranker error
▲
buying a better reranker fixes only THIS part
3. How it works
3.1 Bi-encoder vs cross-encoder
BI-ENCODER vec(query) · vec(doc) embed docs ONCE, offline
no interaction ~1 microsecond per doc
scales to 10^9 docs
CROSS-ENCODER model(query ++ doc) -> score nothing precomputable
full attention between them ~10 milliseconds per doc
scales to ~10^2 docs per query
The four-orders-of-magnitude cost gap is the reason cascades exist. A cross-encoder cannot precompute anything, because the document's representation depends on the query it is paired with — that dependency is exactly what makes it accurate.
3.2 The LLM as reranker
A cross-encoder is a trained model that emits a number. An LLM can do the same job zero-shot: hand it the query and a chunk, ask for a relevance score against a written rubric, parse the number.
System: Score how well the passage answers the question, 1-10.
1-3 unrelated or only topically adjacent
4-6 related, but does not directly answer
7-10 directly answers the question
Also return a one-line justification and a category label.
User: QUESTION: ...
PASSAGE: ...
This is slower and more expensive per candidate than a dedicated cross-encoder, and it buys three things a cross-encoder cannot give you:
- A rubric you can edit without retraining. Change the wording, change the ranking.
- Structured labels alongside the score. The same call can return a category, a quality tier, an extracted date, a flag for "this contradicts the question's premise." You are already paying for the forward pass; the extra fields are nearly free and feed downstream ranking.
- A justification string, which makes ranking decisions auditable — worth a great deal in regulated settings and for debugging.
The costs are real: latency measured in seconds not milliseconds, per-call price, output that needs parsing and can fail to parse, and score drift between model versions. Batching several candidates per call cuts the call count but degrades scores, because candidates in the same context window get compared against each other rather than against the rubric.
3.3 The knob that matters: candidate depth k
Everything about a cascade reduces to choosing k.
Too small and you have capped your quality before stage 2 runs — the best document may not be in the candidate set at all, and the reranker cannot conjure it. Too large and you pay linearly for candidates that were never going to make the top 5.
The measured curve in §5 has a pronounced knee. Quality rises steeply to about k=20, then flattens while cost keeps climbing linearly. Find your knee empirically and sit on it; do not inherit someone else's k.
3.4 Score thresholds and the empty-result problem
Reranking gives you calibrated-ish scores, so it is tempting to drop everything below a threshold rather than always returning the top 5. This is genuinely useful — it lets the system say "I found nothing relevant" instead of padding the prompt with the five least-bad documents, and irrelevant context measurably degrades generation.
The trap is that the threshold interacts with query difficulty. On hard queries every candidate scores low, so an aggressive threshold returns nothing at all. Measured below: a threshold of 6.0 returns an empty set on 7.5% of queries and drags mean nDCG@5 from 0.968 down to 0.751. Always define the fallback behaviour — return the single best candidate, ask a clarifying question, or say plainly that nothing was found.
3.5 Fusing several rankings instead
When candidates arrive from multiple retrievers rather than one, you can combine rankings directly instead of rescoring. Reciprocal rank fusion — RRF(d) = sum_b 1/(k + rank_b(d)) — needs no score calibration because it reads only positions. It is a fusion method, not a reranker: it reorders using the rankings you already have and adds no new information about the query-document pair. Derived with a worked example in Query Transformation: Fixing the Question Before You Retrieve.
Fusion and reranking compose: fuse the retrievers into one candidate list, then rerank the top k of it.
3.6 Where reranking stops helping
If stage 1 already returns the right documents in roughly the right order, a reranker has nothing to fix and you have added latency for noise. If stage 1 recall is terrible, reranking polishes a candidate set that does not contain the answer. Reranking pays in the middle: stage 1 finds the right documents but orders them badly. That is common, which is why reranking is usually worth it — but check that it describes your system before assuming it.
4. The math
4.1 Decomposing the loss
Let C(k) be the ceiling: the score a perfect reranker would achieve given the candidate set from stage 1 at depth k. Let A(k) be what your actual reranker achieves. Then total loss against a perfect system splits cleanly:
total loss = (1 - C(k)) + (C(k) - A(k))
^^^^^^^^^^ ^^^^^^^^^^^^^
stage-1 recall reranker error
fix by raising k fix by a better reranker
or a better retriever
Measured at k = 20:
C(20) = 0.979 A(20) = 0.968
stage-1 recall loss = 1 - 0.979 = 0.021
reranker error = 0.979 - 0.968 = 0.011
ratio = 0.021 / 0.011 = 1.9
Recall loss is nearly twice the reranker error. So the higher-leverage fix at this operating point is a deeper k or a better first stage — not a more expensive reranker. Computing this split for your own system takes one afternoon and routinely redirects the optimisation effort.
4.2 Why depth saturates
A relevant document survives to the final top-5 only if stage 1 ranks it within k. Raising k from 20 to 60 only helps for documents stage 1 ranked between 21st and 60th — documents it scored quite poorly. Those are rare, and each additional slot is less likely than the last to contain anything good, while every slot costs a full reranker call.
marginal quality per call between k=5 and k=20 : (0.968-0.780)/15 = 0.0125
marginal quality per call between k=20 and k=60: (0.989-0.968)/40 = 0.0005
A 25× drop in marginal value per call. That ratio, not the absolute numbers, is what transfers to other systems.
4.3 nDCG, and why not precision
The metric here is normalised discounted cumulative gain, which handles graded relevance (a document can be perfect, useful, or marginal — not just relevant/not) and rewards putting the best item first:
DCG@n = sum_{i=1..n} (2^grade_i - 1) / log2(i + 1)
nDCG@n = DCG@n / DCG@n of the ideal ordering
The log2(i+1) discount is why reranking shows up at all: moving a grade-3 document from position 5 to position 1 changes nothing about which documents are retrieved and still improves nDCG substantially. Plain precision@5 is blind to that reordering, and reordering is precisely what a reranker does. Use nDCG when evaluating a reranker; precision@k will under-report it.
4.4 Cost
cost per query = k * (cost per reranker call)
latency = ceil(k / concurrency) * (latency per call)
Linear in k, with no batching discount unless you pack multiple candidates per call — which trades accuracy for cost. At k=20 with an LLM reranker at ~1s per call and concurrency 5, that is ~4 seconds added to every query. This is usually the single largest latency line item in a RAG system.
5. Real code
Standard library only, deterministic, no model required. It simulates both stages with known, tunable accuracy — the point is to study the cascade's structure, which is impossible with a real model because model quality and cascade geometry get conflated.
import math
import random
CORPUS_SIZE = 60
FINAL_N = 5
N_QUERIES = 200
SEED = 0
W_RETRIEVER = 1.0 # stage-1 signal strength: score = w*grade + N(0,1)
W_RERANKER = 3.0 # stage-2 signal strength
def make_world():
"""Fixed set of queries: graded relevance + both stages' noise, per doc.
Separate RNG streams so that changing k or the reranker does not
perturb the other stage's inputs.
"""
rc, r1, r2 = random.Random(SEED), random.Random(SEED + 1), random.Random(SEED + 2)
world = []
for _ in range(N_QUERIES):
grades, n1, n2 = [], [], []
for _ in range(CORPUS_SIZE):
r = rc.random()
grades.append(3 if r < 0.02 else 2 if r < 0.07 else 1 if r < 0.17 else 0)
n1.append(r1.gauss(0, 1))
n2.append(r2.gauss(0, 1))
world.append((grades, n1, n2))
return world
WORLD = make_world()
def dcg(grades):
return sum((2 ** g - 1) / math.log2(i + 2) for i, g in enumerate(grades))
def ndcg_at_n(ordered_grades, all_grades, n):
denom = dcg(sorted(all_grades, reverse=True)[:n])
return dcg(ordered_grades[:n]) / denom if denom else 0.0
def cascade(k, w_rerank=W_RERANKER, threshold=None):
"""Mean (reranked nDCG, stage-1 nDCG, ceiling, empty-result rate)."""
re_ = s1_ = ceil_ = 0.0
empty = 0
for grades, n1, n2 in WORLD:
docs = range(CORPUS_SIZE)
ranked1 = sorted(docs, key=lambda d: -(W_RETRIEVER * grades[d] + n1[d]))
cand = ranked1[:k]
scored = [(w_rerank * grades[d] + n2[d], d) for d in cand]
if threshold is not None:
scored = [(s, d) for s, d in scored if s >= threshold]
if not scored:
empty += 1
scored.sort(key=lambda x: -x[0])
re_ += ndcg_at_n([grades[d] for _, d in scored], grades, FINAL_N)
s1_ += ndcg_at_n([grades[d] for d in ranked1], grades, FINAL_N)
# ceiling: best any reranker could do with THIS candidate set
ceil_ += ndcg_at_n(sorted((grades[d] for d in cand), reverse=True),
grades, FINAL_N)
n = len(WORLD)
return re_ / n, s1_ / n, ceil_ / n, empty / n
print("DEPTH SWEEP -- how deep should stage 1 go before reranking?")
print(f"{'k':>4} {'nDCG@5 stage1':>14} {'nDCG@5 reranked':>16} "
f"{'ceiling':>9} {'rerank calls':>13}")
print("-" * 62)
rows = {}
for k in (5, 10, 20, 30, 40, 60):
re_, s1, ceil, _ = cascade(k)
rows[k] = (re_, s1, ceil)
print(f"{k:>4} {s1:14.3f} {re_:16.3f} {ceil:9.3f} {k:13}")
print(f"\nk=5 -> k=20 : nDCG@5 {rows[5][0]:.3f} -> {rows[20][0]:.3f} "
f"(+{100*(rows[20][0]/rows[5][0]-1):.0f}%) for 4x the reranker calls")
print(f"k=20 -> k=60: nDCG@5 {rows[20][0]:.3f} -> {rows[60][0]:.3f} "
f"(+{100*(rows[60][0]/rows[20][0]-1):.0f}%) for 3x more again")
print(f"\nWhere quality is lost at k=20 (ceiling {rows[20][2]:.3f}):")
print(f" reranker error {rows[20][2]-rows[20][0]:.3f}")
print(f" stage-1 recall {1-rows[20][2]:.3f} <- the dominant term")
print("\n\nRERANKER QUALITY -- a weak reranker is worse than none")
print(f"{'w_rerank':>9} {'nDCG@5':>8} (stage 1 alone = {rows[20][1]:.3f})")
print("-" * 44)
for w in (0.0, 0.5, 1.0, 2.0, 3.0, 6.0):
re_, s1, _, _ = cascade(20, w_rerank=w)
print(f"{w:9.1f} {re_:8.3f}" + (" <- WORSE than no reranking"
if re_ < s1 else ""))
print("\n\nTHRESHOLD CUTOFF -- dropping everything below a score")
print(f"{'threshold':>10} {'nDCG@5':>8} {'empty results':>15}")
print("-" * 36)
for t in (None, 0.0, 2.0, 4.0, 6.0):
re_, _, _, empty = cascade(20, threshold=t)
print(f"{'none' if t is None else f'{t:.1f}':>10} {re_:8.3f} {empty:14.1%}")
# claims made in the prose
assert rows[20][0] > rows[5][0]
assert rows[60][0] - rows[20][0] < 0.2 * (rows[20][0] - rows[5][0])
assert (1 - rows[20][2]) > 1.5 * (rows[20][2] - rows[20][0])
assert cascade(20, w_rerank=0.0)[0] < rows[20][1]
assert cascade(20, threshold=6.0)[3] > 0.05
# stage-1 baseline must not move with k
assert len({round(v[1], 9) for v in rows.values()}) == 1
print("\nasserts passed")
# Output:
# DEPTH SWEEP -- how deep should stage 1 go before reranking?
# k nDCG@5 stage1 nDCG@5 reranked ceiling rerank calls
# --------------------------------------------------------------
# 5 0.706 0.780 0.783 5
# 10 0.706 0.916 0.922 10
# 20 0.706 0.968 0.979 20
# 30 0.706 0.981 0.992 30
# 40 0.706 0.987 0.997 40
# 60 0.706 0.989 1.000 60
#
# k=5 -> k=20 : nDCG@5 0.780 -> 0.968 (+24%) for 4x the reranker calls
# k=20 -> k=60: nDCG@5 0.968 -> 0.989 (+2%) for 3x more again
#
# Where quality is lost at k=20 (ceiling 0.979):
# reranker error 0.011
# stage-1 recall 0.021 <- the dominant term
#
#
# RERANKER QUALITY -- a weak reranker is worse than none
# w_rerank nDCG@5 (stage 1 alone = 0.706)
# --------------------------------------------
# 0.0 0.259 <- WORSE than no reranking
# 0.5 0.581 <- WORSE than no reranking
# 1.0 0.793
# 2.0 0.934
# 3.0 0.968
# 6.0 0.978
#
#
# THRESHOLD CUTOFF -- dropping everything below a score
# threshold nDCG@5 empty results
# ------------------------------------
# none 0.968 0.0%
# 0.0 0.968 0.0%
# 2.0 0.967 0.0%
# 4.0 0.918 0.5%
# 6.0 0.751 7.5%
#
# asserts passed
The reranker-quality table is the one to sit with. At w_rerank = 0.0 the second stage carries no information about relevance, and nDCG@5 collapses to 0.259 — against 0.706 for shipping stage 1 untouched. A reranker does not merely fail to help when it is bad; it shuffles a decent ordering into a random one. The crossover is between w = 0.5 and w = 1.0, i.e. exactly where the reranker becomes better than the retriever it is correcting. Below that line, reranking is negative value.
6. Real-world example
A team added an LLM reranker to a documentation search that already used a decent embedding model. Offline, on their labelled set, nDCG@5 improved from 0.71 to 0.86. They shipped it.
Search satisfaction went down.
Two things had gone wrong, and neither was visible in the offline number. First, the reranker added 3.5 seconds at p50 and over 9 at p95, because k had been set to 50 "to be safe" and concurrency was 4. Users who had been skimming ten results in three seconds were now waiting nine for five. Their labelled set measured ranking quality; it could not measure that people stopped waiting.
Second, and worse: they had also enabled a score threshold, and on the roughly one query in twelve where nothing scored above it, the system returned an empty page. Previously those queries returned mediocre-but-sometimes-useful results. An empty page reads as "this product is broken" in a way that a mediocre result does not.
The fix was almost entirely about k and the fallback. Dropping k from 50 to 15 cost 0.02 nDCG and removed two thirds of the latency. Replacing the empty page with the top result plus an honest "nothing scored highly for this query" banner recovered the rest. The reranker was fine — the cascade parameters around it were the product.
7. Interview questions companies actually ask
Q1 [easy] "Why not just use a cross-encoder for retrieval and skip the bi-encoder?"
A A cross-encoder can't precompute anything -- the document's representation
depends on the query it's paired with, which is exactly why it's accurate. So
you'd run a full forward pass per document per query: ~10ms x 10^6 docs. The
bi-encoder embeds documents once, offline, and compares with a dot product.
Four orders of magnitude apart; that gap IS the reason cascades exist.
Q2 [easy] "Your reranker isn't improving results. What do you check first?"
A Whether stage 1 recall is the binding constraint. Compute the ceiling -- the
score a PERFECT reranker would get on your candidate set. If the ceiling is
close to what you're already achieving, the reranker isn't the problem and a
better one won't help; raise k or fix the retriever. Measured above: at k=20
recall loss (0.021) was 1.9x the reranker error (0.011).
Q3 [medium] "How do you choose k?"
A Empirically, by finding the knee. Quality rises steeply then flattens while
cost stays linear. Measured: k=5->20 gave +24% for 4x the calls; k=20->60 gave
+2% for 3x more. Marginal quality per call fell 25x between those regimes. Never
inherit someone else's k -- it depends on your stage-1 quality.
Q4 [medium] "Can reranking make results worse?"
A Yes, badly. If the reranker's signal is weaker than the retriever's, it shuffles
a decent ordering into a worse one. With zero signal, nDCG@5 fell to 0.259
against 0.706 for no reranking at all. The crossover is where the reranker
becomes better than the thing it's correcting -- verify you're above it before
shipping.
Q5 [medium] "Why nDCG rather than precision@k for evaluating a reranker?"
A Precision@k is invariant to ordering within the top k -- but reordering within
the top k is exactly what a reranker does. nDCG's log2(i+1) discount rewards
putting the best document first, and its graded relevance distinguishes
"perfect" from "vaguely useful". Precision@k systematically under-reports
reranker value.
Q6 [medium] "What does an LLM reranker give you that a cross-encoder doesn't?"
A An editable rubric (change the prompt, change the ranking -- no retraining);
structured labels emitted alongside the score in the same call, nearly free
since you're paying for the pass anyway; and a justification string that makes
ranking auditable. You pay in latency (seconds vs milliseconds), price, parse
failures, and score drift across model versions.
Q7 [hard] "You threshold reranker scores to filter irrelevant results. What breaks?"
A Hard queries, where EVERY candidate scores low and you return nothing. Measured:
threshold 6.0 gave empty results on 7.5% of queries and dropped mean nDCG@5 from
0.968 to 0.751. Thresholds interact with query difficulty, not just document
quality. Always define the fallback -- best-effort result plus an honest banner
beats an empty page.
Q8 [hard] "Would you batch multiple candidates into one reranker call?"
A It cuts call count roughly linearly, which matters when you're rate-limited or
paying per call. The cost is that candidates in one context get scored relative
to EACH OTHER rather than against the rubric, so scores stop being comparable
across batches -- which breaks any absolute threshold. Batch when you need
ordering within a page; don't batch when you need calibrated scores.
8. When to use / tradeoffs
REACH FOR RERANKING WHEN:
+ stage 1 finds the right documents but orders them badly (the common case)
+ your context budget is tight -- fewer, better chunks beat more, worse ones
+ you need auditable relevance decisions (LLM reranker with justifications)
+ irrelevant context is measurably degrading generation quality
SKIP IT WHEN:
- stage-1 ordering is already good (compute the ceiling and check)
- stage-1 recall is terrible -- you'd be polishing the wrong candidate set
- latency budget is under ~1s end to end
- your reranker is not clearly better than your retriever
| Situation | Why it breaks | Use instead |
|---|---|---|
| Ceiling ≈ achieved score | Reranker is already near-optimal on this set | Raise k, or improve stage 1 |
| Reranker weaker than retriever | Shuffles good ordering into worse — 0.259 vs 0.706 | Ship stage 1 alone |
Very large k "to be safe" | Cost and latency linear in k, quality flat past the knee | Find the knee; sit on it |
| Hard threshold on scores | 7.5% empty results at threshold 6.0 | Threshold + explicit fallback |
| Candidates from several retrievers | Rescoring all of them is wasteful | Fuse with RRF first, then rerank top k |
| Need calibrated absolute scores | Batching makes scores relative to the batch | One candidate per call |
Honest limits. Both stages here are simulated as w * grade + Gaussian noise. That is a deliberate choice — it isolates cascade geometry from model quality — but it is also a strong assumption: real rerankers have correlated, systematic errors (they consistently over-rank long chunks, or documents echoing the query's phrasing), not independent noise. Correlated error makes the ceiling harder to approach than shown here, and it means a reranker's mistakes cluster on particular query types rather than spreading evenly. The graded-relevance distribution (2% grade-3, 5% grade-2, 10% grade-1) is invented; corpora with far fewer relevant documents per query will see a sharper knee and a higher optimal k. The absolute nDCG values are therefore meaningless outside this simulation — the transferable results are the shapes: diminishing returns in k, the ceiling decomposition, and the sign flip when the reranker is weaker than the retriever. Finally, latency here is a stated model rather than a measurement; measure your own p95, since that is what users experience.
9. Summary + related articles
- Retrieval is a cascade: cheap scorer for recall over everything, expensive scorer for precision over
kcandidates. The split exists purely because the good scorer does not scale. - Depth
kis the knob. Measured:k=5→20gave +24% nDCG@5 for 4× the calls;k=20→60gave +2% for 3× more. Marginal value per call fell 25×. Find the knee. - Decompose the loss into
(1 − ceiling)and(ceiling − achieved). Atk=20, recall loss was 1.9× the reranker error — so the leverage was in stage 1, not in a better reranker. - A weak reranker is negative value, not zero value: 0.259 vs 0.706 for no reranking. Verify your second stage beats your first before shipping it.
- Thresholds cause empty results on hard queries — 7.5% at threshold 6.0. Always define the fallback.
- Evaluate with nDCG, not precision@k, which is blind to the reordering a reranker performs.
- Boundary: reranking pays only when stage 1 finds the right documents but orders them badly. It cannot recover what stage 1 never returned, and it cannot help what stage 1 already got right.
Related:
- Query Transformation: Fixing the Question Before You Retrieve — the other lever on the same cascade: §4.3 derives the RRF formula referenced here, and the ceiling this article measures is set by the query stage 1 receives
- Chunk Size, Overlap, and Retrieval Thresholds in RAG Systems — what the reranker is ordering, and why chunk boundaries cap the ceiling too
- Search Systems — cascades in general web-scale search, where the pattern originated
- Hallucination Detection & Grounding — LLM-as-judge scoring, the same machinery pointed at generated text instead of retrieved passages
- Backtesting, Baselines & Sensitivity Analysis — why "reranking improved nDCG offline" did not mean the product improved
Resources
- Nogueira & Cho, "Passage Re-ranking with BERT", arXiv:1901.04085, 2019 — the paper that established the BERT cross-encoder reranking cascade.
- Järvelin & Kekäläinen, "Cumulated Gain-Based Evaluation of IR Techniques", ACM Transactions on Information Systems 20(4), 2002 — the original definition of DCG and nDCG.
- Cormack, Clarke & Buettcher, "Reciprocal Rank Fusion Outperforms Condorcet and Individual Rank Learning Methods", SIGIR 2009 — RRF, the fusion alternative to rescoring.
- Reimers & Gurevych, "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", EMNLP 2019 — the bi-encoder side of the comparison, and §1 states the cost argument for cascades directly.
- Xiao et al., "C-Pack: Packed Resources For General Chinese Embeddings", arXiv:2309.07597 — the BGE embedding and reranker family. Reranker-specific evaluation details not independently verified.
- Cohere, "Rerank" — https://cohere.com/rerank (hosted cross-encoder reranking API).