Cross-Encoder Reranking

~22 min · hybrid, reranking

Level 0Scout

0 XP0/41 lessons0/10 achievements

0/120 XP to next level120 XP to go0% complete

Two-stage retrieval

Vector search and BM25 are bi-encoders — they encode the query and each document independently, then compare the results. Fast, but the model never sees the query and document together.

A cross-encoder takes the query and a candidate document and runs them jointly through a transformer. It produces a single relevance score that reflects pairwise context. Much more accurate, much slower (you cannot precompute it), so the standard architecture is:

Stage 1 — fast retriever (vector + BM25) returns top-50 candidates.
Stage 2 — cross-encoder reranks those 50 to find the true top-5.

Open-source cross-encoders worth knowing

BGE-reranker-v2-m3 — multilingual, best of the open-source pack in early 2026
cross-encoder/ms-marco-MiniLM-L-6-v2 — tiny English, runs on CPU
jina-reranker-v2-base-multilingual — fast, multilingual, very small

Code

Rerank top-50 with sentence-transformers CrossEncoder·python

from sentence_transformers import CrossEncoder

reranker = CrossEncoder('BAAI/bge-reranker-v2-m3', max_length=512)

def rerank(query: str, candidates: list[dict], top_k: int = 5) -> list[dict]:
    pairs = [(query, c['text']) for c in candidates]
    scores = reranker.predict(pairs, batch_size=16)
    for c, s in zip(candidates, scores):
        c['rerank_score'] = float(s)
    candidates.sort(key=lambda c: -c['rerank_score'])
    return candidates[:top_k]

stage1 = vector_then_bm25(query, k=50)
final  = rerank(query, stage1, top_k=5)

Hosted reranker (Cohere)·python

import cohere
co = cohere.Client()

resp = co.rerank(
    model='rerank-english-v3.0',
    query=query,
    documents=[c['text'] for c in stage1],
    top_n=5,
)
for r in resp.results:
    print(r.relevance_score, stage1[r.index]['text'][:80])

External links

Exercise

Take 30 queries with manually labeled top-3 ground truth. Compare nDCG@5 for: vector only, vector + BM25 fused with RRF, RRF + cross-encoder rerank. The biggest jump usually comes from adding the reranker — measure how big.