BM25 Isn't Dead — Pair It With Vectors

Why pure vector search loses some queries

Embeddings are great at meaning but mediocre at tokens. They will not reliably rank a chunk that contains the exact error code 'ERR_CONNECTION_REFUSED' above a chunk that paraphrases the concept. Product names, SKUs, version numbers, and rare technical terms tend to live in the surface form — exactly what BM25 (a 1994-vintage probabilistic keyword scorer) is built for.

What BM25 actually does

For each query term, BM25 scores a document by term frequency (more occurrences = higher), inverse document frequency (rare terms count more), and a length normalization. The result is a ranking that loves exact matches, punishes verbosity, and ignores meaning. The opposite failure mode of vector search.

The cheapest hybrid

Run vector search → top-N candidates.
Run BM25 → top-N candidates.
Merge ids, take the union, then re-rank by either a fusion score or a cross-encoder.

Two retrievers, one merge step. That alone often beats vector-only retrieval by 10–20% on factoid queries.

Code

BM25 with rank-bm25 (pure Python)·python

from rank_bm25 import BM25Okapi

corpus = [doc.lower().split() for doc in documents]
bm25 = BM25Okapi(corpus)

def bm25_search(query: str, k: int = 10):
    scores = bm25.get_scores(query.lower().split())
    top = scores.argsort()[::-1][:k]
    return [(int(i), float(scores[i])) for i in top]

Hybrid in Postgres (pgvector + tsvector)·sql

-- Add a full-text column once
ALTER TABLE chunks ADD COLUMN tsv tsvector
    GENERATED ALWAYS AS (to_tsvector('english', text)) STORED;
CREATE INDEX chunks_tsv_idx ON chunks USING gin(tsv);

-- Hybrid query: union both rankings then re-rank
WITH vec AS (
    SELECT id, 1 - (embedding <=> $1) AS vscore
    FROM   chunks
    ORDER BY embedding <=> $1
    LIMIT 50
),
bm AS (
    SELECT id, ts_rank_cd(tsv, plainto_tsquery('english', $2)) AS bscore
    FROM   chunks
    WHERE  tsv @@ plainto_tsquery('english', $2)
    ORDER BY bscore DESC
    LIMIT 50
)
SELECT  c.id,
        COALESCE(vec.vscore, 0) AS vscore,
        COALESCE(bm.bscore,  0) AS bscore,
        c.text
FROM    chunks c
LEFT JOIN vec ON vec.id = c.id
LEFT JOIN bm  ON bm.id  = c.id
WHERE   vec.id IS NOT NULL OR bm.id IS NOT NULL;

BM25 Isn't Dead — Pair It With Vectors

Why pure vector search loses some queries

What BM25 actually does

The cheapest hybrid

Code

External links

Exercise

Progress

Comments 0