NOTES / RAG AND RETRIEVAL

BM25 and sparse retrieval

Keyword search done well: inverted indexes, BM25 scoring, and why this 1990s method is still a strong baseline and a vital partner to embeddings.

UPDATED 18 NOV 2026#bm25#sparse-retrieval#keyword-search

Sparse versus dense

Sparse retrieval represents text by the words it contains, so a document is a very long, mostly empty vector over the vocabulary. Dense retrieval, covered in Embeddings and vector search, uses short, learned vectors. Open-domain QA retrieval was long dominated by TF-IDF and BM25, which match keywords efficiently with an inverted index3.

Diagram: an inverted index mapping terms to the documents that contain them

Diagram: my own.

An inverted index lists, for each term, the documents containing it. Query terms are looked up and the matching documents are scored. That is fast and needs no model.

What BM25 does

BM25 is the best-known instance of the probabilistic relevance framework that Robertson and Zaragoza describe, and one of the most successful text-retrieval scoring functions1. In plain terms it rewards documents that contain the query’s terms, with diminishing returns for repeats, gives more weight to rarer terms, and adjusts for document length. It has tunable parameters, which the monograph also discusses1.

Why it still matters

  • A tough baseline. The BEIR benchmark found BM25 to be a robust baseline across diverse datasets. Re-ranking and late-interaction models did best on average in zero-shot tests but at much higher computational cost2.
  • Exact strings. Error codes, product IDs, names, citations and rare jargon are things embeddings blur and keywords nail. Anthropic’s example is a query for a specific error code, where BM25 looks for that exact string4.
  • No training, easy to debug. You can see why a document matched.
  • Complements dense. Dense retrieval handles paraphrase; the DPR authors give “bad guy” against “villain” as a case where keyword matching struggles3.

Diagram: an exact code query suits keyword search; a paraphrased query suits embedding search

Diagram: my own.

Practical notes

  • Tokenise sensibly. Lowercasing, stemming and stop words change recall; code and identifiers may need special handling.
  • Index useful fields. Titles and headings often deserve extra weight, which the field-weighted variant BM25F was designed for1.
  • Use a real engine. Search servers and libraries implement BM25 well; avoid hand-rolling it.
  • Add context to chunks. Anthropic prepends chunk-specific context before building the BM25 index as well as the embeddings, which helped in their tests4.
  • Keep it in the mix. Even if you go mostly dense, keeping a sparse index around is cheap insurance.

Next: Hybrid search and reranking.

Sources

  1. [1]The Probabilistic Relevance Framework: BM25 and Beyond — Stephen Robertson, Hugo Zaragoza · Foundations and Trends in Information Retrieval · 2009PARAPHRASED
  2. [2]BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models — Nandan Thakur and co-authors · arXiv, NeurIPS 2021 · 2021PARAPHRASED
  3. [3]Dense Passage Retrieval for Open-Domain Question Answering — Vladimir Karpukhin and co-authors · arXiv, EMNLP 2020 · 2020PARAPHRASED
  4. [4]Introducing Contextual Retrieval — Anthropic · 2024PARAPHRASED

© Ashish Kumar. All rights reserved. Quoted and cited material belongs to its owners. How I attribute sources.

Product, company and framework names are trademarks of their respective owners. Mentioning them does not imply affiliation or endorsement.