Vector search finds passages whose meaning is close to a question, even when the wording differs. It can miss when a question depends on an exact string such as an error code, a function name, a part number, or an unusual product term. BM25 is the classic lexical ranking method for that job, and it is the default text similarity in Elasticsearch. In a retrieval-augmented generation (RAG) pipeline, the practical answer is rarely to pick one retriever for everything. It is to run lexical and semantic retrieval side by side and measure which combination surfaces the passages your language model needs.
What BM25 scores
BM25, also called Okapi BM25, ranks documents by how well their words match the query. It uses term statistics and collection statistics to do this. It is a ranking function, not a language model, and it does not infer that two differently worded passages mean the same thing. Elasticsearch identifies Okapi BM25 as its default text similarity, and its similarity reference describes how the setting is applied at the field level.
As an Amazon Associate I earn from qualifying purchases.
The textbook treatment in Introduction to Information Retrieval by Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze states the design goal this way: “The BM25 weighting scheme, often called Okapi weighting, after the system in which it was first implemented, was developed as a way of building a probabilistic model sensitive to these quantities while not introducing too many additional parameters into the model.” The “quantities” in that sentence are term frequency and document length, which the chapter discusses in detail (see the BM25 chapter).
Free tools Windows power users keep installed
One-click scans. No signup required.
The three signals inside the score
- Query-term frequency. A document that contains a query term more often scores higher for that term, but the gain flattens out. The tenth occurrence of a word adds much less than the first.
- Inverse document frequency. A term that appears in only a few documents in the collection contributes more than a term that appears almost everywhere. A rare error code therefore outweighs common words such as “the” or “error”.
- Document-length scaling. A term in a short passage counts for more than the same term buried in a long one, measured relative to the average passage length in the collection.
The formula
For a query Q with terms qi and a document D, the classic formulation is:
#1 Best Overall
score(D, Q) = sum over i of IDF(q_i) * ( f(q_i, D) * (k1 + 1) )
/ ( f(q_i, D) + k1 * (1 - b + b * |D| / avgdl) )Here f(qi, D) is the frequency of the term in the document, |D| is the document length, avgdl is the average document length in the collection, and k1 and b are tunable parameters. The k1 parameter controls how quickly repeated occurrences saturate, and b controls how strongly length is normalised. Elasticsearch exposes these parameters in its similarity settings, so check the reference above for the exact names and current defaults rather than relying on values copied from older articles.
Where lexical and semantic retrieval diverge
The two approaches fail in different places. The table below describes typical behaviour for the question types that matter most in RAG. It is a diagnostic guide, not a measured ranking, and your own corpus may behave differently.
| Question type | BM25 (lexical) | Vector (semantic) | Hybrid (both candidate lists) |
|---|---|---|---|
| Exact error code, such as a numeric or alphanumeric code copied from a log | Usually finds the passage that contains the exact token, provided the tokenizer keeps the code intact | May return general troubleshooting passages that are similar in topic but lack the code | Keeps the exact-token match in the candidate set alongside semantically related passages |
| Function, class, or flag name, such as a configuration option | Strong when the name is indexed as written; sensitive to how the analyzer splits punctuation | May blur near-identical names into one cluster | Keeps the literal name match while still surfacing explanatory text |
| Paraphrase, such as “why does my upload stall” when the manual says “transfer timeout” | May miss the passage if no query terms appear in it | Designed to match related meaning across different wording | Lets the semantic list supply the paraphrased passage |
| Ordinary natural-language question with synonyms and some shared terms | Often performs well when the question shares key nouns with the passage | Often performs well when wording varies | Depends on how the two lists are merged; must be tested |
Why RAG needs both signals
In RAG, retrieval decides what the generator sees. If the passage that answers the question never reaches the prompt, the model either answers from the wrong context or says it cannot find the answer. Lexical retrieval protects against the first case when identifiers matter. Semantic retrieval protects against the second when users phrase things differently from the documentation.
Do these 3 things before closing this tab:
1Fix the driver behind crashes, sound loss and screen glitches2Repair Windows errors before they cause bigger problems3Scan for outdated or missing drivers - takes under a minuteConsider a hypothetical support corpus. A user pastes the code NET-4012 and asks what it means. A vector index may return passages about network timeouts in general, because the meaning is close. A BM25 index matches the literal token and returns the page that defines NET-4012. In the reverse case, a user asks why a file never finishes uploading, and the manual uses the phrase “transfer timeout” throughout. BM25 may miss the page entirely, while vector retrieval can find it.
Rank #3
Combining the two signals
Hybrid retrieval runs both methods and merges their results. The merge step matters, because a BM25 score and a vector similarity score are not on the same scale. Elastic documents two main ways to combine lexical and vector results, described in its ranking documentation.
Reciprocal Rank Fusion (RRF)
RRF merges the ranked lists using each passage’s position in each list, not its raw score. A passage that ranks well in either list rises in the combined order, and passages that appear high in both rise further. Because it works on ranks, it avoids having to normalise raw scores to a common scale. It is a reasonable first configuration to test when the two score distributions are hard to align.
Rank #4
Linear or convex weighting
Weighted combination scores each candidate as a weighted sum of its lexical and vector scores, often after normalising both. It gives direct control over how much the exact-match signal counts relative to meaning. The cost is that the weights have to be tuned, and the best weights for one corpus may not transfer to another.
Recommended Free Tools
Elastic’s query documentation describes BM25, vector search, hybrid score combination, and reranking as query options. The Elastic hybrid search overview explains the lexical, vector, and hybrid cases in search and RAG terms. OpenSearch documents the same pattern of default BM25 scoring combined with semantic search in its neural and hybrid search tutorial.
Best Value
- Used Book in Good Condition
How to test the options on your own corpus
- Collect real questions from users, support tickets, or logs. Include exact identifiers and rare terms, paraphrased questions, and ordinary natural-language questions, and keep the mix roughly representative of traffic.
- For each question, label the passages that answer it. A few labelled passages per question is enough to score retrieval.
- Index the same chunks three ways: BM25 only, vector only, and hybrid. Keep chunking, the embedding model, and the generator constant so that only the retrieval method changes.
- Measure whether the labelled passages appear in the top results that the generator actually receives.
- Generate answers from each configuration and check whether every claim in the answer is supported by the retrieved passages. A correct-looking answer from the wrong context should count as a failure for this test.
- Review failures by question type rather than by average score. A hybrid configuration that helps identifier queries but hurts paraphrases should show up clearly in this breakdown.
- Tune fusion settings or weights only after the baseline comparison. Hold back a subset of questions that you never use for tuning, so the final numbers are not inflated.
What the evidence does and does not establish
The official documentation describes what each platform can do. It does not publish a benchmark that compares BM25, vector, and hybrid retrieval for RAG on a dataset comparable to yours, so there is no universal winner and no performance percentage to quote. The textbook chapter explains the model; it does not measure your application. Treat hybrid retrieval as a design option that must be validated with the procedure above, not as a guaranteed improvement.
Further reading
The book Introduction to Information Retrieval by Manning, Raghavan, and Schütze is published by Cambridge University Press (2008, print ISBN 0521865719). It is optional reading, since the authors also provide online editions on the companion site, and the BM25 chapter linked above is enough to follow the model.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
