A result that looks most useful to you lands at the bottom of a Whoosh search because Whoosh orders hits by a computed score, not by human judgment. That score comes from which fields matched, which terms matched, and how much each one counts. Nothing in the engine knows what “best” means for your users. Tuning relevance means translating your ranking intent into those inputs, then checking the new order against queries you actually care about.
How Whoosh decides the order
Whoosh normally sorts results by score, and its documented default weighting model is BM25F. BM25F scores each matching term by how often it appears in a field, adjusted for how long that field is, and then combines the field contributions. Two consequences follow. A long body field with many repetitions of a term can outscore a short title that matches the query once, unless you tell the engine that title matches matter more. And a document can rank low even when it is the best answer, simply because its matching terms are weak signals in the fields the engine weighs.
As an Amazon Associate I earn from qualifying purchases.
Find the cause before changing weights
Most “best result is last” reports fall into one of a few patterns. Match them to the cause first, because each one has a different fix.
- The term matches only in the body. The engine has no reason to prefer the title or heading match you expected. Fix with a field boost (see below).
- The result is wrong on identifiers, paths, or codes. The field is probably analyzed as free text, so tokenization or stemming changes what matches. Check the field type.
- The query means something different from what you typed. The parser may split, drop, or reinterpret part of the query. Print the parsed query.
- Only one query shape fails. The relevance signal you need depends on the query, so a global weight will not fix it. Use a query-time boost.
- A boost fixed one query and broke others. The value is too strong for your corpus. Revisit it with a test set.
Step 1: confirm the query parses as expected
- Build the parser for your default field and schema:
from whoosh.qparser import QueryParserthenparser = QueryParser("body", ix.schema), whereixis your open index. - Parse the failing query text and print the result:
q = parser.parse(u"your failing query")thenprint(q). - Confirm each term is attached to the field you intended. If a term lands in the default field when you meant the title, the query needs an explicit field prefix such as
title:. - Confirm the field exists in the schema with the type you expect. A query against a field that is not indexed will not match what you think it matches.
Step 2: use the right field types
Whoosh distinguishes free text from exact values. TEXT fields are analyzed for searching. Identifiers, slugs, and paths should usually use ID, which treats each value as a single unit rather than as words to stem and match loosely. Using TEXT for a path can make a fragment of that path match unrelated documents and push the correct one down.
#1 Best Overall
from whoosh.fields import Schema, TEXT, ID
schema = Schema(
title=TEXT(field_boost=2.0),
body=TEXT,
path=ID(stored=True),
)The title boost of 2.0 comes from the Whoosh schema documentation as an illustration of the mechanism. It is not a recommended value for every index.
Step 3: boost fields that should count more
A field boost tells the engine that a match in that field carries more weight everywhere it occurs. This is the right tool when importance is a property of the field itself, such as titles, tags, or headings matching more meaningfully than body text. Set it in the schema with field_boost on a TEXT field.
The trade-off is that a boost applies to every query. A high value can let a document that merely mentions the term in its title beat a thorough article where the term appears throughout the body. Raise the value in small steps and check the effect on your test set rather than picking a number by feel.
Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Clear out junk files and repair common Windows errorsFree Scan →Step 4: adjust BM25F term-frequency and length behavior
If field boosts alone do not produce the order you need, the BM25F weighting model has two parameters you can change:
Rank #3
- Python Data Science Handbook
- B controls how strongly field length normalizes scores. A higher value penalizes long fields more.
- K1 controls how quickly repeated occurrences of a term stop adding to the score.
The Whoosh BM25F documentation for version 2.7.4 lists the defaults as B=0.75 and K1=1.2. These are library defaults, not measured recommendations for any particular index. The same documentation supports per-field B values, which lets you normalize a short title field differently from a long body field.
from whoosh import scoring
searcher = ix.searcher(weighting=scoring.BM25F(B=0.75, K1=1.2))BM25F depends on field-length information being present in the index. If that information is missing or stale, length normalization will not behave as the model describes, and changing B will look inconsistent. Confirm your index was built with the field types above before tuning these values.
Step 5: use query-time boosts for query-specific intent
Some relevance rules depend on what the user typed, not on the field. Whoosh query syntax lets you boost an individual term or a grouped element. The documented examples are ninja^2 and (open sesame)^2.5. Use this when, for example, a query for an installation error should favor documents about that error over general installation guides, while other queries should not be affected.
Because the boost travels with the query string, it applies only to searches that include it. That makes it the safest of the controls for narrow fixes, but it also means you must maintain the boost wherever queries are built.
Best Value
Step 6: consider a custom weighting function only as a last resort
The scoring API documents FunctionWeighting, and a weighting model can produce scorer instances, so you can encode rules that field and query boosts cannot express. A typical case is a rule that combines a document attribute with the text score. Treat this as application code: it must be versioned, tested against the same query set as every other change, and revisited when the library or your schema changes.
Comparing the tuning controls
| Control | Where it applies | What it changes | Best used when | Main risk |
|---|---|---|---|---|
Schema field boost (field_boost) |
Every query against that field | Weight of matches in the field | A field is broadly more important, such as titles | Over-weighting one field across all queries |
BM25F B |
Scoring model, per searcher or per field | Penalty for long fields | Long documents outrank shorter, better matches | Changes ordering everywhere; sensitive to field-length data |
BM25F K1 |
Scoring model | Diminishing returns on repeated terms | Repetition in body text inflates scores | Changes ordering everywhere; no universal value |
Query-time boost (^) |
Only the query that contains it | Weight of a term or grouped element | Importance depends on the search itself | Must be maintained wherever queries are written |
Custom FunctionWeighting |
Scoring logic in application code | Arbitrary scoring rule | No standard control expresses the rule | Highest maintenance and testing cost |
Apply the simplest control that matches the rule. Schema boosts and query boosts are easier to reason about than changing the scoring model, and a scoring model change affects every query in the index.
How to test a change
A ranking change is only an improvement if it works on your corpus. Test it this way:
- Collect a set of representative queries from real searches or support logs. Include queries where the current order is correct, so you can catch regressions.
- For each query, record the document you expect at or near the top, and the position where it currently appears.
- Run the set against the current configuration and save the ranked results.
- Make one change at a time, rebuild or reopen the index as needed, and run the same set again.
- Compare the target positions and check whether any previously correct result moved down. Keep the change only if the net effect is better and no important query regresses.
Judgments of relevance are yours to make. The engine’s score is an input to that judgment, not a measure of it.
Quick Recap
Check these before you change anything
- The Whoosh API references behind this guide are for version 2.7.4. Confirm the version installed in your application, because parameter names and defaults can differ in other releases.
- The sources behind this guide do not establish Whoosh’s current maintenance status. If you are starting a new project, check the project’s own release history before committing to it.
- No boost value, B setting, or K1 setting is universally correct. A value that fixes one index can distort another.
- Your index was not inspected for this guide. The steps above describe how to diagnose and test your own corpus, not a known outcome for it.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
