Retrieval gives an LLM candidate documents; it does not decide which candidates actually support an answer, which passages matter, or whether the answer is present at all. Those are separate post-retrieval decisions. Reranking changes order, filtering changes membership, compression selects text within a document, and deduplication checks whether a candidate adds evidence already selected. Choose the operation that matches the problem—and treat every result as limited by what retrieval found.
What is still undecided after retrieval?
Consider the query “how long are logs retained?” The sentence “This section explains the log retention period” is on topic but supplies no duration. “Logs are retained for 30 days” contains a direct answer, while “Audited logs are kept for one year” adds an exception. These are illustrative examples, not retention guidance.
As an Amazon Associate I earn from qualifying purchases.
A retrieval system can return all three kinds of text. It takes further judgments to decide what to read first, what to keep, what to extract, and whether another result repeats existing evidence. These operations are not interchangeable: one can improve ordering without removing anything, while another can remove an apparently relevant result because it contains no usable evidence.
Free tools Windows power users keep installed
One-click scans. No signup required.
How do reranking, filtering, compression, and deduplication differ?
| Operation | Question it answers | What changes | Best fit and main caveat |
|---|---|---|---|
| Rerank | How related is each candidate to the question? | Candidate order. | Use when the answer may already be in the set but is not near the top. It cannot add missing information, and a heading may outrank a concrete answer if the judgment is relevance alone. |
| Filter | Does this candidate contain concrete information usable to answer? | Which candidates survive. | Use to remove on-topic but empty results. A threshold is a keep-or-drop rule, not a ranking. Filtering only after a top-N cutoff can miss useful candidates excluded by that earlier cutoff. |
| Compress | Which sentences or lines in a long document matter to this question? | Selected original text within a document. | Use to reduce downstream context. Dropping a condition, exception, or referent can change meaning, and the selection may choose the wrong text units. |
| Deduplicate | Does this candidate add evidence beyond what has already been selected? | Redundant candidates are removed. | Consider when a corpus is dominated by reposts or paraphrases. In the described exploration, deduplication was not shipped because its additional judgments did not produce a worthwhile gain. |
The practical distinction is both order versus membership and document versus passage. Reranking changes what appears first; filtering changes what remains. Filtering evaluates candidates as a whole, while compression selects portions of their contents. Deduplication compares candidates against evidence already retained and can require many more judgments.
#1 Best Overall
What did the reported evidence-filtering experiment find?
For an exploratory test, Shinsuke Kagawa labeled 220 candidates across 11 deliberately difficult queries for whether they contained evidence. With no more than five candidates per query and a filter threshold of 0.5, the reported comparison was:
| Method | Results returned | Results judged to contain clear evidence |
|---|---|---|
| Plain relevance reranking | 55 | 22 |
| Evidence filtering in input order | 39 | 20 |
| Sort by evidence score, then filter | 39 | 29 |
The score-sorted variant retained the most clear-evidence candidates in this comparison. The implementation described as shipped preserves input order instead, respecting the retriever’s ranking but retaining fewer evidence-bearing candidates in this run. Sorting by evidence score makes that score govern both selection and order. The labels were generated by Codex before it saw Jev’s scores; they are not multi-annotator ground truth, so the counts are exploratory rather than a general performance guarantee. See Kagawa’s article for the experiment and its context.
What did the compression test establish—and what did it not?
On 40 answerable SQuAD 2.0 questions, the prototype reduced 31,440 characters of source text to 8,290 and preserved the published answer span in 38 cases. This is a character reduction and answer-span preservation result, not a token count or an end-answer accuracy score. It also does not show that every surrounding condition survived.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Fix the driver behind crashes, sound loss and screen glitches3Repair Windows errors before they cause bigger problemsThe reported failures illustrate why a compact extract needs scrutiny: one necessary sentence scored too low and was dropped; in another case, a split after a person’s initial meant the full name did not survive. Keeping the original text alongside the extract makes such losses inspectable. If reducing context is the goal, downstream processing can use the compressed field while the original remains available for checking.
Can reranking improve a retrieval result?
Kagawa’s retrieval comparison used mcp-local-rag over 59 arXiv papers (27,563 chunks), with 36 queries and 20 retrieved candidates per query. Jev reranking changed the top result for 31 of 36 queries and replaced an average of 2.92 items in the top five. Fusion with retriever distance changed the top result for 8 of 36 and replaced an average of 1.08 top-five items.
Those figures measure changes in ordering, not improvement by themselves. Independent language-model evaluators assessed answer-supporting candidates on smaller, different query subsets; their average counts favored Jev reranking over retriever-only results in those samples. One query produced disagreement about source diversity. The author observed that headings, titles, figure captions, and bibliography lines often matched query terms without carrying answer information. Latency and cost were not measured. The account and its qualifications are in the article’s retrieval comparison.
The project README separately reports BM25 reranking results on three BEIR datasets:
The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →| Dataset | BM25 nDCG@10 before reranking | Project-reported nDCG@10 after reranking |
|---|---|---|
| SciFact | 0.68 | 0.76–0.77 |
| NFCorpus | 0.27 | 0.33 |
| FiQA | 0.24 | 0.36–0.37 |
These are project-reported benchmark values, distinct from the exploratory tests above. The README discusses setup, candidate depth, run-to-run variation, and limitations; the figures are not an independent replication or a promise for a particular corpus. See the jev-reranker README.
Best Value
How should you apply these operations in a pipeline?
- Check candidate coverage first. Confirm that the retrieved set contains material capable of answering the question. A reranker or filter cannot recover evidence that is absent.
- Choose ordering or removal deliberately. If useful evidence may be buried, rerank. If candidates are on topic but empty, filter. Do not mistake a filter threshold for a relevance score or assume filtering will preserve the retriever’s top results.
- Use compression only where document length warrants it. Select original sentences or lines, then retain the source text for inspection. Verify conditions, exceptions, names, and other context that could be lost at unit boundaries.
- Evaluate on the intended task. Separate order changes, evidence-bearing candidate counts, character reduction, and answer quality. Compare with the same corpus, query set, candidate depth, and evaluation method; none of those measures can stand in for all the others.
- Check privacy, latency, and cost boundaries. The described compression process judges sentence or line units with the full parent document available; long documents may need multiple batches, and the full text is sent again with each batch. The article says the question and selected text go to an external API. A local retrieval system therefore does not itself ensure local post-processing.
What are the implementation limits?
The described filter keeps candidates with evidence scores of at least 0.5 by default, preserves their input order, and can be configured with a different threshold. If no candidate meets the threshold, it returns an empty result rather than backfilling. Treat 0.5 as a starting point to tune against your own data. Filtering for body evidence is also a poor fit when the desired result is itself a title or citation, as in some searches for papers.
Compression judges sentence or line units with the full parent document available to help preserve referents and conditions, then extracts the original wording rather than rewriting it. Longer documents can require multiple batches, with the full text sent in each batch; context saved downstream must be weighed against selection cost and latency. The described Jev API receives the question and selected text, so using local retrieval alone does not guarantee that post-retrieval processing stays on-device or within your environment. Implementation details are described in the article.
Why does evidence in the results still not prove the question is answerable?
Reranking and filtering operate only on retrieved candidates. Even if one or more passages survive as evidence, a compound question may ask for multiple facts and have support for only some of them. The caller still needs to check which parts remain unanswered, search again when appropriate, or state what is missing rather than treating a full candidate list as proof of a complete answer. The article’s concise warning is: “Evidence surviving is also not the same as the whole question being answerable.” (Shinsuke Kagawa)
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
