For a production recommender, keep data and features, candidate retrieval, ranking, and feedback and experimentation as distinct layers. An LLM can help retrieve or rerank items, but it does not make the catalog, behavioral signals, evaluation, or improvement loop unnecessary. “Only touches two” is a useful design boundary—not a universal rule about what LLMs can do.
What are the four layers, and where can an LLM help?
The four-layer framing is useful because each stage has a different job and a different way to fail. A recommender typically narrows a large catalog to candidates, orders those candidates for a user, and learns from what happens after they are shown. The title’s original article argues for keeping those responsibilities separate; the table below makes the LLM’s possible role explicit.
| # | Preview | Product | Price | |
|---|---|---|---|---|
| 1 |
|
Recommender Systems: The Textbook | $54.99 | Buy on Amazon |
| 2 |
|
Practical Recommender Systems | $49.99 | Buy on Amazon |
| 3 |
|
Building Recommendation Systems in Python and JAX: Hands-On Production Systems at Scale | $48.49 | Buy on Amazon |
| 4 |
|
Recommendation Systems in Software Engineering | $131.98 | Buy on Amazon |
As an Amazon Associate I earn from qualifying purchases.
| Layer | Primary responsibility | Possible LLM contribution |
|---|---|---|
| Data and features | Maintain user events, item information, and collaborative signals used by the system. | May help interpret or enrich text, but does not replace reliable source data or event collection. |
| Candidate retrieval | Find a manageable set of items that could suit the user. | May support semantic retrieval or query understanding; retrieval still needs an index, candidate sources, and recall measurement. |
| Ranking | Score and order the retrieved items for the current context. | May rerank candidates using detailed or multifactor relevance judgments. |
| Feedback and experimentation | Record exposures and outcomes, evaluate changes, and improve features and models. | May help analyze data, but does not replace trustworthy logging or controlled evaluation. |
What belongs in each layer?
1. Data and features: make the inputs dependable
Recommendations depend on item metadata, user and session events, and collaborative signals such as patterns across users and items. Decide which events count as meaningful outcomes, how item and user identifiers are maintained, and how fresh features need to be. If these inputs are missing, stale, or inconsistent, a more capable ranker cannot reliably recover the information the system never captured.
For a new or sparsely used item, collaborative evidence may be limited. Item attributes and context can help the system make an initial decision; this is one form of the cold start problem. An LLM can interpret descriptive text, but it should not be treated as a substitute for real user behavior when that behavior becomes available.
#1 Best Overall
2. Candidate retrieval: find enough good options
Retrieval searches a large catalog and returns a smaller candidate pool for ranking. It may combine sources such as collaborative recommendations, filters, keyword matching, or vector similarity. The central question is whether relevant items make it into the pool at all: measure recall@k for retrieval, the share of relevant items found within the top k retrieved candidates.
Semantic retrieval can help when meaning matters more than exact word overlap, but it is not the only way to generate candidates. A practical system can combine retrieval methods and preserve their source labels so teams can see which methods supplied useful items. Microsoft’s guidance for retrieval-augmented generation (RAG) describes hybrid full-text and vector retrieval and combining result orderings with Reciprocal Rank Fusion (RRF). Those are useful retrieval techniques, but RAG and recommender systems are different settings with different data and objectives. See the Microsoft Azure Architecture Center guide to RAG information retrieval.
3. Ranking: order the candidates for the user
Ranking evaluates the retrieved pool more precisely and places the strongest choices first. Measure NDCG@k for ranking to account for both relevance and position: a relevant item near the top contributes more than one buried lower down. Evaluate ranking on a defined candidate set so a ranking change is not confused with a retrieval change.
Free tools Windows power users keep installed
One-click scans. No signup required.
Rank #2
A learned or deterministic ranker is one option; an LLM reranker is another. The Microsoft guide describes cross-encoders that score a query and candidate together: this captures their interaction, but costs more latency than encoding each independently, making cross-encoders more practical after retrieval has narrowed the pool. An LLM reranker can handle domain-specific or multifactor relevance, but its per-call cost may be higher. In either case, scores are most safely used to order the candidates; a cutoff threshold needs calibration against actual examples. If using a model that returns scores, validate that output rather than assuming every response is usable.
The guide concerns RAG, where a query is matched to retrieved content, not personalized recommendation. Its retrieval-versus-reranking trade-offs are relevant by analogy, not proof that a RAG design will perform well for a recommender. Benchmark alternatives on representative recommendation traffic, including relevance, latency, throughput, and cost. If candidate text or user-related context goes to a hosted service, consider privacy and compliance requirements before sending it.
4. Feedback and experimentation: close the loop
Record what the system showed and what happened next. Useful fields include the impression, item, position, retrieval source, ranker version, and subsequent outcome. Without impression data, an unclicked item could mean that it was irrelevant—or simply that it was never shown. Use logged outcomes to improve features and models, and assess retrieval recall separately from the quality of the final ordering.
Rank #3
How much does an LLM reranker improve results?
There is no universal answer: results vary with the model, benchmark, query familiarity, and computational cost. A 2025 Findings of EMNLP study evaluated 22 methods and 40 variants across TREC DL19, TREC DL20, BEIR, and the authors’ FutureQueryEval dataset. Its abstract reports stronger performance from LLM-based rerankers on familiar queries, while generalization to novel queries varied; lightweight methods could offer comparable efficiency. That is evidence for testing LLM reranking against the workload—not a guarantee that it will win or lose in a particular recommender. The study is available as a 2025 EMNLP paper.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
There is also a genuine alternative to a four-layer design. Wang and coauthors’ 2023 preprint proposes a “Large Search Model” that unifies query understanding, retrieval, multiple ranking stages, and question answering with one LLM. It is a conceptual proposal and proof of concept, not evidence that a single model is the cost-effective production choice for every workload. Its existence is why “an LLM only touches two layers” should be read as architecture advice, not a technical limit. Read the 2023 Large Search Model preprint.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.How should you fit retrieval and ranking into a latency budget?
Set a latency budget for the complete request, then measure each stage under representative traffic. The original article gives this 100 ms p99 illustration, but its listed components total 75 ms; the remaining 25 ms is not assigned. These figures are the article author’s example, not independently validated benchmark results.
Rank #4
| Illustrative component | Time in the article’s example |
|---|---|
| Feature fetch | 5 ms |
| Retrieval | 20 ms |
| Scoring | 35 ms |
| Business rules | 10 ms |
| Assembly and logging | 5 ms |
| Sum of listed components | 75 ms; 25 ms of the illustrative 100 ms p99 budget is unallocated. |
Use measurements from your own service to decide whether an LLM reranker fits. The candidate pool size, batching opportunities, throughput, model or hosted-service cost, and the time left for other request work all affect that decision. For an LLM call, set a token budget and validate returned scores; a reranker adds processing after initial retrieval, so any relevance gain needs to justify its latency.
How do you build this without mixing up the failure modes?
- Define the recommendation outcome. Specify which user action matters and what context the recommendation serves before choosing a model.
- Make inputs observable. Check that item data, user events, and features have dependable identifiers, timestamps, and freshness appropriate to the product.
- Establish retrieval quality first. Measure recall@k on an evaluation set and inspect whether relevant items are available to the ranker.
- Compare rankers on the same candidates. Use NDCG@k or another suitable ranking measure, then separately test the end-to-end experience.
- Test operational fit. Compare p95 and p99 latency, throughput, inference or hosted-service cost, domain fit, and behavior on novel queries or items.
- Keep a fallback and log the path. A deterministic fallback can preserve service behavior if a model call fails or exceeds its latency budget. Log retrieval source and ranker version so outcomes remain diagnosable.
- Experiment and feed results back. Track impressions and outcomes, then evaluate changes without conflating retrieval improvements with ranking improvements.
A layered design makes it possible to change one part at a time and identify which part helped or failed. An LLM can be a useful component in retrieval or ranking when workload-specific tests justify it; the other layers remain essential to a recommendation system that can be measured and improved.
PC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteQuick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
