In Utkarsh Varshney’s 2026 Olympic-events benchmark, moving from text retrieval to a graph query produced the biggest reported gain: accuracy rose from 18% to 92% while estimated token use fell. Adding an agentic loop raised the reported score to 100%, but also used more tokens than the single-query graph pipeline. The result is a useful case study—not proof that agents generally outperform well-designed GraphRAG systems.
What the TigerGraph benchmark tested
Varshney’s October 3, 2026 article describes three question-answering pipelines tested on a corpus of approximately 2,900 Wikipedia articles about Olympic events. The evaluation included 100 questions with answers and 50 hidden questions. The questions covered straightforward lookups, multi-hop identification, temporal comparisons, aggregation, and superlatives. Roughly 760 documents about films and companies were included as distractors.
As an Amazon Associate I earn from qualifying purchases.
The author parsed Olympic-event infoboxes into structured Event vertices, including fields such as sport, year, season, venue, date, competitor count, nations, and medallists. The resulting graph contained 2,187 events loaded into TigerGraph Savanna. GSQL endpoints handled filtering, counting, and lookups. Varshney’s design principle, expressed in his heading “The LLM plans, the graph computes,” was to let a model plan a query while the graph database carries out deterministic operations.
These corpus and implementation details are author-reported, not an independent audit of the data or systems. The three approaches also differ in more than whether they use an agent: the agentic version is described as checking candidates and gathering further evidence, while the GraphRAG baseline runs one query and returns its result.
#1 Best Overall
How the three approaches differ
| Approach | What it does in this benchmark | Reported accuracy | Estimated tokens per query |
|---|---|---|---|
| RAG | Retrieves the five most similar documents and uses their text to answer. | 18% (Varshney, 2026; 100 evaluation questions) | 1,573 (author estimate) |
| GraphRAG | Has the model produce a plan, runs one graph query, and returns the result. | 92% (Varshney, 2026; 100 evaluation questions) | 247 (author estimate) |
| Agentic GraphRAG | Uses an orchestrator loop to plan and query, judge whether evidence is sufficient, and potentially relax filters, rematch events, or verify against a second source. | 100% (Varshney, 2026; 100 evaluation questions) | 1,295 (author estimate) |
All three rows report results from this author’s experiment, not expected performance in other deployments. The token figures are estimates; the article does not report latency or monetary cost. On the reported estimates, the agentic pipeline used about 5.2 times as many tokens per query as the one-query GraphRAG baseline (1,295 divided by 247). That is a comparison of these implementations, not a general cost ratio for agentic and non-agentic systems.
Why the graph pipeline outperformed text retrieval here
Aggregation and superlatives
Questions that require counting or finding an extreme value across many events are naturally suited to database operations, provided the records and query are correct. A top-five text retriever returns a small sample of passages; those passages may not contain enough evidence to count across the full event set. Varshney reports that the RAG pipeline scored zero on aggregation and superlative questions, while the graph pipeline could execute structured filters and computations.
Temporal comparisons
The author also reports that RAG confused Olympic years in questions such as identifying the Games immediately before 2016. Similar-looking year strings can lead retrieval toward a relevant-looking but incorrect passage. A graph query can express the year constraint directly, assuming the data and query logic encode the intended comparison correctly.
Multi-hop lookups
A question such as “Who won gold at Beijing National Stadium on 16 August 2008?” requires matching more than one condition. The venue can correspond to multiple events, so a venue-only match may not identify the intended record. In the reported example, the agent inspected multiple candidates and used the date to disambiguate them.
Rank #3
- Commemorate Tiger Woods' 25-year journey with a billiant, fully illustrated table book from Sports Illustrated
- Sturdy build and construction. The hand bounded green leather hardcover gives it the perfect vintage look and durability
- Its polished aesthetic perfectly aligns with the golf theme of this book, lending an elegant touch to your bookshelf or coffee table.
- 232 pages full of iconic vibrant photos and some of the best written coverage of Woods’s career
- Beautiful Stories, a good read, and great photographies, the ideal gift book for any Tiger fan
What the agent added—and what the result does not prove
Varshney attributes the final eight percentage points, from 92% for GraphRAG to 100% for Agentic GraphRAG, to ambiguous multi-hop questions. The described loop could check whether evidence was sufficient, examine candidate matches, adjust filters, and seek another source. That is a plausible role for an agent: coordinating multiple evidence-gathering steps when a first query leaves a meaningful ambiguity.
But the benchmark does not isolate the value of the agentic loop from the value of the extra candidate-disambiguation behavior. The one-query GraphRAG baseline is not reported as receiving the same candidate enumeration and date-checking rule. A stronger comparison would add that rule to the graph-query baseline, then measure whether iterative planning and sufficiency checks improve the result beyond it. Because that ablation is not reported, the 8-point difference cannot be attributed solely to agentic planning.
There is also an important answer-quality rule: if available graph fields still leave multiple candidates, a system should preserve the ambiguity rather than use similarity as a tie-breaker that implies unsupported certainty. Similarity can help rank candidates, but ranking is not the same as establishing which answer is correct.
PC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteHow to decide whether an agent is worth adding
- Start with the question type. If records are structured and the task is filtering, counting, or finding a maximum or minimum, test a direct database query before adding an agent loop.
- Inspect errors by category. Aggregate accuracy can hide whether failures come from retrieval, incorrect query plans, missing fields, or unresolved entity matches. Evaluate lookups, multi-hop questions, temporal comparisons, and aggregations separately.
- Give simpler systems the same evidence rules. If an agent enumerates candidates and applies a date constraint, test those same steps in the non-agentic baseline. Otherwise, a comparison may credit the loop for improvements caused by a better query strategy.
- Measure resource use alongside accuracy. This report includes estimated tokens, but not latency, dollar cost, confidence intervals, or run-to-run variation. Those measures are needed to decide whether a gain is worth the operational overhead in a particular application.
- Define what happens when evidence is insufficient. Require the system to return uncertainty or ask for clarification when structured fields do not uniquely identify an answer.
- Validate on the intended corpus. This benchmark uses Olympic-event records and one implementation. It does not establish results for different graph schemas, models, corpora, or production workloads.
Reproducing the TigerGraph setup
The benchmark article describes operational details specific to its Savanna 4.x setup. Varshney says token retrieval used /gsql/v1/tokens rather than the older /restpp/requesttoken, that Auto Resume needed to be enabled to avoid HTTP 500 responses from a suspended workspace, and that REST calls to installed GSQL queries required every parameter to be supplied, making no-op defaults useful. These are the author’s deployment observations, not universal guarantees for every TigerGraph version or configuration.
TigerGraph’s official GraphRAG repository is a separate project from Varshney’s benchmark code. It describes Classic and Agentic modes, including planned and reactive retrieval, and lists Docker Compose or Kubernetes, TigerGraph DB 4.2 or later, and an LLM provider key among its prerequisites. Its documentation should not be treated as independent validation of the benchmark’s scores. TigerGraph’s data-plane documentation separately describes Savanna workspace database requests and authentication using a database secret or bearer token.
How strong is the evidence?
The reported result is a single-author experiment on one Olympic-events corpus with 100 evaluation questions. The article does not report confidence intervals, repeated-run variance, independent replication, latency, monetary cost, or a full set of model and prompt controls. It also does not report the baseline ablation that would determine how much of the agent’s improvement remains after giving GraphRAG equivalent candidate-checking behavior. Treat the figures as findings about these implementations on this evaluation set, not as a ranking of architecture classes.
Sources: Utkarsh Varshney, “When do AI agents actually matter? Benchmarking RAG vs GraphRAG vs Agentic GraphRAG on TigerGraph,” DEV Community, October 3, 2026; TigerGraph, “TigerGraph GraphRAG,” official repository; TigerGraph, “Data-plane APIs (REST++ and GSQL),” official Savanna documentation.
Free tools Windows power users keep installed
One-click scans. No signup required.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
