What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Structured, multi-step evidence gathering performed better than a single keyword search in my small blind test of JudgeStack, a Magic: The Gathering rules agent—but the result is a promising project evaluation, not proof that the system can answer Magic questions reliably in general. The test also exposed a more basic lesson: retrieving a card’s status is not enough if the system does not know what that status means.
What JudgeStack is meant to prove
JudgeStack is a rules-question agent built to retrieve the source that supports each part of an answer. Its aim is not just to produce plausible rulings, but to identify whether a claim comes from card text, a rules definition, a format status, or a dated announcement.
As an Amazon Associate I earn from qualifying purchases.
That distinction matters because superficially similar Magic questions can require different authorities. “Is this card legal in Vintage now?” is a current-status question. “When did the restriction take effect?” asks for a historical effective date. Current legality data may establish the first; it does not, by itself, establish the second. A dated announcement is needed for the effective-date claim. Combining those questions can tempt a system to invent a date while sounding certain.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallMatch the evidence to the question
- What does this card say now? Use its current Oracle wording.
- How does an interaction work? Use the relevant Oracle text together with the current Comprehensive Rules.
- What were the rules or wording at an earlier point? Use rules and card wording appropriate to that date.
- Is the card legal in a format now? Use current format-legality data.
- When did a ban or restriction begin? Find a dated announcement that establishes the effective date.
- Why does a physical card differ from the current wording? Compare the printed text with current Oracle text.
Wizards of the Coast describes the Comprehensive Rules as a reference for rules and corner cases, intended for consultation on specific questions rather than cover-to-cover reading (Wizards of the Coast’s rules page). The Magic Judges rules resource lists the Comprehensive Rules version effective September 25, 2026 (Magic Judges rules resource). Rules documents are versioned, so the applicable version depends on the question’s date.
#1 Best Overall
- Includes a mix of AT LEAST 25 Rares/Uncommons which is half of the cards.
- Absolutely NO... Basic lands, Foreign, or silver/gold bordered cards.
- Some may contain Foils or Mythics but not all.
- Sets can range from Beta to the current Magic the Gathering set.
- Mint/Excellent condition only.
How the evidence was organized
In a 2026 account, I reported that JudgeStack’s corpus contained 496 documents in ten types: card, printing, ruleParagraph, glossaryTerm, formatEvent, claim, decision, textDifference, adjudicationCase, and authoritySource. Reported component counts included 30 cards, 77 printings, 16 rule paragraphs, 208 legality claims, and 74 detected differences between printed wording and current Oracle text. These figures describe this project’s corpus, not a general Magic dataset.
The system used two Sanity Context MCP endpoints. One exposed structured documents through filtered GROQ queries; the other exposed the Comprehensive Rules as a knowledge-base file. They had to remain separate: an endpoint configured with a dataset source ignores its knowledge-base sources, so combining the sources would have removed access to the rules file.
The public structured dataset includes the 16 rule paragraphs cited by reviewed cases. The complete rules file is used as a retrieval source rather than republished as hundreds of individual dataset documents. This kept the duplicated public rules text narrower; it does not resolve the licensing question.
The Tool Desk
Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →What the comparison tested
The evaluation used 30 questions across three areas: printed wording versus current Oracle text, current format legality, and historical rules changes. Ten questions were held out and never used during development. Both answering conditions used the same answer model and prompt: DeepSeek Flash. Temperature and output-token limits were not set, so provider defaults applied.
| Condition | Evidence gathering | Model-step budget |
|---|---|---|
| One-shot keyword retrieval | One BM25 search over a flattened corpus; the top 12 chunks were supplied in one retrieval pass. | One retrieval pass. |
| Structured evidence gathering | Could query the Sanity dataset with GROQ, read the rules knowledge base, and follow references. | Up to ten model steps. |
This is a comparison of evidence-gathering architectures, not a controlled BM25-versus-GROQ test. The structured condition could take more model turns and choose its next query based on what it had found; the one-shot condition could not. The results therefore reflect both retrieval design and different interaction budgets.
Rank #2
- LEARN THE BASIC ELEMENTS OF MAGIC—Your Magic: The Gathering journey begins with a friend beside you! Play your first game in a guided battle of Aang versus Zuko. Choose your side and send your forces to your opponent while learning essential gameplay lessons
- GUIDED LEARN-TO-PLAY EXPERIENCE—Start by playing a tutorial game with two 20-card decks, each with a step-by-step guide booklet that will walk you through your first game
- CREATE THEMED DECKS—Once you’ve conquered the basics, master the remaining elements by combining any two of the eight 20-card half-decks into a full 40-card Avatar: The Last Airbender-themed deck; mix and match to try different combos!
- EVERYTHING YOU NEED TO PLAY—This Beginner Box includes everything you and a friend need to play, including 2 Playboards that will show you where to place your cards, 2 Spindowns to track your life totals, and 1 Rules Reference booklet to answer any questions you have along the way
- WELCOME TO THE GATHERING—Magic: The Gathering is a collectible card game that weaves deep strategy, gorgeous art, fantastical stories, and a thriving fan community all together into a card game experience like no other
What the blind holdout showed
For the blind evaluation, the 20 answers—ten from each condition—were shuffled and stripped of condition labels. The judge received the questions, expected verdicts, and rubric, but not the labels or the number of answers from each condition. In my 2026 report, the results were:
| Measure on ten held-out questions per condition | One-shot keyword retrieval | Structured evidence gathering |
|---|---|---|
| Verdict correct | 1/10 | 9/10 |
| Reasoning rested on something unretrieved | 7/10 | 1/10 |
On this small holdout, structured evidence gathering produced more correct verdicts and less reasoning based on unretrieved material. Ten questions are too few to support a broad claim about Magic rules questions, and the setup does not isolate the effect of structured queries from the benefit of extra model steps.
The full 30-question suite also produced useful diagnostics, but it includes questions used during development and is not an unbiased test set:
| Full-suite diagnostic | One-shot keyword retrieval | Structured evidence gathering |
|---|---|---|
| Required rules cited | 19/30 | 30/30 |
| Required cards retrieved | 22/30 | 30/30 |
| Cited rules actually retrieved | 24/30 | 30/30 |
| Unsupported citations | 6 | 0 |
Those counts suggest that retrieval coverage and citation support were weaknesses in the one-shot setup, but they should not be read as held-out performance estimates.
The failure that retrieval alone could not fix
The most instructive retained failure involved Sol Ring. JudgeStack retrieved format claims, including that Sol Ring was restricted in Vintage, yet concluded that the card could only be registered in Commander. The answer was wrong: the corpus did not explain what “restricted” means, and the model effectively treated the term as if it meant banned.
Rank #3
- Duplicate-free assortment of 25 random Rare cards.
- May contain Foils, Mythic Rares, or Planeswalkers.
- (No card pictured is guaranteed.)
This was a schema gap, not simply a missed card record. The system had the status but lacked the concept needed to interpret it. The proposed fix is a legality-term concept defining legal, banned, and restricted and explaining how restrictions apply. More retrieval cannot reliably supply a definition the corpus does not contain.
Why one favorable metric was withdrawn
I withdrew an automated “date discipline” score of 10/10 for the structured condition. Its check only asked whether the system had retrieved any format event; it did not test whether that event concerned the card in the question. The two stored events were about an unrelated card, so the check could pass even when an answer invented an effective date.
The metric also failed to distinguish “banned as of” a date from “the ban became effective on” that date. Because the saved evaluation rows did not retain the retrieved IDs, I could not recompute the score against the relevant events. The original 10/10 score remains withdrawn.
This is a useful warning about evaluation design: a metric can reward an answer for retrieving something that looks relevant by type while missing the actual evidence relation the claim requires.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Integration bugs that changed what the system could do
Several implementation problems initially obscured JudgeStack’s behavior. These are issues I reported while building the project, not independently reproduced tests:
Rank #4
- Condition:New: A brand-new, unused, unopened, undamaged item -
- 1 Magic the Gathering MTG Cards Lot w/ Rares and Foils INSTANT COLLECTION !!!
- Brand:Wizards of the Coast MPN:215236245 Recommended Age Range:6+ Country/Region of Manufacture:United States Year:215 Gender:Boys & Girls Character Family:Magic the Gathering
- A balanced array of colors every time guaranteed. Nearly equal Blue, Black, Green, Red and White Magic cards plus multi-colored cards, artifacts and non-basic lands. Cards will be near mint condition or better, All Authentic Wizards of the Coast Magic: the Gathering Cards.
- Incompatible AI SDK dependency versions caused tool-call validation failures.
- Two endpoints exposed identical tool names. Merging their tool sets caused a name collision and dropped the dataset schema overview.
- Stringifying the MCP response instead of extracting
content[].textleft document IDs escaped. Card-ID parsing then failed even though rule-number retrieval appeared to work; flattening the response fixed card retrieval.
These failures show why an apparently weak retrieval result may originate in tool wiring or response parsing rather than in the retrieval strategy or answer model. Debugging needs to check the whole path: tool registration, arguments, returned content, identifier parsing, and the evidence ultimately available to the answer step.
What happened with a local model
In three reported runs, a local Qwen3 configuration made no successful calls to the dataset endpoint and produced invalid arguments for parameterless tools. A separate workflow using a JSON retrieval plan and executing it externally could use the corpus. Three runs do not establish how local models perform generally; they show that this particular tool-calling setup failed and that separating planning from tool execution provided another route.
How reproducible is the result?
The answering model and judging model came from different vendors, but the exact judge build is not pinned because grading happened in the ChatGPT interface. The evaluation pack, rubric, and raw judgments are public, so readers can repeat the grading with another judge. That makes the reported comparison inspectable, but not identical to a fully version-pinned evaluation.
My conclusion is deliberately limited: JudgeStack’s structured condition did substantially better on this ten-question blind holdout, under a setup that gave it more opportunities to gather evidence. The Sol Ring case and withdrawn date score are as important as the accuracy numbers. One shows that an evidence store needs concepts as well as records; the other shows that a metric must verify relevance, not merely the presence of a plausible source type. JudgeStack is a rules laboratory, not a replacement for a judge.
Recommended Free Tools
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
