To judge whether an AI search guide is reliable, trace its statistics to original sources, look for uncertainty and repeatability in visibility measurements, check crawler claims against official documentation, and note when the evidence was collected. Those checks matter more than a confident claim that one tactic will make a brand appear in AI answers.
What the 15-guide check can—and cannot—show
In a DEV Community post published September 21, 2026, Devin D described checking the first five Google results for each of three searches: “llm seo,” “ai visibility,” and “how to rank in chatgpt.” The author says the scan took place in one day, on September 18, and involved searching saved HTML rather than reading every guide in full. The post also notes that the guides covered subjects Shruwd was writing about for a tool it was building.
As an Amazon Associate I earn from qualifying purchases.
That disclosure gives readers useful context, but does not by itself invalidate the checks. The author explicitly cautions against generalizing from the sample. It is best read as a practical audit of advice, not a systematic review or a representative survey of AI search guidance.
1. Trace statistics to their original source
A number in a guide is not automatically evidence. Ask who measured it, what they counted, which platforms and pages were included, and whether the guide links to the underlying study. If the original method or population is missing, treat the number as an assertion rather than a dependable basis for strategy.
#1 Best Overall
In the DEV post’s sample, all five “llm seo” guides recommended schema markup at least once. Two claimed schema appeared on close to every page cited by ChatGPT without linking a source. That observation does not establish how common schema is across ChatGPT citations; it shows why readers should look for the evidence behind an impressive-sounding percentage.
What a later schema study found
Ahrefs reported a matched study of 1,885 pages that added JSON-LD schema between August 2025 and March 2026, compared with 4,000 control pages. Using a matched difference-in-differences analysis, Ahrefs estimated changes of −4.6% for Google AI Overviews, +2.4% for Google AI Mode, and +2.2% for ChatGPT. The latter two estimates were statistically indistinguishable from zero; Ahrefs says it cannot determine whether schema had a very small positive effect or no effect. Ahrefs’ study and methodology
These estimates are bounded results from one matched analysis, not proof that structured data is useless in every context. They do caution against making schema the centerpiece of an AI search plan solely because an unsourced guide recommends it.
Rank #2
2. Treat AI visibility as a noisy measurement
A visibility score can look precise while resting on a small, unstable sample. In the DEV post’s “ai visibility” sample, three of five guides recommended one score; none supplied a range or explained how large a change had to be before it should be trusted.
Consider an illustrative sample in which a brand appears in 10 of 30 answers: the observed rate is 33%, but an approximate 95% interval for the underlying rate is about 19%–51%. The interval is wide because 30 observations provide limited precision. A score without its sample size and uncertainty can hide that uncertainty.
The post also reports a SparkToro/Gumshoe exercise in which 600 volunteers ran 12 prompts through ChatGPT, Claude, and Google’s AI, producing 2,961 runs in late 2025. According to Devin D’s account, the study’s authors put the chance of two ChatGPT or Google AI responses naming the same list of brands below 1 in 100. That figure is reported second-hand here, not independently verified against the original study; treat it accordingly.
Rank #3
Devin D also reproduces a comment attributed to Sergei Rogulin, Semrush’s head of organic and AI visibility: a share of voice “that swings between 20% and 40% over a day is normal.” It is a reported comment, not a universal error range for every tool, prompt set, or measurement design.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Fix the driver behind crashes, sound loss and screen glitches3Clear out junk files and repair common Windows errorsCompare like with like
Before comparing scores—your own over time or one provider’s with another—check what was actually measured:
- Prompt set and size: Which questions were tested, and how many observations support the score?
- Platform: Was the result measured in ChatGPT Search, Google AI Mode, Perplexity, or another product? Do not treat them as interchangeable.
- Conditions: Are geography, language, personalization, and other relevant settings stated?
- Outcome: Does “visibility” mean a citation, a brand mention, or something else?
- Repeatability and uncertainty: Were prompts run more than once, and does the report provide a range or margin rather than one point estimate?
- Date and cadence: When were observations collected, and how often are they repeated?
3. Verify crawler claims with the platform’s documentation
“AI bot” is not one setting with one consequence. OpenAI describes separate crawlers for search visibility, possible training use, and user-initiated actions. Check the current documentation before changing robots.txt or interpreting server logs. OpenAI also publishes IP lists; a user-agent string alone does not prove that a request came from OpenAI. OpenAI’s crawler documentation
OAI-SearchBot: ChatGPT search
OpenAI says OAI-SearchBot is used to surface websites in ChatGPT search. A site that opts out will not be shown in ChatGPT search answers, though it can still appear as a navigational link. OpenAI recommends allowing the crawler if a publisher wants its site to appear in search and says changes to robots.txt can take about 24 hours to be reflected by its systems.
GPTBot: possible training use
GPTBot crawls content that may be used to train OpenAI’s generative models. OpenAI says disallowing GPTBot signals that the content should not be used for training. Its controls are independent of OAI-SearchBot’s, so a decision about training access does not automatically determine search access.
ChatGPT-User: user-initiated requests
ChatGPT-User is used for certain actions initiated by a ChatGPT or Custom GPT user. OpenAI says it is not used for automatic web crawling or to determine search inclusion. Because these requests are user-initiated, robots.txt rules may not apply in the same way.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.4. Put dates next to visibility claims
AI answers and the sources they cite can change over time. A trend without a collection window is difficult to interpret, and a past share should not be presented as today’s baseline.
Devin D’s post says Reddit’s share of ChatGPT responses citing it fell from nearly six in ten in early August 2025 to about one in ten by mid-September 2025. A separate Semrush study tracked more than 230,000 prompts weekly across ChatGPT Search, Google AI Mode, and Perplexity from July 14 to October 12, 2025. Semrush reported a sharp mid-September decline in Reddit and Wikipedia citation share in ChatGPT, while both remained its two most-cited domains in October. These are dated, platform-specific observations—not predictions about what AI systems cite now. Semrush’s AI search citation study
Quick Recap
A quick audit before acting on a guide
- Follow the number. Find the original study or documentation. If the guide provides no source, do not treat the statistic as established.
- Read the method. Identify the sample, platforms, measurement, and comparison group. Check whether the result is an association or a tested effect.
- Look for uncertainty. Prefer repeated measurements and ranges over a lone visibility score; small samples can produce wide uncertainty.
- Check the date. Make sure the evidence window is clear and recent enough for the decision you are making.
- Confirm technical claims at the source. For crawler behavior, use the platform’s current documentation and published IP information rather than repeating a guide’s shorthand.
- Keep the conclusion proportional. A study can inform a choice without settling every case. For example, Ahrefs’ schema estimates do not establish a universal rule for all structured data or sites.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
