You measure AI share of voice by running a fixed set of buyer prompts against ChatGPT, Gemini and Perplexity several times, saving every raw answer and its citations, classifying mentions, citations and recommendations as separate fields, and reporting each engine on its own with a written denominator. Python handles the collection loop, storage and counting well. The judgment calls, such as what counts as a mention and which domain belongs to a brand, still need a written rule and a human check.
Define the numerator and denominator before you write code
“Share of voice” is not one number. A brand can be named in 30% of answers, or hold 45% of all brand mentions, or be cited in 12% of answers, and each figure answers a different question. Decide which one you are reporting before you collect anything, because the wrong denominator produces a precise-looking number that answers nothing you actually asked.
As an Amazon Associate I earn from qualifying purchases.
| Measure | Question it answers | Numerator | Denominator to state |
|---|---|---|---|
| Mention rate (answer share) | How often is the brand named at all? | Measured answers that name the brand | All successfully measured answers for that engine (or for a single prompt) |
| Share of mentions | Of all brand names that appear, how many belong to the target? | Mentions of the target brand | Mentions of every tracked brand in the same answers |
| Citation rate | How often does an answer link to a source you specified? | Measured answers with a citation URL on a specified domain | All successfully measured answers for that engine |
| Recommendation rate | How often does the answer explicitly advise the brand? | Measured answers with an explicit recommendation for the brand | All successfully measured answers for that engine |
Per-brand answer shares are counted independently. If one answer names three brands, it adds to three brands’ counts, so the answer shares for a set of brands can sum well past 100%. That is expected behavior for this denominator, not an error. The share-of-mentions version sums to 100% across tracked brands by construction, which is why it is a different measure. Label whichever you use. None of these definitions is a universal industry standard; several commercial tools define share of voice differently, and you should use the definition that matches your question.
Set the category, brands and aliases
Fix four things in a configuration file before the first run: the business category, the target brand, a competitor set, and a list of aliases for each brand. Aliases matter more than they seem. A brand that appears as “Example Analytics”, “Example Analytix” or “example analytics” will be missed by a case-sensitive string match, and a plural or possessive form can break an exact match on its own. Keep the alias map in one place, and keep the competitor set unchanged for the whole measurement period.
#1 Best Overall
category: "project management software for agencies"
locale: "en-US"
target_brand: "Example Analytics"
competitors:
- "Rival Insights"
- "Northwind Metrics"
aliases:
Example Analytics: ["Example Analytics", "Example Analytix"]
Rival Insights: ["Rival Insights", "RivalInsights"]
Northwind Metrics: ["Northwind Metrics"]
engines: ["chatgpt", "gemini", "perplexity"]
runs_per_prompt: 3
prompts_file: "prompts.csv"
Build a prompt library that stays still
The prompt library is the unit of comparison. Each prompt should be a question a real customer would ask, written the way they would type it. Record for each prompt: a stable prompt_id, the prompt wording, the category, the intended audience, the locale, and the date the prompt set was frozen. Store the file under version control or record a hash of it in every output, so you can prove that two measurement periods used identical wording.
If you must change a prompt, do not silently edit the file and keep comparing. Start a new measurement period, keep the old one for reference, and compare only within a period.
Run the same prompts on each engine
Run every prompt on every engine in the configuration, with the same competitor set and the same run count. Your collection code should treat the three engines as separate sources, because each one may expose different models, retrieval behavior and citation formats. For each engine, record the interface or model identifier when the engine exposes one. If you query a consumer web interface rather than an API, record the interface, locale, login state and collection date, and assume the output may differ from what an API returns for the same prompt.
Suggested collection sequence
- Create the configuration file and freeze the prompt file. Record its hash.
- Run a small validation pass with one run per prompt on one engine. Read a sample of answers and confirm that the brand names, citations and error handling behave as you expect.
- Extend the validation to all engines and check that each engine returns usable records or explicit failures.
- Run the full set: engines × prompts × runs per prompt. Cost and time grow with that product, so keep the run count low until the validation pass is clean.
- Write every record to a raw file, one JSON object per line, before any analysis. Never overwrite a raw file.
- Classify answers in a separate step that reads the raw file, so you can re-run classification with a corrected alias map without collecting again.
- Generate per-engine tables first, then any combined view.
Record each answer with the fields you will need later
| Field | What to store | Why it matters |
|---|---|---|
engine |
chatgpt, gemini or perplexity | Keeps results separate by platform |
model_or_interface |
Model identifier or interface name, where exposed; otherwise “not stated by engine” | Lets you explain a shift that comes from a platform change |
prompt_id and prompt_text |
Stable ID and exact wording | Makes each answer traceable to its prompt |
run_id |
Run number for this prompt on this engine | Shows how many samples each prompt received |
collected_at_utc |
Timestamp in UTC | Allows date-based comparison within a period |
answer_text |
Full raw answer | Lets a human audit every classification |
citation_urls |
All URLs the interface links, in order | Needed for citation and cited-domain analysis |
retrieval_used |
Whether the interface shows a search or retrieval indicator, true, false or unknown | Separates answers drawn from live web search from those that are not |
collection_status |
ok, error or timeout, with the error message | Keeps failures visible instead of counted as zero visibility |
Classify mentions, citations and recommendations separately
A mention says the brand was named. A citation says the answer linked to a source. A recommendation says the answer advised a choice. An answer can contain any combination, so store each as its own field. A brand’s own domain and a third-party page that mentions the brand are different citation types; classify them separately if the distinction matters to you.
The matching function below is a starting point. It uses word boundaries so that a short name does not match inside a longer word, and it is case-insensitive.
import re
from collections import defaultdict
def brand_mentioned(answer, names):
for name in names:
pattern = r"(?<!w)" + re.escape(name) + r"(?!w)"
if re.search(pattern, answer, flags=re.IGNORECASE):
return True
return False
def share_by_engine(records, aliases):
denom = defaultdict(int)
hits = defaultdict(lambda: defaultdict(int))
for r in records:
if r["collection_status"] != "ok":
continue # failed runs are excluded here and counted separately
denom[r["engine"]] += 1
for brand, names in aliases.items():
if brand_mentioned(r["answer_text"], names):
hits[r["engine"]][brand] += 1
return {
engine: {brand: hits[engine][brand] / denom[engine] for brand in aliases}
for engine in denom
}
Word boundaries do not solve brand names that are ordinary words or product terms. A brand called “Monday” or “Apple” will match in unrelated sentences. Check a random sample of answers for every brand whose name overlaps with common language, and add a manual-review flag rather than trusting the match. Position in the answer and sentiment are useful secondary measures, but they depend on annotation rules that must be written down and checked; treat automated sentiment labels as a rough signal, not ground truth.
Handle failed runs as failures
A timeout, a refused request or an empty response is not evidence that the brand was absent. Store it with its status, exclude it from the denominator, and report the count of failures per engine next to the results. If failures cluster on one engine or one prompt, that is itself a finding to report, because the shares for that engine rest on fewer measured answers than the configuration implies.
Repeat runs and report uncertainty
A single run is one sample, not a census of what an engine could say. Ronald Sielinski’s 2026 paper examined Perplexity Search, OpenAI’s SearchGPT and Google Gemini and found substantial variation across repeated visibility samples. The paper cautions that single-run figures can look more precise than they are. Its findings are scoped to the platforms, topics and sampling it describes, so apply the method to your own category rather than treating its results as a benchmark for yours.
Report the number of measured answers behind every percentage. For a simple proportion, a Wilson interval gives a more honest range than the raw figure. The function below returns a 95% interval.
from math import sqrt
def wilson_interval(hits, n, z=1.96):
if n == 0:
return (0.0, 0.0)
p = hits / n
denom = 1 + z**2 / n
centre = (p + z**2 / (2 * n)) / denom
half = z * sqrt(p * (1 - p) / n + z**2 / (4 * n**2)) / denom
return (centre - half, centre + half)
Treat intervals as a floor for uncertainty, not a ceiling. Answers to different prompts from the same engine are not fully independent, so an interval computed as if every answer were a fresh draw will be too narrow. Do not claim a meaningful change between two periods unless the ranges clearly separate and the prompt set was identical in both.
Worked arithmetic with hypothetical counts
The figures below are illustrative arithmetic, not measured results. Suppose a Perplexity configuration with 20 prompts and 3 runs per prompt yields 60 measured answers. The target brand is named in 18 answers, Rival Insights in 33 and Northwind Metrics in 27.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Clear out junk files and repair common Windows errors3Fix the driver behind crashes, sound loss and screen glitches- Target mention rate: 18 ÷ 60 = 30%. Wilson 95% interval: roughly 18% to 41%.
- Rival Insights: 33 ÷ 60 = 55%.
- Northwind Metrics: 27 ÷ 60 = 45%.
- Sum of the three answer shares: 130%. This is expected because some answers name more than one brand.
Roll up results only after the engine tables
Report each engine in its own table first. If you need one combined figure, state how engines and prompts are weighted. An unweighted average across engines can hide a large difference between them, and an average across prompts can be dominated by whichever prompts were run most often. If a combined figure is useful for a dashboard, show it beside the per-engine values rather than in place of them.
What Google’s own tools cover
Google Search Central’s guide, “Google’s Guide to Optimizing for Generative AI Features on Google Search”, states: “You don’t need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities).” The same guidance points site owners to the generative AI performance report in Search Console, which covers visibility in Google Search and Discover generative AI features.
That report is first-party measurement for Google surfaces. It is not a cross-engine view of ChatGPT, Gemini and Perplexity, and it does not measure how those three engines answer your prompts. Google also cautions that third-party tools have no access to its internal ranking or AI systems, so any monitoring product’s output should be read as a sampled observation of public answers, not as a view of Google’s internal metrics.
Choose between a Python collector and managed monitoring
A Python collector gives you full control over prompts, run counts and raw data, at the cost of maintaining the code and the platform access yourself. Managed monitoring products can handle collection and reporting for you. Vendor documentation for the category, including Yext’s description of a prompt-library approach with competitor comparisons and SourceWatch’s documentation of visibility and share-of-voice outputs, shows that such tools exist, but it does not establish which vendor fits your needs. Compare them on the points below, checking each vendor’s current documentation.
The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Quick Recap
| Criterion | DIY Python collector | Managed monitoring tool |
|---|---|---|
| Coverage of all three engines | Limited to the engines you build access for | Confirm in current vendor documentation |
| Control over prompts and repeat runs | Full control | Confirm the prompt library and run settings offered |
| Raw response and citation export | Full export in your own format | Confirm export formats and whether raw answers are included |
| Separation of mentions, citations and recommendations | Your schema defines it | Confirm whether these are separate fields |
| Per-engine reporting | Your tables define it | Confirm that results are not only blended |
| Handling of failed runs | Your rule; state it in the report | Confirm whether failures are excluded, and how |
| Maintenance | Your responsibility as interfaces change | Vendor responsibility, subject to its terms |
Limits to state in every report
- Engine interfaces, models, retrieval behavior and citation formats change. Record the collection date on every table.
- Results describe your prompt set, your locale and the collection conditions. They are not a market-wide average for the category.
- Share-of-voice figures from one period are not directly comparable with figures from another if the prompt set or competitor set changed.
- Automated matching and classification need a sampled human check before any figure is published.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
