Recommended Free Tools
In one India-based test of 480 AI answers, changing a skincare or fashion buying question from English to romanized Hinglish coincided with different recommended-brand lists—but the size of the difference varied sharply by engine. Perplexity showed the largest measured language gap, Gemini’s answers included fewer citations for Hinglish prompts, and ChatGPT’s small result was fragile. The experiment is a snapshot, not a universal account of how AI search ranks brands.
What the experiment tested
Depra’s IndicGEO Study 01 asked whether the language of an otherwise equivalent buying prompt changed which brands three AI systems recommended. One example pair was “best face wash for oily skin under ₹500” and “India mein oily skin ke liye 500 rupees ke andar konsa face wash kharidna chahiye?” These are examples from the study, not a representative survey of user questions.
As an Amazon Associate I earn from qualifying purchases.
The researchers used 10 unbranded prompts—five about skincare and five about fashion. Each had an English version and a romanized Hinglish version, independently reviewed for equivalence; three pairs were rewritten after review and before analysis. They ran each prompt eight times in each language on ChatGPT with web search, Gemini, and Perplexity Sonar with live web search. All requests were geolocated to India.
The Tool Desk
Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Collection took place in one randomized, interleaved 57-minute period on 14 August 2026, producing 480 responses. ChatGPT and Gemini were captured from their consumer web apps; Perplexity was queried through the Sonar API answer model, which may behave differently from its consumer app. The sessions were logged out. Depra’s report describes the prompts, collection, and analysis.
#1 Best Overall
How the researchers distinguished language effects from answer variation
An AI answer can change even when a prompt stays the same. To account for that, the study compared the overlap between brand lists for English and Hinglish versions of a prompt with the overlap between repeated runs of that same prompt in the same language. It used Jaccard overlap: the size of the intersection of two lists divided by the size of their union.
This comparison aims to separate language-related differences from ordinary rerun variation. It does not establish why a list changed, or reveal a system’s internal recommendation algorithm. Retrieval, available sources, generation, and other factors could contribute, but the experiment did not isolate them. Citations displayed with an answer show linked sources; they do not explain why a brand appeared.
How much the brand lists changed by engine
The report’s measure of differences beyond each engine’s own rerun variation varied substantially:
| Engine | Measured difference | One-sided permutation-test p-value | How to read the result |
|---|---|---|---|
| ChatGPT with web search | 2.2 percentage points | .046 | Fragile result; the authors characterize ChatGPT as approximately language-invariant. |
| Gemini | 7.8 percentage points | .0098 | A larger measured gap than ChatGPT, in this prompt set. |
| Perplexity Sonar with live web search | 23.4 percentage points | .0010 | Largest measured language gap in this experiment. |
All three one-sided tests cleared the study’s pre-set test, but the ChatGPT result needs particular caution: its two-sided p-value was .092, and four of the ten prompt-level differences were negative. The authors say the overall ChatGPT effect was carried by skincare and describe the result as approximately language-invariant. The study also found Perplexity had the highest same-language rerun overlap among the three systems. That means its answers were comparatively consistent across reruns in this set; it does not mean its recommendations resembled those of other engines.
One prominent Perplexity example was The Derma Co, which appeared in 2 of 80 English answers and 16 of 80 Hinglish answers—a 17.5-percentage-point increase in mention share for those tested prompts. This is an example from a limited prompt corpus, not a general estimate of the brand’s performance or visibility.
Gemini’s Hinglish answers had fewer citations
Gemini included at least one citation in 72 of 80 English responses (90.0%) and 33 of 80 Hinglish responses (41.3%). Average citations per answer fell from 9.4 for English prompts to 5.2 for Hinglish prompts. These figures describe the captured answers, not a general citation rate for Gemini.
The study measured the decline but could not establish its cause. As the report puts it: “The study records the drop. It does not establish the cause, because the design cannot separate how Gemini searches from how it writes.” A citation count alone cannot tell a reader whether a difference arose during retrieval, generation, or another stage.
Which sources appeared in the answers
Source rankings changed depending on whether the unit counted was every citation appearance or each answer citing a domain at least once. In the corpus, YouTube had 534 citation appearances, Reddit 191, and Nykaa 176. By answers citing a domain at least once, Reddit led with 106 of 480 answers, followed by YouTube with 100 of 480.
These are counts from the study’s answers, not measures of overall site authority or a causal explanation for brand recommendations. The difference between the two rankings also shows why a source-mix comparison should state what it counts.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.What the results do—and do not—say about AI recommendations
Language can be worth checking
For these tested questions, English and romanized Hinglish versions did not always produce equivalent brand-list overlap. The degree of change depended on the engine: the largest measured gap was on Perplexity, while ChatGPT’s small difference was qualified as fragile.
This is not a universal ranking formula
The experiment covered two categories, ten prompts, India, logged-out use, and one collection period. It cannot establish a lasting rule for other regions, languages, categories, dates, account states, or prompt styles. It also does not reveal a platform’s internal algorithm. The results should not be turned into a fixed ranking of which engine is best or most accurate.
The capture method matters
ChatGPT and Gemini were accessed through consumer web apps, while Perplexity was accessed through its Sonar API answer model. A comparison across systems therefore also compares different capture routes. The study’s results may not transfer directly to the Perplexity consumer app.
Best Value
The publisher has a commercial interest
Depra, the study publisher, sells a related visibility-monitoring service. That interest is relevant context when weighing the report; it does not by itself invalidate the data. A separate Depra analysis of the same corpus reports that engines cite different websites and share relatively few brands, but it is a vendor re-analysis, not an independent replication. See Depra’s engine comparison with that limitation in mind.
How to check whether prompt language matters for your audience
- Choose audience-relevant questions. Use the actual product category, buying constraints, and language variants that matter to the people you serve. Keep the underlying intent equivalent across versions.
- Test the relevant engine and route. Record whether you used a consumer app or an API, and note settings such as web search when available. Do not treat different access routes as interchangeable.
- Repeat prompts. Run each prompt more than once. A single response cannot distinguish a language pattern from normal answer variation. As study author Ifham Baig advises, “A single manual check tells you almost nothing.” That is the author’s recommendation, not a separate experimental finding.
- Track distinct measures separately. Compare brand-list overlap, rerun consistency, citation frequency and source mix rather than collapsing them into one score. If tracking mention share, record the number of answers in which a brand appears and the total number of answers.
- Repeat over time before calling it a trend. This study captured a short window on one date. A repeated check across relevant questions, languages, engines, and dates can show whether a result persists in your own measurement set.
Depra’s methodology page describes its visibility-measurement approach and caveats; its service context should be considered separately from the evidence in this experiment.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
Do these 3 things before closing this tab:
1Fix the driver behind crashes, sound loss and screen glitches2Clear out junk files and repair common Windows errors3Scan for outdated or missing drivers - takes under a minute
