Measuring AI visibility by sampling prompts

Search platforms report some AI data: Google counts impressions in AI features, and Bing counts citations. See Generative AI performance report: AI Overviews and AI Mode in Search Console and Bing AI Performance report. Other assistants, such as Doubao, DeepSeek or ChatGPT’s app, publish nothing. The only way to know whether they mention you is to ask them many times and count. This entry covers how to do that so the numbers mean something.

Visibility is a proportion

The same prompt to the same model returns different brands from run to run. So “are we recommended?” has no yes/no answer. The measurable quantity is the share of runs that mention you, and a share estimated from a few runs is mostly noise.

Basic statistics sets the floor. For a proportion p measured over N independent runs, the standard error is √(p(1−p)/N). (This calculation is ours, not taken from either source.)

Runs per prompt (N)	Standard error at 20% visibility	at 50%	95% range at 50%
5	±18 pts	±22 pts	±44 pts
10	±13 pts	±16 pts	±31 pts
30	±7 pts	±9 pts	±18 pts
60	±5 pts	±6 pts	±13 pts
100	±4 pts	±5 pts	±10 pts

Runs are not fully independent in practice (same account, same hour, cached retrieval), so real error is usually larger than this.

Re-tested 2026-10 (DeepSeek API (deepseek-flash, default sampling, no web search), Chinese): consistent with this claim. 60 answers to one prompt. At 30 runs, brands mentioned in 15–85% of answers varied by ±8–9 pts (bootstrap standard deviation).

In September 2025, Miaozhen Marketing Science Institute asked three models (DeepSeek, Doubao, Yuanbao) 2,025 times in all. It used one typical question per product category, five categories, and 135 runs per model. It concluded that results reach basic stability only at 30 or more runs. High-visibility brands (over 80%) were more stable than mid (50%) or long-tail (20%) ones. Low-visibility brands need 60 runs or more.

Observed in practice · observed 2025-09 · 中国商务广告协会AI营销应用工作委员会 · 秒针营销科学院

The study’s reported spreads at 30 runs (±6–8%) are close to the standard errors above. Its figures at 100 runs (±2%) are smaller than sampling theory allows (±4–5 points). Treat that row with caution, because the report doesn’t say how the spread was computed.

Test where your customers ask

A 2026 study of Pilates studios in Hefei ran the same questions through vendor APIs and through consumer apps on phones. Doubao gave similar results in both. Qwen recommended the brand in 78% of API runs but 53% of app runs. DeepSeek and Yuanbao were reversed: DeepSeek 69% by API against 1% in the app, and Yuanbao 3% by API against 99% in the app.

Observed in practice · observed 2026-08 · 腾讯新闻（报道合肥图灵象普拉提行业GEO白皮书）

The API and the consumer app can run different models, retrieval settings, system prompts and location handling. So the API is not a cheap proxy for what people see. Measure in the product your customers use, logged out or with a fresh account. Ask from the city your customers ask from. If you also run the API for scale, keep the two sets of numbers apart.

A brand’s visibility on one platform says little about another, and being mentioned is not the same as being mentioned first.

Observed in 2025-06, more than 9 months ago. Answer engines change often; re-test before relying on this.

In Xsignal’s infant formula monitoring (DeepSeek, Doubao, Yuanbao, Kimi; 1–26 June 2025), Friso Prestige had the highest mention rate on DeepSeek (60.6%) and Yuanbao (64.7%). It ranked 5th on Doubao and 13th on Kimi. On Yuanbao, Aptamil’s total mention rate was 38.44% against Friso Prestige’s 60.55%. Yet Aptamil was named first far more often (28.39% against 5.78%). The report does not disclose the number of prompts or repeats. (Brand names and quotes translated from Chinese.)

Observed in practice · observed 2025-06 · Xsignal（奇异因子）

Report each platform separately, and record the position of the first mention as well as whether the brand appeared at all.

Choose prompts that reflect demand

In the same study, the studio appeared in only 34% of answers to niche pre-purchase questions, such as who is good at postnatal recovery. Broad category questions returned it 77% of the time.

Observed in practice · observed 2026-08 · 腾讯新闻（报道合肥图灵象普拉提行业GEO白皮书）

A prompt that contains your brand name nearly always returns your brand. That proves little, because new customers ask category questions (“best Pilates studio near me for back pain”). Build the prompt set from real customer questions. Group prompts by intent and report each group separately. Brand-name prompts are still useful for checking that what assistants say about you is accurate.

A minimal protocol

1. Fix the prompt set before you start, and keep it unchanged between rounds so results compare. Version it when you must change it.
2. Record the environment for every run: product (app or API), model or mode, account state, location, date.
3. Run each prompt at least 30 times per environment per round. Use more runs for prompts where you are rarely mentioned.
4. Report proportions with their range, for example “41% ± 9 pts, n = 30, Doubao app, Shanghai, 2026-10”. Never show a single screenshot as a result.
5. Expect movement between rounds. Models and retrieval change, so a change smaller than the sampling error is not a result. Treat any claim of a “stable #1” position in AI answers with suspicion.
6. Keep the raw answers. Counting rules change, and you will want to recount.

Third-party GEO tools do this sampling for you. They observe answers from the outside, like you would. None of them sees inside Google’s systems (Claim: GEO tools can see your rankings inside Google's AI).