Measuring AI Visibility: What Can Actually Be Counted

Ask an AI engine a question. Ask again five minutes later. About two thirds of the sources it cites will have changed, and nothing about your site moved at all.

The instrument moves more than the thing it measures

Every chapter so far has deferred one question. How would you know any of this worked?

It is the question a board asks first, and it is the one this whole discipline answers worst.

Here is our answer. You will not like the first half.

In April 2026 three researchers published the measurement study this field needed. Four engines, tracked for forty-six days. ChatGPT, Perplexity, Gemini, Google AI Mode.

They compared the sources cited on one day against the sources cited the next. Then they ran the same comparison inside a single day. That second test is the one that matters.

The day-to-day overlap averaged between 0.34 and 0.42. That is the whole chapter in one figure.

Read that as a percentage and it means roughly 65% of cited sources change overnight. Two thirds of your measurement, gone by morning.

Rank order was worse: 0.21 to 0.26. Not only do the sources change, so does the order they appear in. There goes rank tracking.

Now the finding that decides everything.

Read that twice. Inside one day, the churn is the same as across days.

So it is not news moving. It is not your competitors shipping. It is not an algorithm update you missed. It is the machine being a machine.

Their own sentence for practitioners is blunter than ours would be. Query an engine once, they say, and the snapshot "may differ substantially from a second query executed minutes later under identical conditions." Minutes.

Under identical conditions.

That is the phrase to keep, and it is theirs, not ours.

One more number from the same study, and it breaks most dashboards quietly. ChatGPT only runs a web search for some questions. In their sample, 57.8% of runs returned zero citations.

Think about what that does to a percentage. If more than half your runs cite nothing, what is the denominator on your citation share?

Most tools will not tell you. Some do not know. Ask anyway.

What one answer is worth

A second paper, July 2026, asked a narrower question. How much of what an AI says about your brand is actually about your brand?

The design was large: 12,933 responses, 20 brands, 8 languages, 3 models.

Then they decomposed the variance. When an answer differs, what actually drove the difference?

The language of the query explained 26.5% of the variation in a single response.

Brand identity explained 1.5%. One and a half percent of the variation.

Sit with those two numbers side by side. What language you asked in mattered seventeen times more than which brand you asked about.

Their conclusion is flat: "a single AI answer carries almost no brand-discriminating signal."

Reliability for brand ranking came out near 0.01 on one answer. Near zero. It reaches about 0.36 only across the full crossed design of languages and models.

So reliability is bought by spreading, not by repeating. Ask in more places, not the same place more often.

One honest caveat before you quote that at somebody. The outcome measured there was sentiment, not citation share. The shape of the finding transfers. The specific number does not.

So how many times do you have to ask?

Somebody has actually computed this, which surprised us.

The April study asks how many runs are enough. Their answer: at least seven runs per prompt per day for brand visibility, and at least eight when you care which sources were cited.

Seven runs. Per prompt. Every single day.

Is that what you are buying?

Now do the arithmetic on a hundred prompts. Seven hundred queries a day. Daily. You can see why most dashboards skip it.

Then the July paper adds the other half. Repeats have sharply diminishing returns, and it puts a number on that: a repeat past the fifth improves reliability by 0.0003. Which is nothing.

So there is a floor and a ceiling. Below roughly seven runs, you are reading noise. Above roughly five, more runs buy almost nothing. A narrow window.

So spread the budget sideways instead: more prompts, more languages, more engines. Not more repeats.

And do not carry these numbers around as universal. The survey that reviewed this literature says so directly: the figure "derives from a small universe of Swiss queries and at most ten repetitions."

Take the method. Leave the constant behind, because it was measured somewhere else.

Why it moves

Four things happen between your question and an answer. Each one is a coin flip.

First, the engine decides whether to search at all. Sometimes it just answers from memory.

Second, if it searches, retrieval returns a set of documents. That set is not fixed.

Third, only some of those documents fit in the context the model gets. Something is always dropped.

Fourth, the model decides which of what it read to cite. Not everything used gets named.

Four gates, and you sit behind all of them. Most dashboards report the last one. Then they call it visibility. It is not.

The critical survey of this literature writes the chain out properly. Your odds of being cited are the odds a search fires, times the odds you are retrieved given it fired, times the odds you are cited given you were retrieved. Three multiplications. Every one below one.

Which has a consequence worth stating. A tool that computes your share of citations only among answers that had citations has silently dropped the first gate. And the first gate was the biggest one.

The first gate was more than half the runs.

Can you fix this by pinning the temperature to zero? No, and somebody checked.

The survey again. A reported temperature of zero "fixes neither the index, nor retrieval, nor the versions of external services."

The model is one source of randomness. It is not the largest one. The index underneath it is.

For contrast, hold it against classical search. One study of 4,706 queries found month-to-month page overlap of 18% for AI Overviews. Organic Google came in at 45%. Better than double.

Organic results churn too. They churn less than half as much, and you have twenty years of practice reading them.

Do the arithmetic once

Take a number your dashboard might show you. Say it reports 8% citation share on ChatGPT.

Now walk it back through the four gates.

Gate one: did a search fire? In the April study, 57.8% of ChatGPT runs cited nothing at all.

So if that 8% counts only the runs with citations, it describes fewer than half the runs. Across all runs it is closer to 3%.

Same measurement. Different denominator. Nearly triple the number, and nothing about you changed.

Gate two: how many runs produced it? If the answer is one per prompt per day, the published floor is seven.

Gate three: which sources moved? Roughly 65% turn over by tomorrow. Including yours.

Gate four: was your brand named without a link? That does not appear anywhere in the count.

So the honest reading of 8% is not a percentage. It is a range, on an unstated denominator, from a sample size nobody published.

None of that makes the tool useless. It makes the decimal point dishonest.

Report it as a band and you are being accurate. Report it as 8.0% and you are being precise about noise.

What changed in 2026

Now the good news, and there is some: two engines started reporting.

In February 2026 Microsoft shipped AI Performance in Bing Webmaster Tools. Its own framing: "For the first time, you can understand how often your content is cited in generative answers, with clear visibility into which URLs are referenced." First time is right.

Actual citations. Actual URLs. First-party, from the engine that served the answer to a real person.

Then in June 2026 Google launched generative AI performance reports in Search Console.

Two engines, four months apart, both conceding that publishers deserve numbers. That is a real change and this chapter will not undersell it.

Now read the fine print, because both reports are narrower than the headlines.

Take Google's report first. It reports impressions. Google defines them plainly: "Impressions are how many times links to your site were shown to a user in a generative AI feature on Google Search."

Useful. Also the only metric in the report. There is no click number, and no position.

You can group the data by page, by country and by date. There is no query dimension.

So you can learn that a page was shown. You cannot learn what anybody asked. The question stays hidden.

Two more limits worth knowing before you go looking for the tab. Google does not split AI Overviews from AI Mode. The report covers both in one bucket. And it is not everywhere yet, being rolled out in Google's words "to a subset of website owners." Check before you look.

Bing goes further on substance and is franker about its gaps.

It reports total citations, the pages cited, and the key phrases the AI used when retrieving your content. That last one is the closest thing to a query anybody publishes.

Then Microsoft says this about it: "The data shown represents a sample of overall citation activity."

A sample. It does not say how large. It does not say how drawn. You cannot scale it.

Which is more honest than most vendors manage. It also leaves you unable to scale the number up. How would you?

Both reports are worth turning on. Neither is a measurement system. They are two windows into two engines, and there are more engines than two. Five, at least.

What an impression is not

Read Google's definition once more, slowly.

Impressions count "how many times links to your site were shown." Links. To your site.

So what happens when a model names you and links nobody?

Nothing happens. It does not appear at all. Not in Google's report. And not in Bing's, which counts citations "displayed as sources."

Both instruments count the same object: a link. And Chapter 8 spent itself on the case that accumulated mentions are what a model learns you from.

Which leaves the most valuable outcome unmeasured by design. A model says your name, recommends you, describes what you do, and cites somebody else's review page as the source.

You were the answer. Somebody else was the citation. Our clients hit this constantly.

Neither first-party report will ever show you that. Nor will most vendor tools, which key on domains.

So keep two questions apart from here on. Was my page cited? Was my brand named?

They are different questions with different answers. Only one has an instrument.

Cited is not the same as correct

One more thing a citation does not tell you.

An early study of four answer engines checked whether cited sources actually supported the sentences attached to them. Only 51.5% were fully supported.

That work is from 2023 and the engines have improved. The shape of the risk has not changed.

Being cited says a system reached for you. It does not say the system read you correctly, and it certainly does not say the sentence next to your name is true.

Which is worth an hour each quarter. Ask the four big engines about your own category, read what comes back, and check whether the claims attached to you are ones you would make.

That check is manual. It is unscalable and irreplaceable. No dashboard does it for you.

The referral you cannot count

Traffic should be the easy part. It is not.

In July 2025 Cloudflare published crawl-to-referral ratios per AI platform. The spread across one week was six orders of magnitude. At one end, about 70,900 crawls for every referral. At the other, ten referrals per crawl. Same week.

Then it added a caveat that undoes more than it admits.

Read the last clause again. It is unclear by how much.

That is the company sitting on a fifth of the web. Saying it cannot size its own blind spot.

So the mechanism is simple and the consequence is not. A person reads about you in a phone app, opens your site later, and arrives with no referrer. Your analytics files them under direct.

They are not direct at all. They came from an answer you never saw, about a question you will never know.

OpenAI mitigates this in one place. Links rendered in the ChatGPT web product carry utm_source=chatgpt.com. A convention, not a commitment.

No engine guarantees a referrer. Not one has published such a commitment.

So treat every AI referral number you own as a floor. Never as a count.

What the vendors say in their own documentation

We expected to have to argue this section. We did not, because two of the largest vendors have already written it down.

Ahrefs publishes a methodology page for Brand Radar, and it is the most candid document in this category. Start with what it discloses. Prompts run "through the free, publicly available web interfaces" of the engines, not through an API. It lists monthly query volumes per platform too.

That distinction matters more than it looks. An API answer is not the product answer. Different system prompt. Different retrieval. Different everything.

Semrush says the same about itself. Their words: "Prompt responses are captured from real requests and not via any APIs of LLMs."

Credit both. It is the single most load-bearing fact about how a vendor collects, and most never mention it. Does yours?

Then Ahrefs says something we have not seen another vendor say.

Read what that is. A vendor telling you its flagship number is modelled, and naming the assumption underneath. And it goes further, calling its metrics "directional indicators, not exact traffic counts."

Semrush concedes the general case on its own help page. Their sentence: "AI search and LLM responses are fast-changing and highly personalized, which means no platform can provide exact numbers on visibility."

No platform can. Their words, about their own category, published on their own help page.

So this chapter is not the one making that claim. Two of the biggest vendors got there first. They wrote it in their own documentation, where almost nobody reads.

One more Ahrefs disclosure, because it changes how you read any citation count: it does not filter hallucinated links. Their reasoning is defensible, that malformed output "reflects real model output."

But it means a citation in your dashboard may point at a URL that never existed. Have you checked?

Traced to source

Five numbers you will be quoted. Here is where each one actually comes from.

Row two is the more interesting one, because somebody finally ran the experiment.

A June 2026 field study took one high-traffic domain. It applied a defined bundle of optimizations to a subset of pages. Then it used the untouched remainder of the same site as a control.

That design is right. The untreated pages absorb the tailwind. What is left over is yours.

Treated pages grew 5.7 times. Untreated pages on the same site grew 3.5 times. Read those together.

Their interrupted time-series estimate of the real effect: 1.82 times.

So the intervention did something. The headline number was mostly the tide coming in. And then the authors do the thing almost nobody does: they run a placebo test on their own result, and report that it fails at p equals 0.16.

Their word for their own finding is "suggestive, not conclusive."

That is the best evidence this field currently has that GEO work moves anything. One domain. One bundle. One outcome that is referral traffic rather than citations, and a placebo test the authors could not clear.

The effect is probably real. You should still know that is the state of the evidence before you build a business case on it. All of it.

The four things you can actually count

Enough demolition. Here is what survives contact with the evidence, and it is shorter than you want.

All four are first-party. None needs a subscription. And none tells you what you wish it told you.

The one number worth arguing about

If you keep only one thing from this chapter, keep the ratio.

Count the crawler hits you get from AI operators. Then count the referral sessions they send you.

Divide the first by the second. That is your crawl-to-refer ratio.

Then write it down somewhere you will look again.

Cloudflare publishes the same measure across its network, so you have a benchmark. And the spread there was enormous: about 70,900 to one at the extreme.

Why does this one matter more than a visibility score?

Because both halves are yours. Both come from your own logs. Neither depends on a vendor prompt set, a sampling method you cannot see, or an engine choosing to report.

It also asks the commercial question directly. How much are you being read, against how much are you being sent?

A ratio that widens means more extraction and less return. That is a real business fact, and it is measurable today.

It will not tell you whether your content is good. It will tell you whether the trade is worth making.

What this costs

Almost nothing, which is the uncomfortable part.

The four counts above are a half day of setup and an afternoon per quarter. Log parsing you already have. Two engine reports you switch on. Two searches you run by hand.

The visibility subscription is the line item to examine. They run from a few hundred to several thousand a month. The honest question is not whether the tool works.

It is which decision the number changes. Anything?

If the answer is a slide, you are buying a slide. If it is a prioritization you would otherwise get wrong, that may be worth real money. Which is it?

Ask the runs-per-prompt question first. The answer tells you whether you are buying a distribution or a snapshot.

If you are already doing it

Three quick corrections for teams already tracking something. All free.

Stop reporting a single number. Report a range, or report the count of runs behind it. A visibility score with no distribution is a point estimate. On a distribution the researchers measured as unstable within minutes.

Check your denominator. If more than half of ChatGPT runs cite nothing, then a share of cited runs and a share of all runs are wildly different numbers. With the same name on the chart.

Then separate the two claims you are making. That you appeared is one claim. That appearing did anything is a second claim. And no published study has established that one under experimental control.

What to tell your board

This chapter creates a problem you have to manage upward. So here is the script.

Do not say measurement is impossible. It is not, and it sounds like hedging.

Say this instead: the category has two engines reporting first-party data, both of them partial, and a research base that says single-point measurements are unreliable. Then show the four counts.

Give them the direction and the confidence separately. Crawler hits and off-site counts are solid, so state them flatly. Referral numbers are floors. Label them. Engine reports cover two engines, so name which two.

And put a number on what you do not know. Not a guess, a gap: three of the five engines that matter publish nothing to publishers at all.

Then the sentence that will save you a quarter of grief. Appearing and mattering are two claims, and the second one has never been demonstrated under experimental control by anybody, including your competitors.

Somebody in that room is being sold a dashboard that implies otherwise. Better they hear it from you.

Two more, if you have budget and patience.

Log the full distribution, not the daily winner. If your tool exposes individual runs, keep them. A month of runs answers questions a month of averages cannot.

Then run one honest control. Leave a comparable set of pages untouched for a quarter. It is the cheapest experiment in this book and almost nobody runs it, which is why the evidence base looks the way it does.

The objections this chapter has to answer

"You spent a chapter saying measurement is impossible, then sold a measurement stack."

Not impossible. Just narrower than advertised. The four counts are real, first-party and cheap. None of them is a visibility score, which is the thing this chapter says cannot be delivered.

"Our vendor's numbers move consistently with our campaigns."

They might. They would also move without your campaigns, which is the entire point of the June 2026 study. Treated pages grew 5.7 times. Untreated pages on the same domain grew 3.5 times. Without a control you cannot tell those apart, and almost no case study has one.

"The academic work is preprints and small samples."

Some of it, yes. But the volatility study is four engines over forty-six days, the variance study is 12,933 responses, and the survey covers forty-five studies. It is the best evidence available. If you have better, we would like to read it.

"You are describing August 2026. This will change."

It already has, twice this year. Bing shipped citation reporting in February and Google shipped impressions in June. Re-read Table 11.2 before you quote it.

Then the disclosure. We sell GEO services, and this chapter tells you the category's central metric cannot be measured. That is either integrity or a strange sales strategy. Judge it by whether the four counts work when you run them.

What would change our mind

Three things, in order of how much they would matter.

A randomized study. Same site, matched pages, one variable, citations as the outcome. The June 2026 design is close and measures referral traffic instead.

An engine publishing clicks. Google gives impressions. Bing gives citations. Neither closes the loop to a visit, and one of them could.

A vendor publishing its prompt set and its runs per prompt. One doing it forces the rest. The category improves in a quarter.

What is left

So the ledger for measurement, in four lines:

The instrument is less stable than anything it measures. And the instability is stochastic, not seasonal. Not fixable by you.

Two engines now report first-party, one with impressions and one with citations, and three report nothing.

Every referral number you hold is a floor. Native apps send no referrer, and nobody can size the gap.

The causal claim underneath this entire industry rests on one controlled study. Its own placebo test does not clear significance.

Which is the honest position. It is not a comfortable one to end a book on, so we will not end here.

Eleven chapters have handed you a mechanism, a set of artifacts, and a measurement stack you can defend. What is missing is an order to do them in. Just an order.

That is Chapter 12, and it is ninety days long.