Your shopping cart is empty!
Two AI search trackers gave one brand 41% and 23% visibility
Publication date: 18.09.2026
In short
- Mike King of iPullRank writes that AI search has no single "correct" visibility value a tracker could be checked against.
- Model answers are random, so a single prompt run guarantees nothing; reliability comes only from the number of repeats.
- Two trackers can report 41% and 23% for the same brand, and both will be "right" in their own way.
The situation Mike King describes at iPullRank is familiar to anyone who has bought such tools.
A client runs two AI visibility platforms. One says the brand appears in 41% of the important prompts, the other says 23%.
Someone in marketing has to put a number in a board deck, and the question of which one is correct comes up every week.
±44points is the 95% confidence interval for a tracker that runs a prompt once; fifty runs squeeze it to ±14.
Why "accuracy" does not exist here
Classic rank tracking has a ground truth: the page is fourth or it is not. AI answers have no ground truth.
The model answers at random, adjusted for the user's context, the model's parameters and the moment it reached out to search. King suggests treating it like an opinion poll: nobody asks whether a poll is "accurate", they ask whether the method is stable and what sample it rests on.
The figures he cites illustrate that well. In Mike Saunders' research, out of a hundred ChatGPT runs only about 11% of brands appeared in 80% or more of the answers to a prompt.
An earlier study found that 72% of the brands mentioned turned up in fewer than one answer in five. Semrush found only 25% domain overlap between ChatGPT's Instant and Thinking modes.
Method choices that break comparison
The gap between two tools is often explained not by an error but by the different choices their authors made.
| Choice | What gets measured |
|---|---|
| API or scraping | Model behaviour in controlled conditions versus what the user sees in the interface |
| Logged out or logged in | A clean baseline versus personalisation that account history can contaminate |
| One run or many | An unreliable yes/no versus a share with a calculated margin of error |
In other words, two trackers are two different instruments measuring two different distributions.
Comparing their numbers is as pointless as adding degrees Celsius to degrees Fahrenheit. We looked at a similar picture in our text on AI search metrics worth tracking: without shared counting rules, numbers stop being metrics.
What to ask a vendor and what to do with your own tracking
King advises asking at the demo about the collection method, account state, model selection, query geography and what exactly counts as a "mention". Ask for a changelog of the methodology and for raw answers.
If you measure it yourself, freeze the configuration, run enough repeats to see a real business effect, and line visibility up against referrals, conversions and server logs.
Change the method mid-year and the trend is broken; comparing with last quarter no longer works.
For the Ukrainian market there is a further complication the original does not cover. Most trackers collect answers from English-language or US IP addresses, while your customers ask in Ukrainian and often from a phone.
So before a visibility figure goes into a report, find out which language and which country it was collected from. We wrote about measuring this in practice in our guide to AI search visibility.
In that situation I wouldn't choose between 41% and 23% at all. I'd ask both tools for the number of runs and the spread. A single number without a spread means nothing in this field.
I hope that next time someone shows a "41%", they will also be asked about the confidence interval. My bet is that most vendors will answer by reaching for a slide about something else.
