Methodology
How the number is produced
A score is only worth having if you can see how it was made and repeat it. This page is the whole method, including the parts that are inconvenient for us.
The measurement, end to end
Five steps, in this order, every time
Define
We read your website and write down the questions your buyers actually ask when they are deciding what to buy and have not heard of you yet. One rule governs the set: a question belongs in it only if your business could plausibly be a correct answer. A question you could never win is not a finding, it is padding that makes your own score look worse than it is. You see the whole set and cut anything you disagree with.
We also ask you for a rough split of the markets you sell to — for example India 40%, US 20%, UK 40%. It does not need to be exact. The same question asked from two countries returns different companies, so that split is what decides where we ask from, and it is recorded so next month’s run matches this one. It is the one thing we will not assume: a market we guessed at produces a measurement of buyers you do not have.
Query
Each approved question is put to the chatbots with web search turned on, asked the way a buyer would ask it. We keep the complete answer and the address of every page the chatbot read before it answered. Nothing is summarised at this stage, because a summary cannot be audited later.
Measure
From those saved answers we count every company named, the order it was named in, and every page cited. Being named in passing and being recommended are counted separately, because they are not the same event.
Analyse
We look at who was named instead of you, which pages put them there, and which questions named nobody at all. That last group is usually the cheapest ground to take, and it is what we write for first.
Track
The same frozen questions are asked again the following month, and the report shows what moved. A single run tells you where you stand; the second and third tell you whether anything you did worked.
The question set
Frozen, and fingerprinted
Once you approve the questions we freeze them and record a fingerprint of the set. Every later run is checked against that fingerprint before it starts. If a question is ever added or changed, the report says so on its front page and marks the comparisons the change affects.
This is the whole difference between a measurement and a demonstration. A tool that quietly rewords its questions between runs can show you any trend you would like to see.
| What is fixed | Why it has to be |
|---|---|
| The questions | Reworded questions produce different answers, and the difference would read as progress. |
| How many times we ask | The same question can come back slightly differently. Asking more often gives a steadier number. |
| Which chatbots | Each one reads the web differently. Adding one halfway through the year breaks the comparison. |
| The counting rules | Named is named. A passing mention is not a recommendation, and the report keeps them apart. |
| The country we ask from | An answer depends on where the buyer is sitting. Every report states which country its questions were asked from. |
How often we ask
More runs, steadier number
Asking a question once tells you what one answer looked like on one day. Asking it nine times, across three chatbots, is what turns a single answer into a measurement.
Tracking and Growth run the same measurement. Growth costs more because the month's writing comes with it, not because the check is deeper — we would rather say that than imply you are buying a better number.
The free check is a real measurement, not a teaser: you get the whole report. It is simply a thinner one, and it says so on its front page.
The part most people skip
A report that disagrees with its own data will not print
Every figure written into a report carries a marker naming the record it came from. Before a report is delivered, a separate checking step re-reads the saved answers and recomputes each of those figures. If a number in the text and the number in the record disagree, the report fails and is not sent.
This is not a flourish. On a client report, that check caught four wrong figures before they reached the client. In the published wood furniture study it missed one — a company counted under two of its own names — which we found later and corrected in public rather than quietly.
What you can ask for
The evidence behind any number
The raw answers
The complete text of every answer we recorded, not an extract. Any figure in your report traces back to the words that produced it.
The pages cited
Every web page the chatbot read, listed at the back of the report, so you can open them and judge them yourself.
The question set
The frozen set with its fingerprint, so you can confirm this month's questions are last month's questions.
What we left out
The questions we considered and deliberately excluded, and the reason for each. It is the first thing to re-read when a score looks wrong.
Limits
Where this method stops
No method is worth reading without its limits. Ours are written out in full — what nobody can control, why one report is a snapshot, how we reach the chatbots, and why the country we ask from changes the answer.
See it run on your own business
Send us your website address. We will write the questions your buyers ask, ask them, and send you the whole report — not a teaser.