How we measure, and why a single answer never becomes a score

What "API with web search" means

We ask the engines the way a current answer is produced.

ChatGPT, Gemini and Claude are asked through their official APIs with web search or grounding turned on. That is the mode in which these engines look up current products and cite sources, and it is the only mode in which changes to your site can change the answer. Google AI Overviews is read from the live Google results page for the prompt.

Every run records whether the engine actually searched. A prompt the engine answers from memory is marked as such, because no page you publish will move it until the model itself is retrained.

Engine runs, spot checks and consistency tests

Three units, used the same way in the product, the reports and the pricing.

Two spot check misses in a row on a prompt that last tested STRONG or MODERATE queue a new consistency test on that engine. The alert comes from that test, never from the spot check.

The labels

The share of valid runs that named you, and what we do when runs fail.

A zero counts as INVISIBLE only when all runs completed and search fired. Zero mentions with errors, or zero mentions from an engine that answered from memory, are INCONCLUSIVE and are never charted as a drop.

When a mention counts

Your name in the answer is not enough on its own.

• Echoes are stripped first. When the prompt itself contains your name, an answer that only repeats the question does not count as a mention.
• The entity is confirmed. An answer that uses your name for a different company, product or domain does not count, and it is shown to you as an entity confusion, because that is its own problem to fix.
• Every answer is stored with its sources and engine version, so a mention can always be checked by reading the answer.

Trend lines stay honest

Controls, frozen prompt sets and engine versions on the chart.

Two control prompts per brand are never targeted by a fix, so a rise that also shows up on the controls is the engine changing, not your work. Prompt sets are versioned and frozen for a case, and each prompt keeps a stable key across versions. When an engine version changes, the chart marks the period so a model update is not read as progress. Results imported from earlier tools or earlier engine versions are drawn as their own segment.

The same method on our own brands

We publish what the method says about us, with dates.

We run this exact pipeline on our own SaaS brands and publish their scores with a dated log of every fix. The word proven stays out of our copy until one of those cases clears the success criteria we wrote down before the first test.