How do you choose an AI visibility tracking tool in 2026? The 12-criteria grid

Twenty platforms have been telling the same story for eighteen months: your customers ask ChatGPT questions, and your brand is nowhere in the answer. The diagnosis is correct. The trouble starts when you reach for your card, because every available comparison hands you the same four selection criteria: how many engines, how many prompts, which metrics, what price.

Those four criteria are easy to check. They are also the least useful. Two tools with identical spec sheets can produce scores forty points apart on the same brand and the same prompt, because they do not query the models the same way.

I sell an AI visibility tracking tool. Let me get that out of the way: I co-founded Cockpyt AI, so I have an obvious interest in you picking mine. The grid below is built to be used against Cockpyt AI as readily as in its favour, and I will show you at the end of this article which criteria my own tool fails.

How do you choose an AI visibility tracking tool?

You choose an AI visibility tracking tool by scoring twelve weighted criteria, eight covering measurement reliability and four covering functional scope. A tool scoring below 60 out of 100 produces numbers you will not be able to defend in front of a client or a board.

Here is the grid. It is deliberately scored so you can compare two quotes side by side instead of comparing two sales pitches.

The four scope criteria, the ones every comparison publishes, sit at the bottom of the table. They total 25 points. The eight reliability criteria, which almost nobody asks a vendor about before signing, total 75.

The 4 visible criteria, and why they fall short

Public criteria are elimination filters, not decision tools. They tell you whether a tool is out of the running. They never tell you whether it is good.

Engines covered. Counting engines is the most expensive false debate on this market. A tool tracking nine models when 85 % of your audience uses ChatGPT bills you for eight decorative dashboards. The question that matters is whether Mistral is covered for a French-speaking B2B market, and whether Google AI Overviews are included, live in France since July 2026.

Prompt volume. The number on the pricing page means nothing on its own. A 50-prompt plan testing each prompt once a week on one engine yields 200 responses a month. A 15-prompt plan tested five times a week across three engines yields 900. Always multiply prompts, variations and engines before comparing two prices.

Metrics. Almost every tool advertises mentions, citations, share of voice and sentiment. The word that counts is often missing: recommendation. Being named in a list of ten players is not the same as being singled out as the best choice. A tool that merges those two states into a single score leaves you flying blind.

Price. Entry tickets start around 29 dollars a month for lightweight trackers and climb to 295 dollars a month for premium platforms. In its June 2026 comparison, Justa notes that more than twenty platforms are competing around the same narrative. Price alone says nothing. Price divided by responses generated says a lot.

Engines actually included. The advertised coverage is almost never the coverage you subscribe to. Several vendors announce six or seven engines and include only three per plan, the others being billed as add-ons. At Hikoo, Perplexity only arrives on the €139 excl. VAT annual tier, the first three tiers covering one or two engines. At Otterly, Gemini, Google AI Mode and Claude are sold as add-ons on every plan. Always read the “included” line before the “available” line, and recalculate the price at identical engine scope.

The 8 reliability criteria vendors do not publish

The reliability of an AI visibility tracking tool rests on its model querying protocol, and that protocol is rarely documented. L’Agence WAM made the point back in January 2026: most solutions reveal only part of their methodology, without specifying which prompts, which models or which weighting. Here are the eight questions to put to a vendor in writing before you sign.

1. Raw API or interface simulation?

A tool querying the OpenAI API measures a model’s response under laboratory conditions. A tool simulating a browser measures what a real user sees. Both measurements are legitimate. They are not comparable. LLM-GEO.fr documented this in April 2026, pointing out that most tools never specify which of the two they perform.

In my view, this criterion should be disqualifying. A vendor who will not answer it in writing is selling a score, not a measurement.

2. Is web search enabled?

An API call without a search tool enabled returns what the model retained from training, several months out of date. The same prompt with search enabled returns an answer built from pages crawled that day. A tool measuring the first case will tell you your new brand does not exist, while ChatGPT has been naming it correctly for three weeks.

3. How many repetitions sit behind one tracked prompt?

Language models are non-deterministic. The same question asked twice produces two different answers. A study published by SparkToro in January 2026 ran 12 identical prompts nearly 3,000 times through 600 volunteers: the odds of getting the same list of brands twice came out below 1 in 100.

So ask for the number of actual calls per prompt per cycle, and get it in writing. A single pass does not support a conclusion on an isolated question, whatever the measurement frequency.

The useful threshold is lower than people claim. Research published in September 2025 on the reliability of model evaluations shows that two repetitions remove about 83 % of the ranking flips produced by a single pass, and that going from one to three repetitions only cuts the standard error by about 5 %. Beyond that, returns collapse. A tool selling 30 passes a day is mostly selling compute cost.

Watch out for a common confusion, one I make myself if I am not careful. Repeating an identical prompt measures model variance. Varying the wording of a question measures how robust your visibility is to the ways people phrase the request. Both are useful, they are not the same exercise, and a vendor who answers “we test three variations” has not answered the repetitions question.

4. Which questions is the score calculated on?

This is the heaviest criterion, and the most recent. Two families coexist and produce numbers that cannot be compared.

Aggregated-database tools build their own prompt set. Ahrefs starts from its keyword database, extends it through “People also ask” and semantic expansion, then queries the platforms and stores the answers. Semrush relies on a database of more than 300 million prompts drawn from clickstream data and its keyword set. You then query that database for your brand.

Declared-question tools measure the queries you enter yourself, and only those.

Both approaches are legitimate and serve different needs. But a brand can show up well on its sector’s generic questions and never on its ten real commercial queries. Ahrefs documents this honestly: its index works best for established brands with significant search demand, and coverage can be limited for brands that are rarely searched. For a small or mid-sized business, that changes how you read the number.

Ask the question in one sentence: is this score calculated on my questions or on yours?

5. Is the score composition published?

A score out of 100 is not a measurement, it is a formula. And formulas diverge far more than people imagine.

• HubSpot’s free diagnosis gives up to 40 points out of 100 to sentiment, 20 to awareness, and only 20 in total to exposure and competitive position.
• Semrush calculates a relative score: your mention frequency divided by the median of competitors the tool identifies itself. It can therefore rise because your competitors fall back.
• The Cockpyt Score combines presence, position and the presence of a link, and drops to zero without a citation.

The version of the queried model matters just as much, and almost nobody publishes it. HubSpot documents its own, including Gemini 2.0 Flash, a lighter version different from the model served by default in the Gemini app.

In my view, a vendor that publishes neither its weighting nor its model versions is selling a number you cannot interpret, so cannot defend in a meeting. That is reason enough to rule it out.

6. Are mention, citation and recommendation kept separate?

Three very different states hide behind the word visibility:

• Mention: your name appears in the answer, with no link and no favourable context.
• Citation: your URL is referenced as the source of the information.
• Recommendation: the model names you as the answer to the stated need.

A tool that adds these three states into a single percentage produces a flattering, unusable indicator. You will know you moved from 30 to 45 %, without knowing whether you gained recommendations or plain mentions inside a list.

7. Is the data replayable?

A score with no raw response behind it cannot be verified. You need to open any data point, read the full text generated by the model, and see the date, time and exact prompt sent. Without that trace, you can neither challenge a number nor defend it in a meeting. The Media Leader framed this well in April 2026: media audience studies also rest on samples, but those samples are standardised and audited, which this market has not achieved yet.

8. Is real traffic tied to the measurement?

Knowing you appear in 40 % of answers says nothing about visits generated. A tool connecting to neither Google Analytics 4 nor Search Console leaves you with an awareness metric disconnected from the business. This is the point that sinks most GEO results presentations at board level.

The 20-minute reproducibility test

You can verify a tool’s reliability yourself during its free trial, with no technical skill required. The protocol runs to five steps and takes twenty minutes.

Step 1 – Pick a commercial prompt

Choose a purchase-intent query from your market, along the lines of “best invoicing software for small businesses”. Avoid branded queries, which are too stable to reveal anything.

Step 2 – Run three on-demand scans

Trigger the same prompt three times, at three different points in the day. If the tool offers no manual scan, log that as a replayability failure.

Step 3 – Record presence and rank

For each run, note whether your brand appears and where it sits in the answer. Record the first three competitors named as well.

Step 4 – Calculate the spread

Subtract the lowest score from the highest across your three runs.

Step 5 – Interpret

A zero spread across three runs signals caching: the tool is serving you the same answer. A spread above 30 points signals sampling too thin to produce a trend. A spread between 5 and 20 points matches the normal behaviour of a model queried properly several times.

Finish by asking the same prompt manually inside the engine’s own interface. If the manual result bears no resemblance to the tool’s, you have just discovered that the vendor measures an API with no web search.

The 4 tool families and who they suit

The market splits into four families, and choosing a tool starts with choosing a family. This segmentation, proposed by the agency Justa in June 2026, remains the most workable to date.

Full-stack platforms cover monitoring, analytics and content production in one interface. Broad coverage, average depth on each component, entry tickets between 99 and 300 dollars a month. Suited to marketing teams with no existing stack.

Pure trackers handle monitoring and citations well, with no optimisation layer. Good cost per response, but you see the problem without knowing what to fix. Suited to teams that already run SEO and content in-house.

Modules bolted onto an SEO suite spare you an extra subscription. Clean integration with your existing tooling, prompts often fixed, limited model coverage. Suited to heavy users of a suite they already pay for.

Repositioned content generators produce LLM-optimised text without measuring anything. A useful complement, never a substitute for a tracker.

In my view, the one structural mistake is buying a full-stack platform without someone dedicating at least two days a week to working the data. A dashboard with no operator is a fixed cost line, not a management tool.

The 5 mistakes that lead to the wrong purchase

• Buying before writing your prompts. The relevance of your tracked queries determines the value of the measurement. With no list of thirty strategic prompts, you are not ready to subscribe, you are ready to run a workshop.
• Counting engines instead of counting responses. Nine models queried once are worth less than three models queried five times.
• Confusing presence with recommendation. The first metric climbs easily. The second is the one that generates leads.
• Accepting a proprietary score without its formula. Ask what the score aggregates and with what weighting. A refusal is an answer.
• Ignoring the state of your site. A measurement tool pointed at a poorly structured site with no authority produces data, not results.

Full disclosure: where Cockpyt AI lands on this grid

Cockpyt AI is the tool I co-founded with Laurent Séjourné. Here is its honest run through the grid above.

What it clears. Measurement under identical conditions, with no logged-in account, with web search enabled, and an indication on every recorded answer of whether the engine used web search or its memory. Questions declared by you, so a score calculated on your commercial queries. Cockpyt Score composition published. ChatGPT, Gemini and AI Mode tracked every day on all four plans, AI Overviews every week on your keywords, with no per-model surcharge. Raw responses viewable, Google Analytics 4 and Search Console connections, and an on-demand GEO audit with a prioritised action plan, run with the AI credits included. Pricing starts at €20 excl. VAT per month on the annual plan (€24 monthly), with unlimited projects and users and a 14-day trial with no credit card.

What it fails. One pass per question per day, where the grid recommends at least two repetitions. The headline score is therefore read as a 7-day average: on the Solo plan, it aggregates 420 observations a week (20 questions, 3 engines, 7 days), which makes the trend readable, but a single day’s reading on one question should not be over-interpreted. It is a real limitation and I do not work around it.

What is also missing. Mistral, Claude and Copilot are not tracked, and Perplexity is a paid add-on. Projects are unlimited, but volume stops at 420 questions on the Scale plan: beyond that, a quote-based platform will serve you better.

Frequently asked questions

Does an AI visibility tracking tool replace an SEO tool?

No. The two measure different surfaces: your position in a results page on one side, your presence inside generated text on the other. A GEO tool reports no search volumes, no backlinks and no technical indexing errors. Budget for both, not one.

How many prompts do you need for a reliable measurement?

Count on 20 to 30 prompts for a single-product business, and 80 to 150 for a catalogue or a multi-segment market. Beyond that, the difficulty is no longer volume but quarterly review: your prospects’ queries evolve, and a frozen list loses relevance within six months.

Why do two tools give different scores for the same brand?

Because they do not measure the same thing. Five causes add up: the questions measured, the score weighting, the engines and versions queried, the number of repetitions, and the intrinsic variability of the models. Two tools can show 62 and 18 for the same brand on the same day without either being wrong. The five causes and the comparison protocol are covered in a dedicated article.

Can you measure AI visibility for free?

Yes, at small scale. Manually querying ten prompts each month across ChatGPT, Gemini and Perplexity, then logging results in a spreadsheet, costs two hours a month and gives a usable trend. Automation pays off beyond thirty tracked prompts.

Do GEO tools cover Mistral?

Rarely. The model most relevant to part of the French-speaking B2B market is absent from the majority of mainstream platforms. If your audience is French and technical, ask before subscribing. The question will eliminate several candidates.

How often should you measure AI visibility?

Daily measurement read as a rolling 7-day average gives the most stable trend. A weekly reading is enough for light monitoring. Either way, do not react to an isolated reading: model variability moves a number from one day to the next without anything having changed.

Does a small local business need a tool?

Only if you have identified local commercial-intent queries where your competitors appear. Start by manually testing five queries along the lines of “best [trade] in [city]”. If you are absent from all of them, a tool becomes useful for tracking the fix.

Sources

• Rand Fishkin and Patrick O’Donnell, New Research: AIs are Highly Inconsistent When Recommending Brands or Products, SparkToro, January 2026: sparktoro.com
• Berk Atıl et al., Non-Determinism of “Deterministic” LLM System Settings in Hosted Environments, Association for Computational Linguistics, December 2025: aclanthology.org/2025.eval4nlp-1.12
• Miguel Angel Alvarado Gonzalez et al., Do Repetitions Matter? Strengthening Reliability in LLM Evaluations, arXiv:2509.24086, September 2025: arxiv.org/abs/2509.24086
• Justa, “Les meilleurs outils GEO et AI visibility en 2026 : comparatif et framework de décision”, Benoît Eveillard, June 2026: justa.fr
• LLM-GEO.fr, “Outils de tracking GEO : Peec.ai, Profound, Otterly et les vraies questions sur la précision”, April 2026: llm-geo.fr
• L’Agence WAM, “Outils GEO : promesses, biais et méthode pour mesurer votre visibilité”, January 2026: agence-wam.fr
• The Media Leader FR, “Comment les outils de mesure GEO accèdent aux réponses des LLM (et leurs limites)”, April 2026: fr.themedialeader.com