AI visibility tool evaluation checklist: 10 questions to ask before you buy

How to use the checklist

Start with a shortlist. Our AI visibility tools comparison lines up prices, engines and prompt limits for 22 trackers. This checklist is for the next step: finding out what the numbers behind those limits are made of.

Download the checklist as CSV; the same ten questions are also in the checklist table after question 10. Each row is one question, with why it matters, how to test it during a trial and what a good answer looks like. Three empty columns are for your shortlist. The fourth is filled in for Llamat, so you can see what a plain answer looks like, including where the answer is no. Most rows can be answered from the pricing page, the methodology or a short e-mail to the vendor. The rest need a trial and five questions your buyers really ask.

1. How many times is each prompt asked?

An AI engine writes every answer anew, so one run of a prompt is one sample, not the answer. On 1 October 2026 we asked logged-out ChatGPT four real Google searches about AI visibility tools, three times each. Each search got the same first tool in all three runs, but the lists below it moved. Counting each search separately, 31 tools were named at least once: 23 of them in all three runs and 8 in only one or two. For “ai visibility checker free”, the second run named four tools and the other two runs named five.

A tool that asked once would have reported whichever of those lists it happened to get. The raw answers are public; that first file misses one lower-case mention of IsVisible in the second run, which the counts above include. A tracker that asks once a day can still build a fair picture over a month; what matters is how many answers stand behind each number and how they are combined.

• Ask: how many times is each prompt asked on each engine per update, and are the runs combined as a share of answers or by majority?
• A good answer: a stated number of runs per prompt, engine and update, each in a fresh session, and a stated rule for combining them.
• Test it in a trial: ask one prompt by hand three times, each in a new chat, on a day the tool runs it, and compare the brands you saw with what the tool shows.

2. Does it read the app people use, or an API?

The same question can get different answers depending on where it is asked. An API call goes to a model. The chat app adds its own instructions, web search and interface, and a signed-in user adds personal context: OpenAI's Memory help page, read on 3 October 2026, says that when Memory is enabled ChatGPT “can remember relevant preferences and details from your chats and other available sources”.

So API answers, logged-out app answers and signed-in answers are three different measurements. None of them is wrong; they describe different audiences. Some vendors say which one they use. SE Visible states that it collects “real AI responses using a browser and graphical user interface, not simulated answers or API-based results” (read 3 October 2026). If a vendor does not say, ask.

• Ask: for each engine, are answers collected through an API, the logged-out web app or a signed-in account?
• A good answer: the collection method named per engine, so you know whose answers the numbers describe.
• Test it in a trial: open one stored answer in the tool and ask the same prompt in a private browser window on the same day. If the tool never shows the sources or the format the app shows, ask how the answers were collected.

3. Which country are the answers asked from?

Where the collector sits is part of the measurement. OpenAI's ChatGPT search help page, read on 3 October 2026, says ChatGPT “may use your IP address to estimate your general location, such as your country, state, or city”, and that a VPN or network location may affect it. Answers about shops, prices and local providers can change with that location.

Vendors publish their markets in different ways. On 3 October 2026, SE Visible listed seven countries (the US, UK, Canada, France, Germany, the Netherlands and Spain) and five languages. Profound's trial listed one language and one region. Ahrefs Brand Radar counts one check per prompt for each “platform, location and update”, so every extra country uses more of the plan.

• Ask: from which country is each engine asked, and is the country set separately from the language?
• A good answer: a stated exit country per engine that matches your market.
• Test it in a trial: ask a price or “where to buy” question for your market. Prices in another currency or shops from another country show where the answers were collected.

4. Does it separate mentions, recommendations and citations?

Three different things can happen in an answer. The brand is named (a mention), the answer suggests it as a choice (a recommendation), or the answer links a page as a source (a citation). A brand can be named in most answers and recommended in few, and a site can be cited without the brand being named. One blended score hides which of these is your problem. Our guide to AI visibility defines each one.

Ask how each one is detected, too. Matching names and links in the text is repeatable, but it needs the variants of a brand name. Deciding whether an answer recommends someone takes judgment, usually from a language model, and a model can be wrong.

• Ask: is there a separate rate for mentions, recommendations and citations, and which are matched by rule and which are judged by a model?
• A good answer: three separate rates, the detection method for each, and the answer text behind every count.
• Test it in a trial: read five answers the tool marked as mentioning you and five it marked as not mentioning you, then check one it marked as recommending you.

5. Where do the prompts come from?

The prompt list decides the result more than any other setting. Prompts come from three places: you type them, the vendor suggests them for your site or industry, or they come from a prompt database built from search data. On 3 October 2026, Ahrefs described its Brand Radar index as built from “real queries from its keyword database”, expanded into questions and run as more than 449 million prompts, and said the index may have limited coverage for brands with little or no search volume; its Custom Prompts are ones you define yourself. Profound's pricing page said trial users receive “a recommended prompt set based on their industry” and cannot customize it, while Enterprise users can edit, disable or add prompts.

Each source has a trade-off. Search phrases show real demand but are worded for Google. Invented prompts read naturally but may be questions nobody asks. A database is broad but rarely fits a niche. And a prompt that contains your brand name will mention you by design, which inflates the rate.

• Ask: where does each prompt come from, can I see every prompt, and can I edit, add and remove prompts on the plan I would buy?
• A good answer: every prompt visible with its source, mostly unbranded buyer questions in your market's language, editable on your plan.
• Test it in a trial: list the prompts and count how many contain your brand name. Take those out and see how much the mention rate drops.

6. What does a prompt, a check or a credit buy?

Plans count in different units, and the same number can buy very different coverage. On 3 October 2026, SE Visible counted the prompts you add, not the checks; by its own example, 200 prompts on five engines checked daily is about 30,000 checks a month. Ahrefs Brand Radar used one check per prompt for each platform, location and update, and eight checks for a Claude update. ZipTie charged $0.01 for “one prompt, one engine, one check”. AthenaHQ said “1 credit = 1 AI response”, and Rankscale said each query to an AI engine uses “a fraction of a credit (typically 0.25)”.

To compare plans, turn every unit into answers: prompts × engines × countries × runs per update × updates per month. Twenty prompts on three engines in one country, asked once a day, is 1,800 answers a month. Asked three times per update, it is 5,400.

• Ask: what is one unit on the plan, and do extra engines, countries, runs or updates use more of it?
• A good answer: a unit you can turn into answers, with the rate per engine and country stated and a calculator or worked example.
• Test it in a trial: work out the answers per month for your own prompt set, and after a few trial days compare it with the usage the dashboard shows.

7. Which engines does your plan cover, and how?

Engine lists on pricing pages often describe the top tier. On 3 October 2026, ZipTie's 7-day trial covered ChatGPT, Google AI Overviews and Microsoft Copilot, with Perplexity, AI Mode, Bing AI Overview and Gemini unlocking on a paid plan. Profound's trial covered ChatGPT, Gemini and Google AI Overviews, and its Enterprise plan up to nine engines.

The collection method matters per engine, too. An engine that requires sign-in cannot be measured logged out. When we asked Perplexity four searches three times each on 1 October 2026, logged out, none of the 12 runs got an answer: every attempt hit its sign-in wall. If a tool reports Perplexity answers, ask whether they come from an account, an API or the logged-out app, and from which country.

• Ask: which engines are on the plan I would buy, not the top tier, and for each one, is it an API, the logged-out app or an account?
• A good answer: a per-plan engine list with the collection method for each engine.
• Test it in a trial: compare the engines in the trial with the plan you would buy. A trial without the engine you care about cannot test it.

8. Can you export the raw answers?

Rates are only as good as the answers behind them. Without the raw answers you cannot check a count, re-analyse old data or keep your history if you switch tools. On 3 October 2026, ZipTie listed CSV, Excel and PDF export on every plan, with API and MCP access as a $10-a-month add-on. AthenaHQ listed CSV export on its Starter plan, and SE Visible a CSV export of its report. Profound's trial listed exports and API as “None”, with CSV, JSON and an API on Enterprise.

• Ask: does the export contain every answer — prompt, engine, country, run, time, full text and cited URLs — or only totals, and on which plan?
• A good answer: one row per answer in CSV or JSON on the plan you would buy, plus an API if you need one.
• Test it in a trial: export on day one and open the file before you rely on the dashboard.

9. Is the methodology published?

Two tools can report different rates for the same brand because they count differently: runs, countries, what counts as a mention, how position is averaged. A public, dated method is what lets you compare them or recompute a number. Ahrefs, for example, links a Brand Radar methodology post from its product page. How Llamat counts every rate is set out in our methodology.

• Ask: is there a public page with runs, collection method, countries, detection rules and formulas, and does it say what changed and when?
• A good answer: a dated public page with formulas you could apply yourself.
• Test it in a trial: pick one number on the dashboard and recompute it from the export with the published formula.

10. How often does it run?

Daily runs show change sooner, and they also show day-to-day noise; weekly or monthly runs cost less. On 3 October 2026, ZipTie sold daily, weekly and monthly prompt slots, with a weekly prompt costing a seventh of a daily one. Ahrefs let you pick daily, weekly or monthly per platform for Custom Prompts. SE Visible refreshed daily, and Profound's trial ran 50 prompts daily for seven days.

Runs per update and cadence together decide how many answers stand behind a trend. One run a day for 30 days is 30 answers per prompt and engine; a one-off report with three runs is three.

• Ask: how often is each engine asked, and can I set the cadence per prompt or project?
• A good answer: a cadence you choose, with the date and time of every run visible.
• Test it in a trial: watch the same prompts on every day of the trial. A rate that jumps while nothing changed on your side is noise, so judge by weekly averages, not single days.

Where Llamat falls short on this checklist

Filled in for our own product, the noes are easy to find. Llamat is not a daily tracker: the report is one round of answers, and its free dashboard repeats that round once a week, so a trend builds up one point a week. Each round asks every prompt three times on ChatGPT and three times on Gemini and searches it once for Google AI Overviews, which is a small sample next to a daily tracker. All three engines are checked through our own proxy farm with Czech IP addresses for every market: ChatGPT has no country setting, Gemini's country setting did not change the location it saw (in a test on 4 October 2026 with the country set to the United States, Gemini placed the visitor in Prague, Czechia), and the AI Overviews searches carry the market's country and language but leave from the same Czech addresses. The free report covers only these three engines, has no raw-data export, and picks its prompts automatically, with no way to edit them before the run. On the other side: every ChatGPT and Gemini run is a new logged-out session in the web app, mentions and citations are matched by rule, the prompts are written from real search phrases and all of them are listed in the report, and the method is public. The row-by-row answers are in the table below and in the CSV.