What to look for in an AI visibility tool, from a team that runs them daily
Our checklist for choosing an AI visibility tool: engine coverage, prompt sourcing, repeatability, citations, crawler logs, attribution, action and plan fit.
We get asked this by clients who are about to buy and by teams who bought something and are not sure it was right. After running programs on Promptwatch, here is the checklist we would use ourselves.
Engine coverage, and how engines are checked
Write down where your buyers ask questions. Then find out how each engine is queried. A live interface and an API call can give different answers, and a good vendor says which one you get. Promptwatch monitors ChatGPT (Search), Google AI Overviews, Google AI Mode, Microsoft Copilot, Gemini, Perplexity, Amazon Alexa and Meta AI through the real UI. Claude, DeepSeek, Mistral and Grok are tracked through APIs, so we call those directional when we report. Every paid plan includes all active models, and the free Explore plan is ChatGPT only.
Where the prompts come from
This is the question vendors dodge. Prompts may be yours, seeded from your data, generated by the vendor or drawn from a panel of real prompts. Panels are rare and enterprise-priced. For most clients we build prompts ourselves, starting from Prompt Explorer, which suggests prompts from Search Console, ranking keywords, People Also Ask or any page. Tags, topics, intent and personas keep the panel organized.
Repeatability
Answers change between runs. We want daily runs and a trend, not a snapshot. Monitors run per model, country and language. We read direction over weeks, and we spot-check by hand each month.
Metrics you can explain
A single score does not survive a client meeting. We use Visibility Score, Share of Voice, Sentiment and Position in Answer together. Sentiment is a 1 to 100 score per response, with a weekly AI summary. The Brand Book, brand aliases and analysis instructions decide what counts as a mention, which keeps counts consistent over time.
Citations
If the engine named someone else, we need to know which page it used. Citation analytics gives page and domain level data, URL positions, a Page Tracker for specific URLs, offsite mentions and Reddit, YouTube and LinkedIn reports. Citation Trends classifies each citation by content type and source type, with ramp-up days, peak rate and decay half-life per page.
Crawler logs
This one is a deal-breaker for us and a rare feature among trackers. IP-verified logs of real agents, including GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot and PerplexityBot, with explained errors. If a bot cannot fetch the page, content work comes second. They start on Professional at $245/mo. Essential ($95/mo) has none, which catches buyers out.
Attribution
Do AI-referred visits arrive, and do they convert? Visitor analytics installs via a script, a GTM template or the WordPress plugin. Reports to clients go much better when they include visits.
Action
Findings need owners. The Action Board prioritizes work, a Discovery agent suggests up to five evidence-backed actions per week, and tasks take assignees and comments. Content gap analysis and Content Refresh support the content side. On Business and Enterprise, the Content Agent plans, writes and publishes through a review inbox, and we keep the review step on.
Pricing model and plan fit
Compare the cost of a realistic panel, not the entry price. Credit-metered tools count one prompt on one engine as one unit, so wide panels spend quickly. For Promptwatch, Essential is $95/mo (50 prompts, 6,000 responses), Professional is $245/mo (150 prompts, 18,000 responses, crawler logs) and Business is $579/mo (350 prompts, 42,000 responses, Content Agent, state and city tracking, Ads Radar). Agency plans are $199, $399 and $799/mo, with unlimited projects and prompts, 10 seats, crawler logs and visitor analytics, but not the AEO articles, knowledge base or automated Content Agent.
Data access and security
Exports, scoped API keys, a hosted MCP server and REST API let us automate reports. For clients with procurement teams, Promptwatch is ISO/IEC 27001 certified and GDPR compliant.
Red flags we have learned to look for
- No clear answer on where prompts come from.
- Only a score, with no way to open the underlying responses.
- No crawler or traffic view, so a drop has no explanation.
- A promise of a guaranteed recommendation.
How to test before buying
Bring ten real buyer prompts and one competitor. Ask for the answers, the cited sources, a look at crawler access for key pages and the AI-referred visits. Use the free Explore plan (10 prompts, 310 responses) or a 7-day trial on monthly Essential, Professional, Kick-off or Growth.
The tool we pick is the one that answers all four. That is why our programs run on Promptwatch, and why we say plainly that a tracker alone is a reporting tool, not a program.