How can I fact-check what AI is saying about my brand? Our audit process
The audit process our team uses to fact-check what ChatGPT, Gemini and Perplexity say about a brand, trace errors to their sources, and verify the fix worked.
Almost every new client has a story about a wrong answer. A chatbot gave the wrong price, mixed them up with a competitor, or said they lack a feature they shipped last year. The instinct is to complain about the model. Our approach is calmer: treat it as an audit with a source of truth, a sample of answers and a trace back to where each error came from. Most errors turn out to live on web pages, and web pages can be fixed.
Before you test anything
We ask the client for their source of truth. Current pricing and plans, feature list, integrations, locations, founding details, and a list of things they do not do. Each point should link to the page on their site that states it. If a fact appears nowhere on the site, we note that, because it is a likely source of invention. Engines fill gaps.
Run the audit
We build a prompt set around brand questions: price, fit, comparisons, ownership, integrations, reviews. We add a handful of category questions where the client should appear. Then we run them across engines and save the full response text. Summaries lose the exact wording that makes a claim checkable.
Next we mark every checkable statement as correct, outdated, wrong or unverifiable. Typical findings from audits like this:
- Old pricing or plan names, usually lifted from an archived page or a third-party roundup.
- A competitor's feature credited to the client, or the reverse.
- Missing information, where the engine says nothing about a feature that is clearly documented, because the page is hard to find or not crawled.
- Location, date and ownership errors.
- Claims that sound plausible and have no source at all.
Trace the source
For each wrong claim we look at the cited sources for that response. The wrong number is often on a page we can name: the client's old pricing page, a two-year-old review, a forum thread. Query fan-outs help too, since they show which searches the engine ran to build the answer, and therefore which angle pulled in the bad page.
This is the step that changes the outcome. A wrong claim with a traceable source gets fixed. A wrong claim with no traceable source is flagged and re-tested, because it may be model memory that updates over time.
Fix, then confirm access
Fixes fall into three types. Update the client's own page, and redirect or remove the stale one. Ask the third-party publisher for a correction, with the evidence. Or publish a plain page stating the fact where none exists. After that we check whether AI crawlers can fetch the corrected page. If a bot is blocked or hitting errors, the engine keeps reading the old version, and no amount of rewriting helps.
The platform we use
We run client programs on Promptwatch, and each stage above maps to a feature. The Brand Book holds analysis instructions that decide what counts as a mention, with brand aliases for name variants. Monitors store every response, with a per-response sentiment score from 1 to 100 and a weekly AI summary. Sentiment measures tone, not truth, so we still read the text against the source of truth. Citation analytics gives page and domain level sources, and the Page Tracker lets us watch the stale URL we suspect. Query fan-outs show the engine's searches per model.
For the fix, Content Refresh proposes diff-style updates to existing pages. For crawl checks, Agent Analytics shows IP-verified AI crawler visits, errors and failure explanations per page, so we can see whether bots like OAI-SearchBot or PerplexityBot are reaching the corrected page. The Action Board keeps the work list, and A/B period comparison shows whether answers changed after the fix.
There is also a free Chrome extension, the GEO Inspector, which shows ChatGPT's fan-outs and the fetched versus cited sources. It runs locally, which makes it a good first step for a quick spot check before you set up anything formal.
Plan notes
Essential ($95 a month) stores responses and citations but has no crawler logs. Professional ($245) adds crawler logs. Business ($579) adds the Content Agent for publishing corrections. Agency self-serve plans (Kick-off $199, Growth $399, Scale $799) include crawler logs and unlimited projects, though not the Content Agent. The free Explore plan has 10 prompts and ChatGPT only.
What we tell clients
Three things. The tool shows what the engines said and where it came from, and a human decides what is true. Fixes take time to be picked up, so we re-test weekly for a month. And some answers will always vary, so we judge success by the trend across a fixed prompt set, not by one lucky run.
If you do one thing today, write the source of truth and run ten brand questions. You will find at least one error, and you will know where to start.