How do I find out which pages on my site are cited by AI search engines?
How we find which pages on a client's site AI search engines cite: referral data, verified crawler logs and prompt-level citations, joined page by page.
When a client asks which of their pages AI search engines cite, we do not trust a single source. We build the answer from three kinds of evidence and look at where they disagree. Here is the process.
Source one: real visits
In the client's analytics, we look at sessions from AI assistants by landing page. ChatGPT, Perplexity, Gemini, Copilot and others appear under their own referrers, and ChatGPT links often carry a utm_source parameter that names it. This tells us which pages real people reached through AI answers.
It undercounts. People read citations without clicking, and some visits have no clean referrer. We treat it as the floor, not the total.
Source two: bots
Server and CDN logs show which AI crawlers requested which URLs: GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-User, Claude-SearchBot, PerplexityBot, Perplexity-User and Google-Agent. Two cautions from experience. A fetch is not a citation. And user agents can be faked, so we prefer logs that verify the source IP.
In Promptwatch, Agent Analytics provides real-time, IP-verified crawler logs, which filters spoofed user agents, with a crawl-to-citation path and a citation rate per page. It tracks errors with failure explanations, supports stackable path filters and exports to CSV. Integrations cover Cloudflare (Logpush or an auto-deployed Worker), AWS CloudFront, Fastly, Vercel, Netlify, Akamai, Google Cloud CDN, custom HTTP and a WordPress plugin, so most client stacks are covered.
Source three: the answers themselves
We build a prompt panel and run it daily per model, country and language. For each prompt, citation analytics shows the cited pages and domains with URL positions. The Page Tracker follows specific URLs, ours or competitors', and links them to the prompts and responses where they appear.
This is the only source that shows what an engine does with a page today. It is also a sample, which is why we run it daily rather than checking once.
Joining them
The useful view is per page. 360 Insights joins crawler logs, citation clicks and AI-referred visits for each page. Topic Insights shows where a page underdelivers for its topic. Search Insights overlays Google Search Console and Bing Webmaster data, including Microsoft Copilot traffic, on AI crawls and citations.
From that, we sort pages into four buckets:
| Bucket | What it means | What we do |
|---|---|---|
| Not fetched | Bots do not reach it | Fix robots.txt, firewall, rendering or sitemap |
| Fetched, not cited | Reachable but not chosen | Improve the answer, check competitors' pages |
| Cited, no visits | Seen but not clicked | Rework the title, the opening and the call to action |
| Cited, with visits | Working | Refresh before it decays |
The signals we watch over time
Citation Trends classifies every citation by content type and source type and shows citation rate over time, with first cited, ramp-up days, peak rate and decay half-life per page. We use ramp-up days to avoid judging a new page too early, and decay half-life to schedule refreshes. Content Refresh gives diff-style update suggestions.
Setup for a new client
- Connect the crawler log source and the visitor script, GTM template or WordPress plugin.
- Build 30 to 50 prompts, tagged by topic and intent.
- Pull the 50 most important pages and the question each should answer.
- Run two weeks of baseline.
- Build the bucket table and fix the not-fetched group first.
Plans
Crawler logs are not in Essential ($95 a month, 50 prompts). They start on Professional at $245 a month, which includes 25M crawler logs, and every agency plan includes them (Kick-off $199, Growth $399, Scale $799). The free Explore plan covers 10 prompts on ChatGPT only and is only for learning the interface.
What we say to clients
Nobody can guarantee a page will be cited, and no data source sees everything. We build an evidence trail from visits, bots and answers, and we act on the gaps. The rest is iteration.