1001 SEO Media
All posts
By 1001 SEO MediaChatGPT citationsOAI-SearchBotcrawler logsagency workflow

How do I know which pages on my site ChatGPT is actually reading and linking to?

How we find which pages on a client's site ChatGPT reads and links to, from crawler logs and citation data to referral traffic, and the Promptwatch features we rely on.

Clients ask this in different words. "Is ChatGPT even seeing our site?" "Which of our pages does it use?" Underneath, two questions: what does ChatGPT read, and what does it link to? We answer them separately, then put them together.

Reading is a log question

When ChatGPT's systems fetch a page, the request lands in your server or CDN logs. OpenAI uses several crawlers for different purposes. OAI-SearchBot is tied to search results, GPTBot to training, and ChatGPT-User appears when a user request triggers an on-demand fetch. We read OpenAI's current documentation for each before we draw conclusions, because roles can change.

What we look for in the first pass:

  • Which of those bots appear at all
  • Which URLs they fetch, and how often
  • Which response codes they get
  • Which important pages never get a visit

Two traps. First, user agents can be faked, so a count based on the string alone includes impostors. Second, the log shows a fetch, not a use. A page can be read ten times and never appear in an answer.

Linking is a citation question

For this we need to know which pages ChatGPT shows as sources for the prompts that matter. There are three ways in.

Manual prompts. We ask the questions the page should answer and note the sources. Cheap, but a single run proves little because answers shift with wording, place and day.

Referral traffic. Visits from chatgpt.com in analytics show pages that were linked and clicked. It undercounts, since an uncliked link leaves no trace.

Citation tracking. A platform that runs a fixed prompt set daily and records which URLs get cited. This is the one that scales.

What we run it on

We use Promptwatch, because it handles both sides and joins them.

Agent Analytics is the reading side: real-time, IP-verified crawler logs, with spoofed user agents filtered out. It tracks OAI-SearchBot, GPTBot and ChatGPT-User, and the Claude, Perplexity, Google and Meta agents. Error tracking explains failures, stackable path filters isolate sections of a site, and CSV export lets us share data with a client's developers.

Citation analytics is the linking side. It shows citations at page and domain level, with URL positions. Page Tracker follows specific URLs, ours or a competitor's. Citation Trends classifies citations by content type and source type, and gives first cited date, ramp-up days, peak rate and decay half-life per page. That last set of numbers is useful for conversations about how long content takes to start getting used.

The join. A crawl-to-citation path gives each page a citation rate, and the 360 Insights view combines crawler logs, citation clicks and AI-referred visits per page. This is the view we open when a client asks about a specific URL.

We also point clients to a free tool from Promptwatch, the GEO Inspector Chrome extension. It shows ChatGPT's query fan-outs and which sources were fetched versus cited, and it runs locally. It is handy for spot checks during a meeting.

Sorting pages into four groups

Once the data is joined, every important page lands in one of four boxes:

  1. Read and cited. Working. Protect it, keep it accurate and link to it internally.
  2. Read, not cited. The content may be thin, vague or outranked. We rewrite with a direct answer and specifics, or look at what is cited instead.
  3. Cited, rarely read. Probably a cached version in use, or fetched through paths we did not expect. We check freshness and canonical setup.
  4. Neither. If the page matters, fix access and discoverability. If it does not, leave it.

The setup

Logs reach Promptwatch from Cloudflare (Logpush or an auto-deployed Worker), CloudFront, Fastly, Vercel, Netlify, Akamai, Google Cloud CDN, a custom HTTP endpoint or the WordPress plugin. Crawler logs are not on Essential. They start on Professional ($245/mo) and are included on every agency plan from Kick-off ($199/mo), which is how our client work is set up.

For a first look without any tool, Cloudflare's own AI Crawl Control dashboard is free for sites behind Cloudflare. It shows crawler requests by path but has no citation data.

What we tell clients

  • Reading is a prerequisite for linking, not a promise of it.
  • Citations vary by prompt wording, country and day. We report trends.
  • We cannot control what ChatGPT chooses to link. We can remove obstacles and improve what is on the page.

Common causes of "read but not linked"

We see the same few patterns. The page buries the answer under introduction. It makes claims with no specifics. It duplicates a better source. Or it answers a question nobody asks in that form. None needs a new tool, only an editor who has the citation data open.

If you want us to look at a specific page, send the URL and the question it should answer.