How do AI search engines decide which websites to cite in their answers? What we see in practice

Clients ask this one in different words: why did ChatGPT cite that site and not ours? The honest answer starts with what nobody outside the vendors knows. No AI search engine publishes its full source-selection rules. What we can say comes from official documentation, from repeated observation and from client data. We try to keep those three apart.

The pipeline, as far as it is known

Most engines that cite sources follow a recognizable shape, though details vary and change.

1. The model interprets the question, often rewriting it.
2. It issues several sub-queries, called query fan-outs. A question about the best accounting software for a small agency can turn into searches about pricing, integrations, reviews and comparisons.
3. Those queries retrieve pages from an index or by live fetch.
4. Candidates are filtered and ranked by criteria the vendors do not disclose.
5. The model writes an answer grounded in selected passages and shows citations for some of them.

A page can fall out at any step. It might be blocked from the crawler, not indexed, never retrieved for any sub-query, retrieved but passed over, or used but not shown as a citation. When a client's page is missing, the work is to find which step failed.

What is documented

Google's guidance on AI features in Search says the same foundations as classic search apply: indexable pages, eligibility for snippets, helpful content. OpenAI documents separate crawlers by purpose, with OAI-SearchBot for search results and GPTBot for training. Anthropic and Perplexity document their own bots too. That is a solid base: if the right bot cannot reach the page, nothing else matters for live-fetch engines.

What is not documented is any precise weighting. Any claim of the form "this schema type makes AI cite you" or "pages with 1,500 words get cited more" deserves suspicion unless it comes with a method and sample size.

What we have seen

These are observations from client work, not laws.

• Access problems are common and boring. A CDN rule or bot challenge blocks a crawler, and the client spends weeks on content instead.
• Direct answers get used. Pages that state the answer in the opening lines tend to be easier for engines to quote.
• Coverage across sub-queries matters. A page that covers one angle of a fan-out has fewer chances than one that covers several, or a cluster of pages that cover them together.
• Off-site sources often win. Answers frequently cite forums, video and review or comparison sites. A client with a strong site can still lose because the engines lean on a Reddit thread or a trade article.
• Citation behavior has a life cycle. Some pages are cited quickly then fade. Others climb slowly and hold.

How we check it on a client site

We use Promptwatch to look at the real data rather than guess.

Fan-outs. The platform shows query fan-outs per model. For the client's top prompts we list the sub-queries and ask whether we have a page for each. The free GEO Inspector Chrome extension shows ChatGPT fan-outs and fetched versus cited sources, and runs locally.

Citations. Citation analytics gives page and domain level data, URL positions, Page Tracker for specific URLs (ours or a competitor's), offsite mentions, and Reddit, YouTube and LinkedIn reports. Citation Trends classifies citations by content type and source type, and gives first cited, ramp-up days, peak rate and decay half-life per page. That life-cycle view is where the pattern above came from.

Crawling. Agent Analytics gives IP-verified logs and a crawl-to-citation path with a citation rate per page, so we can tell a page that is fetched and never cited from one that is never fetched. Crawler logs begin on the Professional plan, not Essential.

Traffic. Visitor analytics shows AI-referred sessions and conversions, so we see whether citations send people.

One public example from Promptwatch's own case studies: TreeHouse, behind Pararius and Huurwoningen.nl, reported AI search growing to about 25% of organic conversions, 99% of it from ChatGPT, after fixing crawler access and automating weekly reporting with the Promptwatch MCP. That is their result. We do not promise it for anyone else.

A diagnostic routine

1. Pick one prompt you should win and lose.
2. List the fan-outs. Do you have a page for each?
3. Check the logs. Was the page fetched? By which bot? Any errors?
4. Check who was cited instead. What kind of page is it? Which kind of source?
5. Decide the fix: access, content, or off-site presence.
6. Make one change, log the date and watch the citation rate.

Limits

Answers vary between runs, engines update their behavior, and personalization adds noise. We do not claim to know the formula, and no tool can promise a citation. What we can do is make the unknowns smaller for each client.

Promptwatch has a free Explore tier (10 prompts, ChatGPT only) if you want to start looking at your own prompts today.