Internal linking audit for AI search: our process

Every site we audit has a few pages the owner assumes are doing their job. A guide that took two weeks to write. A comparison page the sales team uses. Then we look at the link graph and find that almost nothing on the site points to them. They sit in the sitemap, technically alive, reached by nobody who is not already holding the URL.

For classic search this was a known weakness. For AI search it raises a practical question: does a crawler ever get there? This is our process for answering it, in the order we run it.

Why internal links matter for crawlers

A crawler discovers pages by following links and by reading sitemaps. A page linked from several well-visited pages is easy to reach and gets revisited. A page with one inbound link, or none, depends entirely on the sitemap, and sitemap-only discovery tends to be slower and less certain.

We want to be careful about what we claim here. We do not say that orphan pages lower AI citations as a rule, because we have no evidence for a general effect and the engines do not publish how they weigh links. What we can say is narrower and testable: if no crawler visit ever reaches a page, that page cannot be fetched for an answer. Linking is one lever for changing that. The audit exists to find pages where it is worth pulling the lever.

Step one: build the list of pages that matter

Before looking at the graph, decide which pages deserve to be reachable. Not every URL does. Our list usually has four groups: pages that make or support a sale, pages that answer questions buyers ask AI engines, pages already cited somewhere, and the pages the client is currently trying to rank.

Everything else, such as tag archives, thin utility pages, and old campaign leftovers, goes on a separate list for pruning or leaving alone. This keeps the audit from turning into a hunt for every orphan on the site.

Step two: count inbound internal links

With any crawler you can export a list of URLs and their internal inlinks. We sort ascending and mark three bands: zero inlinks (orphans), one or two inlinks (weak), and everything else. We also flag URLs that redirect or error, because links pointing into a redirect waste crawl effort and often hide a broken target.

Check where each URL was discovered too. A page that appears only because it is in the sitemap is a different case from one found through a link. Pages that are only in the sitemap are the ones we look at first.

Step three: see whether crawlers actually arrive

The graph shows what could be reached. Crawler logs show what was. For each priority page we check whether AI crawlers such as GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, or PerplexityBot requested it in the last few weeks. A weak or orphaned page with regular crawler visits is not urgent. A weak page with none, which also happens to be a priority page, goes to the top of the list.

This second check stops us from fixing links that were never a problem. It also gives us a before and after comparison that does not depend on citation behaviour we cannot control.

Step four: fix links in the order of value

The fixes are simple and the discipline is in choosing which to do:

Add contextual links to priority orphans from pages that already get crawled and read. A relevant paragraph beats a footer link.
Link up from new pages at the moment of publication, so a page is never born orphaned.
Repair links that point into redirects and errors. Update the link, not just the redirect.
Review navigation and hub pages. A hub that links to everything dilutes its own signal, so choose.
Remove or consolidate pages that should not exist, rather than linking to them.

We resist the urge to add dozens of links per page. A handful of relevant ones, written with descriptive anchor text, does more than a block of forced ones.

Where Promptwatch fits

Promptwatch is the platform we run client programs on, and its October 28 changelog entry covers most of this audit. Site health gained a Linking view: a map of the site by URL path where orphan pages, pages with only one or two inlinks, and redirects or errors each get their own colour. You can click a section to zoom in or filter to the weak spots. Each page's detail view has a Linking tab showing the pages that link to it, the pages it links to, and broken links in red.

Two details save us time. Every page list shows where a page came from: sitemap, a link from another page, Search Console, or a site map crawl. And broken internal link issues list each broken target with its status code and anchor text. There is also a sitemap-only audit option that checks only the pages in your sitemaps, which new projects start with on.

The linking map does not tell you whether AI crawlers visit a page. That is what the crawler logs in Agent Analytics are for, and they start on the Professional plan. The Essential plan has no crawler logs, so on that tier you get the linking map but not the arrival evidence. We mention this in proposals because the two views together are what make the audit convincing.

The changelog also says the Content Agent adds internal links from the first draft, as of October 2. That helps with the "never publish an orphan" rule, but we still read each draft's links. An automated link is only as good as the page it points to. The Content Agent's scheduled mode sits on Business and Enterprise, so do not assume it on lower plans.

What we report back

The client report for this audit has a short structure. A count of priority pages by link band before and after the work. The list of fixes made, with URLs. Crawler visits to the priority pages across the period, where logs are available. And a note on what we did not change, such as pages we left orphaned on purpose.

We avoid headline claims such as "fixed 300 orphan pages." Most orphans do not matter. What matters is that the pages a buyer might be sent to are reachable, and that we can show a crawler reaching them.

If you do one thing this week, export your inlink counts and sort the priority pages ascending. The bottom of that list is usually a surprise.