How to tell if AI search can actually read your site

Everyone wants to be quoted in AI answers. Most sites have a more basic problem to fix first: the engine cannot read the page properly. Worth checking that before you spend a budget on anything fancier.

Here is the order I work through it.

Read the page the way a machine does#

Open the page source, not the rendered page. Better still, fetch it with curl and look at what comes back. If your main content only appears after JavaScript runs, a lot of AI crawlers will not see it.

Quick version: turn JavaScript off in your browser and reload. If the heading, the answer and the body text are still there, rendering is not your problem. If the page goes blank, that is your first job.

Why it matters: in a December 2024 study, Vercel and MERJ found that GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot and PerplexityBot fetched pages without running JavaScript. Googlebot did run it, and so did Applebot. The OpenAI, Anthropic and Perplexity crawler docs, re-checked at the source in October 2026, still make no claim to render JavaScript.

Google renders JavaScript in headless Chromium, but a page can sit in a queue before that happens. To see what Google actually got, open the page in Search Console's URL Inspection tool and look at the rendered HTML.

One more trap. A browser agent that clicks around your site in a real browser is not the same thing as the crawler that feeds AI search answers. If an agent can see your content, that does not prove the crawler can.

Check the bots are let in#

A page can pass the JavaScript test and still be invisible, because the bot never gets the page at all.

• robots.txt. The search bots to allow are OAI-SearchBot (ChatGPT search), Claude-SearchBot (Claude), PerplexityBot (Perplexity) and Googlebot (AI Overviews and AI Mode). OpenAI's bot docs say sites that block OAI-SearchBot will not be shown in ChatGPT search answers. GPTBot and ClaudeBot are training crawlers, and blocking those is a separate choice. Blocking Google-Extended does not take you out of Google Search.
• Your CDN or bot manager. Cloudflare changed its default to block AI crawlers in July 2025. Since 15 September 2026, its "Block" and "Block on pages with ads" settings also stop Googlebot, Bingbot and Applebot, so they can cost you normal search too. New domains on ad-funded sites are now offered a preset that blocks AI agents on pages with ads.
• Server logs. A curl request with a bot's user agent is a start, but Vercel's August 2026 guide warns it cannot confirm whether a real agent is blocked, because bot blockers care more about where a request comes from. Check your logs against the IP ranges OpenAI, Anthropic and Perplexity publish.
• Google's own switch. Since 31 August 2026 every site has a Search generative AI setting in Search Console (Settings, then Search generative AI). Include is the default. A site has to be included, and the page indexed and eligible to show with a snippet, to appear in AI Overviews or AI Mode.

Put the answer near the top#

If the answer to "what does this product do" sits in paragraph nine, it is easy to miss. Say it in paragraph one.

I put a short, direct answer right under the heading, then the detail underneath for the people who want it. That is my own habit, not a rule any engine publishes. Google's AI guidance says you do not need to chop content into small pieces or rewrite it just for AI. Write for a reader who skims, and a clean, quotable answer comes with it.

Make it clear who you are#

Make it easy for a person or a machine to work out who you are. That means a consistent business name, a proper About page, and sameAs links out to places that confirm you exist: LinkedIn, Companies House, your professional profiles. Organization schema ties it together.

No engine publishes this as a condition for being cited. It removes doubt about who wrote the page, and that is reason enough. This is the unglamorous part of GEO work, and it is easy to skip.

Check your headings describe the content#

Headings are the table of contents an engine reads first. "Our approach" tells it nothing. "How I price a technical SEO audit" tells it exactly what the section answers. Write headings as if someone will only read those.

Schema that matches the page#

Add Article or Organization where they genuinely fit. FAQPage is still valid markup, but Google stopped showing FAQ rich results on 7 May 2026, so do not add it expecting a search feature. Google's AI features guidance also says there is no special schema you need to add to appear in AI Overviews or AI Mode.

Schema is not decoration and it will not rescue thin content. It confirms what the visible page already says, in a format the machine cannot misread. In tests Vercel and ora.ai published in August 2026, adding JSON-LD or llms.txt did not change whether a client could find an answer already in the page's HTML. That was one small set of tests, not a rule, but it points the same way.

The five-minute test#

Ask ChatGPT, Claude and Perplexity a question your page should be the best answer to. See who gets named. One chat answer is a snapshot, so ask more than once.

For Google and Bing you can measure instead of guess. Search Console's Generative AI performance report, live for all sites since 31 August 2026, shows AI Overviews and AI Mode impressions by page. Bing Webmaster Tools' AI Performance report, launched as a public preview in February 2026, counts how often your pages are cited in Copilot and Bing's AI summaries.

If it is not you, and the page exists, work back through the list above. Is the page indexed? Is the bot allowed in? Does the content load without JavaScript? Is the answer easy to find?

Start there before you think about anything more advanced. A page the engine cannot read cannot be cited, no matter how good the writing is.