AIO
AI Crawlers Don't Render JavaScript: 5-Minute Check
ChatGPT, Claude and Perplexity crawlers mostly read raw HTML and skip JavaScript. How to check what they see in five minutes, and how to fix a client-rendered site.
A growing share of discovery now happens inside AI assistants. Someone asks ChatGPT for a supplier, a comparison or a how-to, and the answer cites a handful of pages. If your page is not one of the pages the assistant could read, it cannot be cited, however good it looks in a browser.
The most common reason a page cannot be read is simple: the text only exists after JavaScript runs, and most AI crawlers do not run JavaScript. This article explains what is known about how the crawlers behave, how to check your own pages in five minutes, and what to do if a page fails.
What the crawlers actually do
Two primary sources are worth knowing.
Vercel’s crawler analysis (December 2024). Vercel studied AI crawler traffic across its network and reported that none of the major AI crawlers it observed rendered JavaScript. OpenAI’s and Anthropic’s crawlers fetched JavaScript files but did not execute them, which means they could not read client-side rendered content. Google’s crawler, whose infrastructure Gemini uses, and Apple’s crawler did render JavaScript. Source: The rise of the AI crawler.
OpenAI’s crawler documentation. OpenAI separates its crawlers by purpose. OAI-SearchBot is the one tied to ChatGPT search: sites that opt out of it are not shown in ChatGPT search answers, although they can still appear as navigational links. GPTBot relates to training foundation models, and ChatGPT-User covers visits that a user triggers rather than automatic crawling. Source: Overview of OpenAI crawlers.
Put together: for ChatGPT search, the page you need to get right is the HTML that OAI-SearchBot receives, and you should assume that HTML is read as-is.
| Crawler | Used for | Renders JavaScript |
|---|---|---|
| Googlebot | Google Search, Gemini, AI Overviews | Yes |
| OAI-SearchBot | ChatGPT search answers | Assume no |
| GPTBot | OpenAI model training | No (per Vercel, 2024) |
| ClaudeBot | Anthropic | No (per Vercel, 2024) |
| PerplexityBot | Perplexity | No (per Vercel, 2024) |
Crawler behaviour can change without notice, which is exactly why the next section matters more than this table.
The five-minute check
You do not need special tools. Any one of these three methods answers the question for a given page.
Method 1: View Source
Open the page, then use View Page Source (not the Elements panel, which shows the page after scripts have run). Search for a sentence from your hero and a price from your pricing section. If they are not in the source, a crawler that skips JavaScript will not see them.
Method 2: Disable JavaScript and reload
In Chrome DevTools, open the command menu, choose “Disable JavaScript” and reload. What remains on screen is close to what a non-rendering crawler gets. A blank page, a spinner or a page with only the navigation is a fail.
Method 3: Fetch it the way a crawler does
Request the page with curl and count the words in the HTML, ignoring scripts and styles:
curl -sS -A "OAI-SearchBot/1.3" https://example.com/ -o page.html
python3 -c "import re,html;t=open('page.html',encoding='utf-8').read();t=re.sub(r'(?is)<(script|style|noscript)[^>]*>.*?</\1>',' ',t);print(len(html.unescape(re.sub(r'(?s)<[^>]+>',' ',t)).split()),'words')"
We ran this against our own partner page on 29 October 2026. The server returned 43,764 bytes of HTML containing 2,700 words of readable text, one H1 and nine H2 headings, before any script ran. That is a pass: everything a reader sees is already in the HTML.
Running the same check on a minimal client-rendered test page, an empty <div id="root"> that a script fills with a heading and 500 words, returns 1 word: the page title. None of the 500 words the visitor sees are in the HTML. That is the pattern to look for. Single-file exports from design tools and single-page apps built without server rendering often look like this, even when they look finished in a browser.
Launching a Japanese version?
Japanese localization, built as HTML every AI crawler can read.
FrontierLine writes the Japanese version as native speakers in Tokyo and delivers it as complete server-side HTML, with Japanese structured data, hreflang, llms.txt and a before / after answer check on ChatGPT, Claude and Gemini. Static HTML rebuilds of JavaScript-only sites from USD 900.
Why Google indexing is not proof
The trap is that a client-rendered site can do perfectly well in Google. Googlebot renders JavaScript before indexing, so the pages appear in results, Search Console looks healthy and the team concludes the site is crawlable.
It is crawlable by Google. For crawlers that read only the initial HTML, the same site can be close to empty. The result is a site that ranks but is never cited by ChatGPT, Claude or Perplexity, and nobody notices because nothing is visibly broken.
How to fix a page that fails
The goal is always the same: put the content in the initial HTML response. The right method depends on the stack.
| Situation | Practical fix |
|---|---|
| React, Vue or Svelte app with a framework available | Static generation or server-side rendering for marketing and content pages |
| Single-page app you cannot restructure | Prerender the public pages at build time and serve the HTML snapshots |
| Small site exported from a design tool as one file | Rebuild the pages as plain static HTML, keeping the design |
| WordPress or another server-rendered CMS | Usually already fine; check that page builders and sliders do not hide the main copy behind scripts |
Once the content is in the HTML, the usual readiness work starts to pay off: a clear title and description per page, one H1, headings that match the questions people ask, Organization and FAQPage structured data, and an llms.txt file that points AI systems to your key pages.
Also check robots.txt. If you want to appear in ChatGPT search answers, do not block OAI-SearchBot. Blocking GPTBot is a separate decision about training data and does not remove you from ChatGPT search.
A note for sites in more than one language
Language versions are where this problem hides most often. The English site may be server-rendered while the Japanese, German or Spanish version was added later as a client-side layer that swaps text after load. In that case every crawler that skips JavaScript sees the original language on every URL, hreflang points at pages with the wrong text, and the localized version effectively does not exist for AI search.
When we localize a site into Japanese, we check each Japanese URL this way before launch. Our Japanese localization checklist covers the rest of what a Japanese version needs.
For agencies outside Japan
Your Japan-side production partner, under your name.
Japanese localization, AI search readiness, Japan hosting and email deliverability, WordPress engineering and white-label maintenance. Written communication only, a reply within one Japan business day, and a fixed quote before any work starts.
Summary
Most AI crawlers read the HTML your server sends and do not execute JavaScript, while Google does. A client-rendered page can therefore rank in Google and still be invisible to ChatGPT, Claude and Perplexity. Check any page in five minutes with View Source, a JavaScript-disabled reload or a curl word count. If the content is missing, move it into the initial HTML with static generation, server rendering, prerendering or a static rebuild, then do the usual readiness work on top.
Frequently Asked Questions
The questions below summarise the points above for quick reference.