JavaScript rendering and AI crawlers

Googlebot

Google processes JavaScript pages in three phases: crawling, rendering and indexing. Pages are queued for rendering, and “once Google’s resources allow, a headless Chromium renders the page and executes the JavaScript”. A page may wait in this queue for a few seconds, but it can take longer.

Officially documented · Google Search Central

Google still recommends server-side rendering or pre-rendering. The reason: “it makes your website faster for users and crawlers, and not all bots can run JavaScript.”

Officially documented · Google Search Central

AI crawlers

Observed in 2024-12, more than 9 months ago. Answer engines change often; re-test before relying on this.

In tests published in December 2024, Vercel and MERJ found that none of the major AI crawlers they examined rendered JavaScript. They named OpenAI (OAI-SearchBot, ChatGPT-User, GPTBot), Anthropic (ClaudeBot), Meta (Meta-ExternalAgent), ByteDance (Bytespider) and Perplexity (PerplexityBot). Gemini, which uses Googlebot’s infrastructure, did render JavaScript. So did AppleBot.

Observed in practice · observed 2024-12 · Vercel & MERJ

Observed in 2025-10, more than 9 months ago. Answer engines change often; re-test before relying on this.

In searchVIU’s October 2025 test, five AI systems were asked for prices placed in different parts of one test page. Fetching the page live, ChatGPT and Gemini found prices in the visible HTML, but a price that JavaScript inserted after load was found live only by Gemini. Claude returned no prices at all, even for visible text. Google AI Mode and Perplexity answer from their indexes, which the authors say cannot be compared with live fetches; both reported the JavaScript-inserted price once the page was indexed.

Observed in practice · observed 2025-10 · searchVIU

In searchVIU’s test, ChatGPT and Gemini matched the December 2024 findings. Both are single studies: the first is almost two years old, and the second used one page and one question and gives two different run counts in its write-up. Crawler behaviour can change without announcement. None of the companies named document whether their crawlers render JavaScript. Treat the studies as the best available observations, not a specification.

What follows for a site

• Put the main content, title, description, headings and links in the HTML the server returns.
• Don’t rely on JavaScript to inject structured data, canonical tags or noindex for crawlers other than Googlebot. In searchVIU’s test, assistants fetching the page live did not use JSON-LD even when it was in the HTML (Structured data and AI search).
• Check what a crawler receives: fetch the page without JavaScript (curl -A "OAI-SearchBot" …) and compare it with what users see.

How this wiki handles it

Every page is pre-rendered to static HTML at build time. Search, filters and theme switching are enhancements on top. The Markdown twin of each entry (/path.md) gives agents the same content without any HTML or JavaScript.