# Crawl budget

> Most sites can ignore crawl budget, which is set by crawl capacity limit and crawl demand. Google's guide targets very large or fast-changing sites: 1 million+ pages changing weekly, or 10,000+ changing daily. Other sites only need an up-to-date sitemap and a regular look at the Page Indexing report.

- URL: https://geo.casa/crawling-indexing/crawl-budget/
- Reference · Officially documented
- Reviewed: 2026-10-25

## Do you need to care?

> **Officially documented:**
> Google says sites without many rapidly changing pages, or whose pages are crawled the day they are published, don't need its guide. For Google Search, "keeping your sitemap up to date and checking the Page Indexing report regularly is adequate."
>
> Sources: [Google: Crawl budget management](https://developers.google.com/crawling/docs/crawl-budget)

> **Officially documented:**
> The guide is intended primarily for three kinds of site. The first is large sites (1 million+ unique pages) with content that changes moderately often (once a week). The second is medium or larger sites (10,000+ unique pages) with very rapidly changing content (daily). The third is sites with a large share of URLs reported as "Discovered – currently not indexed".
>
> Sources: [Google: Crawl budget management](https://developers.google.com/crawling/docs/crawl-budget)

## What determines it

> **Officially documented:**
> "A site's crawl budget is determined by two main elements: crawl capacity limit and crawl demand." The capacity limit, also called hostload, caps how much time the server spends on Google's connections so crawling doesn't overload it. Every site starts with the same conservative default. Separate hostnames such as `www.example.com` and `code.example.com` have separate crawl budgets.
>
> Sources: [Google: Crawl budget management](https://developers.google.com/crawling/docs/crawl-budget)

## Crawled is not indexed

> **Officially documented:**
> "Not every page that is crawled will necessarily be indexed." After crawling, each page is evaluated, consolidated and assessed for suitability for the index.
>
> Sources: [Google: Crawl budget management](https://developers.google.com/crawling/docs/crawl-budget)

## Related

- [HTTP status codes and crawlers](https://geo.casa/implementation/http-status-crawlers/) — 5xx and 429 responses slow crawling down.
- [XML sitemaps](https://geo.casa/crawling-indexing/xml-sitemaps/) — the minimum Google suggests for everyone.

## Sources

- [Google: Crawl budget management](https://developers.google.com/crawling/docs/crawl-budget)