Quick answer · Updated July 29, 2026
What Is Crawl Budget?
Direct Answer
Google defines a site's crawl budget as the set of URLs Googlebot can crawl without overloading the host and wants to crawl based on demand. The two components are the crawl capacity limit and crawl demand. Most small, normally updated sites do not need a dedicated crawl-budget project.
Should You Worry About It?
Google's current guide is intended mainly for sites with roughly one million or more pages that change moderately often, sites with roughly ten thousand or more pages that change daily, or sites with many URLs in the “Discovered – currently not indexed” category. Google calls these rough classification estimates, not exact thresholds.
If new pages are found promptly and Search Console does not show a large discovery backlog, keep your sitemap current and focus on useful, unique content rather than trying to force more crawling.
First Checks for a Large Site
- Remove or consolidate duplicate and unimportant URL variants.
- Return 404 or 410 for permanently removed content.
- Keep sitemaps current and avoid long redirect chains.
- Investigate server errors, rate limiting, and slow responses.
- Use Crawl Stats and server logs to confirm where requests go.
Need the implementation detail?
Continue with the large-site crawl budget guide for URL-inventory controls, logs, sitemaps, status-code decisions, and a measurement workflow. This answer intentionally stays short.