Crawl Budget

The number of pages a search engine will crawl on your site within a given timeframe. Crawl budget is finite and can be wasted on low-value pages, pagination, and duplicate URLs. Large sites actively manage crawl budget by using robots.txt, canonical tags, and noindex directives.