Crawl budget
Crawl budget is the volume of pages a search engine will explore on a given site within a set period. It depends on the server’s technical health and the engine’s assessed interest in the domain.
In practice
Below a few thousand pages the topic is minor. Beyond that — our engine manages 226,000 pages — it becomes central. Every unnecessary URL explored (filters, duplicates, parameters) delays indexation of pages that matter. Segmented sitemaps, clean pagination, and fast response times direct the crawler where it should go.