Crawl budgets and the visibility of small websites | Dead Internet Theory
Dead Internet Theory: Search crawlers have finite resources, but small-site visibility problems are usually better investigated through discoverability, linking, indexing, and server health than through dramatic assumptions about crawl budget.