Google has updated its Googlebot documentation to clarify how file size limits are applied when content is crawled for Google Search, separating Googlebot-specific limits from infrastructure-wide defaults used across its crawlers. The company recorded the update in its official documentation changelog for Search and in its crawling resources.
Key details: Googlebot file size limit documentation
According to the updated Googlebot documentation, Google now lists explicit file size limits for content fetched for Google Search. The page documents a 2 MB limit for HTML and other supported text-based file types, and a 64 MB limit for PDFs. It also notes that resources referenced in HTML, such as CSS and JavaScript, are fetched separately.
Google's broader crawler documentation states that the default file size limit across its crawlers and fetchers is 15 MB. This default is now documented on the general crawler overview instead of the Googlebot page. In its changelog, Google characterizes the update as a clarification of documentation rather than a change in crawler behavior.
Key factual points include:
- Google moved default file size limit information from the Googlebot page to the crawler overview documentation.
- The crawler overview lists a 15 MB default file size limit for Google's crawlers and fetchers.
- The Googlebot page now documents a 2 MB limit for HTML and supported text-based files when crawled for Google Search.
- The same page lists a 64 MB limit for PDFs when crawled for Google Search.
- Google describes the update as a documentation clarification in its official documentation changelog, not a change in how crawlers operate.
- The crawler overview covers infrastructure-wide defaults, while the Googlebot documentation focuses on Google Search behavior for Googlebot.
Background context
Google has been restructuring its technical crawling documentation since late 2025. In November 2025, Google migrated its core crawling documentation to a standalone Crawling section on developers.google.com, separating it from Search Central resources.
In December 2025, more documentation followed, with additional guidance such as faceted navigation and crawl budget information added to the same crawling documentation set.
The 15 MB file size limit for content processing by Googlebot first appeared in Google's public documentation in 2022. At that time, Google stated that only the first 15 MB of HTML or text-based content would be considered for indexing, a limit related to how much of a file Google Search would use for indexing rather than whether the file could be fetched.
The latest update clarifies the distinction between infrastructure-wide defaults and product-specific limits in one place. The crawler overview now documents the 15 MB default that applies across all Google crawlers and fetchers, while the Googlebot page documents the specific file size limits that apply when Googlebot crawls content for Google Search.
Source citations
The following official Google resources describe the current crawler and Googlebot file size limit documentation:
.svg)
.svg)
.svg)