Cloudflare has launched Markdown for Agents, a feature that converts HTML pages to markdown when AI crawlers request it. Running on Cloudflare's edge network and powered by HTTP content negotiation, the beta is available at no extra cost on eligible paid plans.
Key details: Cloudflare Markdown for Agents
Markdown for Agents uses the HTTP Accept header to let clients request markdown instead of HTML. When an AI agent sends a request with Accept: text/markdown, Cloudflare intercepts it on the network edge, fetches the original HTML from the origin server, converts it to markdown, and returns the markdown response.
Cloudflare says the conversion is transparent for origins, which continue to serve standard HTML pages. Site owners enable Markdown for Agents at the zone level in the Cloudflare dashboard. The beta is available at no additional charge on Pro, Business, Enterprise, and SSL for SaaS plans.
To illustrate the impact on token usage, Cloudflare compared HTML and markdown versions of one of its own blog posts. The HTML version used 16,180 tokens, while the markdown version used 3,150 tokens. In its announcement, Cloudflare likened sending raw HTML to AI agents to paying by the word for packaging instead of "the letter inside" containing the actual content.
Markdown responses include an x-markdown-tokens HTTP header that estimates the markdown token count. Cloudflare says developers can use this estimate to manage context windows or design chunking strategies. Each markdown response also carries a Content-Signal header set by default to ai-train=yes, search=yes, ai-input=yes, as defined in Cloudflare's Content-Signal policy, announced during Birthday Week 2025.
Cloudflare also notes that some AI coding tools already send Accept: text/markdown in their requests. The company names Claude Code and OpenCode as examples of clients using this header.
Background: content negotiation and search guidance
Cloudflare describes Markdown for Agents as treating AI agents as "first-class citizens" alongside human visitors. The system relies on HTTP content negotiation rather than user-agent detection to decide which format to return, so the same URL can serve either HTML or markdown depending on the client's Accept header.
In a Reddit thread, Google Search representative John Mueller criticized the idea of serving separate markdown pages only to bots. He called that approach "a stupid idea" and questioned whether bots parse markdown links correctly. His comments focused on user-agent based serving and custom markdown versions maintained alongside standard pages.
Google defines cloaking as showing different content to users and search engines with the intent to manipulate rankings and mislead users. Cloudflare's approach is positioned as content negotiation based on format preferences, not as a separate set of pages built specifically for search crawlers.
Cloudflare Radar tracking of AI content types
Cloudflare has added content-type tracking for AI bot traffic to Cloudflare Radar. The AI Insights section now shows the distribution of MIME types received by AI agents, including markdown, HTML, and other formats, for traffic that passes through Cloudflare's network.
Users can filter the Radar view by individual bot to see which content types each crawler receives. Cloudflare highlighted OAI-SearchBot as an example, including the volume of markdown responses served to that crawler. The company says this dataset is also available through its public APIs and Data Explorer.
Cloudflare's announcement also notes upcoming support for custom Content-Signal policies for Markdown for Agents, which will let site owners specify how different clients may use their content.
.svg)
.svg)
.svg)