Key takeaways
- On a machine surface the scarce resource is the agent's token budget, so the page that packs the most verifiable facts per token, the specs, price, GTIN, availability, and policies, gives the agent the most to decide on.
- Serving the same content in a different format (Markdown instead of HTML) at the same URL is standard HTTP content negotiation. Serving different content depending on the user-agent at one URL is cloaking, a Google Search spam violation.
- Same-URL Accept-header negotiation and distinct .md mirror URLs are complementary implementations, not rivals. AgentMint ships distinct .md mirrors as a valid, conservative choice, not because they are safer.
- Markdown content negotiation is documented and working for coding and documentation agents. We have no verified evidence that shopping agents send Accept: text/markdown, so do not build catalog strategy on it yet.
What information density means for an agent
An agent does not read your page the way a shopper does. It pulls text into a context window, and that window is a fixed budget it spends across your page, competing pages, the user's request, and its own reasoning. Every token your page uses on filler is a token the agent cannot spend on a fact. So the useful unit here is not word count or reading time. It is verifiable facts per token: how much an agent actually learns for each token it pays to read you.
Vendors publishing guidance on serving Markdown to agents through 2026, including Vercel, Cloudflare, and Mintlify, each state that agents operate under context-window and token budgets, and that converting HTML to Markdown cuts token consumption and cost while preserving the decision-relevant content.ReportedVercel, Cloudflare, and Mintlify agent-Markdown guides (2026) That is the mechanical case for stripping page weight: a lighter representation of the same facts costs the agent less to read.
The commerce case goes one step further, and it is our inference, not theirs. Structured, verifiable facts (specs, price, GTIN, availability, return and shipping policy) plausibly carry more decision weight per token than qualitative marketing prose, because they are the fields an agent can match against a shopper's query and compare across stores. A sentence of adjectives is expensive to read and hard to act on; a spec line is cheap and directly comparable.Hypothesis (our analysis) This is the practical core of a page's machine-readability score: not how much you say, but how much of what you say an agent can use.
Lead with verifiable facts; cut the marketing prose
The move that follows from this is blunt: put the facts first and the prose last, or nowhere. On a surface built to be read by machines, lead with the fields that decide selection, the exact specifications, the price and currency, the GTIN, the availability, and the shipping and return policy, and treat brand-voice copy as optional garnish rather than the opening.Hypothesis (our analysis) The facts an agent needs are the same facts you already publish through your other channels, so this is not new content to write. It is reordering and trimming what exists.
Those verifiable facts do not live only on the page. The same fields belong in the two structured channels that agents read first: your product feed, which is the merchant-controlled data an engine ingests directly, and your Product schema, which lets a crawler read the same facts off the live page. Information density is not a fourth place to keep your data. It is the discipline of making sure every surface that faces an agent leads with the facts and wastes as few tokens as possible getting there.
Content negotiation is legitimate; cloaking is not
The moment you serve a machine-friendly version of a page, someone will ask whether that counts as cloaking. The answer depends entirely on what varies, and the primary definitions draw a clean line.
Google defines cloaking as "the practice of presenting different content to users and search engines with the intent to manipulate search rankings and mislead users."Spec-factGoogle, Spam policies for Google web search The violation is in the word different: two audiences, two contents, one deceptive intent. Google's own example of cloaking is "Inserting text or keywords into a page only when the user agent that is requesting the page is a search engine, not a human visitor," which names user-agent-conditional content as the violation directly.Spec-factGoogle, Spam policies for Google web search
Now hold that against what content negotiation actually is. MDN defines content negotiation as "the mechanism that is used for serving different representations of a resource to the same URI," selected by the client's Accept header, which lists the media types it is willing to receive.Spec-factMDN, HTTP content negotiation Different representation of the same resource is precisely not different content. It is the same facts, encoded once as HTML and once as Markdown, chosen by an honest signal the client sends about what it can parse.
Bing's Webmaster team made a compatible point in 2018: returning server-rendered content to bots and client-rendered content to people is acceptable and not cloaking, as long as you make a good-faith effort to return the same content to every visitor.ReportedBing Webmaster Blog (2018-10-31) Treat that as dated corroboration of the principle, not current policy, since it is a 2018 blog post. Bing's current Webmaster Guidelines state: "Showing different content to search crawlers than to users is a manipulative practice that undermines trust and visibility. Sites should not rely on cloaking or serving materially different experiences to crawlers and visitors."Spec-factBing, Webmaster Guidelines As of 2026-07-08 the live guidelines no longer carry the 2018 good-faith wording, and they list cloaking among the practices that reduce ranking eligibility and grounding visibility across Bing and Copilot experiences. The boundary is still content equivalence, now extended explicitly to Copilot and grounding; the current guidelines say nothing about dynamic rendering, so do not read Bing as endorsing that technique today.
Put the three together and you get the doctrine at the top of this page: format negotiation keeps the content equal and only changes the encoding, so it does not meet Google's different-content-plus-intent test for cloaking. What would cross the line is trivial to describe. If you served an agent a Markdown page stuffed with keywords or claims that never appear in the HTML a shopper sees, that is different content by user-agent, and it is cloaking. The rule is not the format. The rule is equivalence.
Two honest ways to serve a machine format
There are two well-documented ways to hand an agent a Markdown copy of a page, and they are not rivals. Pronovix, writing in March 2026, frames same-URL Accept negotiation and distinct .md URLs as complementary rather than competing, recommends pairing a Markdown alternate with a canonical link so search engines still consolidate on the HTML page, and notes that its own module does not push Markdown to bots that never requested it.ReportedPronovix, How to serve Markdown to AI agents (2026-03-24)
The first way is same-URL negotiation: the page at /product/weighted-sleep-mask returns HTML by default and Markdown when the request sends Accept: text/markdown. One URL, two representations, chosen by the header. The second way is a distinct mirror URL, for example /product/weighted-sleep-mask.md, that always returns Markdown regardless of the header. For agents that cannot send an Accept header, both approaches lean on the same discovery hint. Vercel recommends advertising the Markdown alternate with a <link rel="alternate"> element so an agent that cannot negotiate can still find it.ReportedVercel, Making agent-friendly pages with content negotiation (2026-02-03) Developer handoff: point the alternate at the Markdown resource and keep a <link rel="canonical"> on the HTML page so ranking signals stay consolidated there.
AgentMint itself ships distinct .md mirror URLs rather than same-URL negotiation. We treat that as a valid, conservative choice, not a safer one: it needs no Accept-header branching at the edge, it caches and links like any other URL, and it stays discoverable through a rel=alternate reference. Same-URL negotiation is equally legitimate; the two are implementation options, not a safety ranking, and no source we verified ranks one above the other.Hypothesis (our analysis) Read our own worked examples at /llms.txt and by appending .md to any page URL.
The honest boundary
This is where the chapter has to be careful, because the evidence stops short of where the hype wants to go. The vendors documenting working Accept: text/markdown negotiation through 2026 are documentation and developer-tooling platforms, and the consumers they describe are coding and documentation agents.ReportedMintlify, Cloudflare, and Vercel agent-Markdown guides (2026-01-29) That is a real, proven pattern in that context.
It is not proven for shopping. We have found no verified public source showing that a shopping agent (ChatGPT Shopping, Perplexity, or Gemini) sends Accept: text/markdown, or otherwise negotiates a Markdown representation of a product page. Treat content negotiation as proven for doc and coding agents and unverified for shopping agents, and do not assume a shopping agent will fetch your Markdown just because you serve it.Hypothesis (our analysis) Re-checked 2026-07-08: the primary crawler documentation from OpenAI, Perplexity, and Anthropic documents robots.txt as the control mechanism and never mentions Accept-header Markdown negotiation, and OpenAI's Agentic Commerce product feed spec describes a structured feed file, not page-format negotiation. Google's AI-features optimization guide, updated in June 2026, states: "You don't need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search."Spec-factGoogle, Optimizing your website for generative AI features on Google Search All of that is absence of documentation, not proof that no agent ever sends the header.
The practical consequence for a merchant: ship the machine format because it is cheap and standards-compliant, but do not attribute a selection outcome to it. If your products are missing from an engine's results, the format of an alternate representation is far down the list of likely causes; walk the real diagnosis in why your store is not in ChatGPT shopping first.
Apply it: llms.txt, schema, and feeds
The same facts-per-token discipline shapes the one machine-index format with a public spec. The llms.txt proposal fixes the file's order: an H1 with the site or project name (the only required element), then an optional blockquote one-line summary, then zero or more non-heading Markdown blocks, then zero or more H2 sections, each an unordered list of links written as [name](url): optional notes.Spec-factllms.txt proposal, llmstxt.org The format itself is owned by llmstxt.org and we link out to it rather than restate it; what belongs here is the judgment about what goes in which slot.
Order is your only lever. The proposal gives exactly one prioritization signal: an H2 section titled "Optional" marks links whose URLs can be skipped when a shorter context is needed.Spec-factllms.txt proposal, llmstxt.org Everything above that section is what an agent should read first, so order your link lists by decision weight, not by site-map convenience: the pages that carry buying-relevant facts near the top, the secondary material under "Optional." The proposal sets no explicit maximum file size, token, or byte cap, so density is a discipline you impose, not a limit the spec enforces.Spec-factllms.txt proposal, llmstxt.org For how llms.txt fits alongside your crawler policy, see AI crawlers, robots.txt and llms.txt for stores.
Two vendor figures get quoted a lot in this space, and both are illustrations rather than benchmarks. In one Vercel example, negotiating Markdown cut a page from roughly 500KB of HTML to about 3KB, which Vercel put at about a 99.37% payload reduction. That is a single illustrative figure from Vercel, not a benchmark you should expect to reproduce.ReportedVercel, Making agent-friendly pages with content negotiation (2026-02-03) Cloudflare's edge feature converts HTML to Markdown on the fly when a request sends Accept: text/markdown, and returns x-markdown-tokens and x-original-tokens response headers so the token difference is measurable per response. Cloudflare did not date this reference page precisely, so we record its date as approximate.ReportedCloudflare, Markdown for Agents (approx. 2026-06-05) If you want a number for your own catalog, that header pair is where to read it, on your own pages, rather than borrowing a vendor's example figure.
The step-by-step machine-surface runbooks now live in the blueprints hub: llms.txt for e-commerce catalogs, the product markdown-mirror template, and token-efficient product JSON-LD patterns carry the build steps, and this chapter carries the doctrine they implement.
What to measure
- Facts in the first screen of tokens. Take a product page's HTML or Markdown, keep the first 200 words, and count how many verifiable facts appear (spec, price, GTIN, availability, policy). If the answer is near zero because the opening is brand copy, reorder.
- Payload delta, read on your own pages. If you serve a Markdown representation through Cloudflare, read the
x-markdown-tokensandx-original-tokensheaders on a real product URL and record the ratio. Do not quote a vendor's example figure as if it were yours. - Equivalence check, the cloaking guard. Diff the facts in your Markdown or .md mirror against the HTML a shopper sees. Any fact, price, or claim present in one and absent from the other is a cloaking risk; fix it so the two are equivalent.
- llms.txt order audit. Confirm the links above your "Optional" section are the buying-relevant pages, and that low-value URLs sit under "Optional." Read your live /llms.txt and rank it by decision weight, not by navigation.
- Alternate discoverability. Verify each HTML page exposes a
<link rel="alternate">to its Markdown resource and a<link rel="canonical">to itself, so agents can find the format and search engines still consolidate on the page.
Once these surfaces exist, the next question is keeping them fast and fresh for agents that refetch often. Continue with serving agent traffic, and return to how AI shopping agents choose products for where density sits among the other selection signals.