Open Questions in Agent Selection

If early list positions win selection regardless of merit, placement can matter as much as the offer itself, and the effect size should guide where merchants spend effort.

Evidence we have

ACES reports that position bias varies by provider and persists even in text-only, 'headless' interfaces, undermining any single notion of a top rank (confirmed in the paper abstract). A separate 2026 hotel-listing simulation (Wadi and Ma, arXiv:2608.22697) reports that in a one-dimensional list, position predicts inspection weakly and non-monotonically, with the middle of a page least likely to be inspected, and that the attributes shown on a results page matter more than placement within it (abstract-confirmed).

Evidence we're missing

The exact per-provider regression coefficients from the study's models are not yet verified against the primary source, so we publish no numeric effect size.

Related signals: position bias audit

If a single model update can reorder which products get recommended, a one-time audit goes stale quickly, and merchants need to know how often to re-test.

Evidence we have

ACES states that model updates can drastically reshuffle market shares, and documents a model-finalization step that reallocated share and inverted the model's position bias (qualitatively confirmed).

Evidence we're missing

The exact numeric magnitude of that reshuffle sits in an appendix we have not yet captured from the primary source, so we publish no figure.

Related signals: retest on model updates

A large simulated effect is only actionable for merchants if it survives on real storefronts, against production models, on their own catalogs.

Evidence we have

ACES reports (v3) that seller-side description rewrites raised average market share substantially across several models in a simulated storefront.

Evidence we're missing

A replication on live merchant catalogs measuring the effect on real recommendation share. Our field test is published in methodology and collecting data.

Related signals: description rewrite experiment

If agents weigh how many reviews there are, how recent they are, and how they are distributed, the work is collecting and displaying real review data, not just lifting the average star number.

Evidence we have

ACES reports that agents are rating-sensitive and, in simulation, that review count carries its own weight per model (both reported from the paper, not from live agents).

Evidence we're missing

A test of whether recency and full star distribution, over and above count and average, change which product a live agent selects. Our review-depth signal is still a labeled inference.

Related signals: review depth, endorsement not sponsored

If a markdown that moves share on one engine barely moves another, price strategy has to be tested and budgeted per engine rather than applied as one blended discount.

Evidence we have

ACES reports per-model price elasticity in simulation, with Gemini the most price-sensitive and Claude and GPT-4.1 similar (reported, controlled simulation, not real sales). OpenAI now states first-party that merchants are ranked on factors including price, and that ChatGPT weights price by intent, focusing more on it when a shopper names a budget (spec-fact, help.openai.com, updated approx 2026-08-28).

Evidence we're missing

The same measurement against the production agents merchants actually face (ChatGPT, Gemini, Perplexity) on live catalogs, with a per-platform magnitude we can publish.

Related signals: model specific price sensitivity, price competitive awareness

Merchants are being told to add llms.txt; they need to know whether it changes discovery or selection or is currently unproven effort.

Evidence we have

The llms.txt spec is published and defines a priority order (spec-fact, llmstxt.org). Google states its Search ignores llms.txt entirely (spec-fact), and there is no evidence that shopping agents fetch or act on it.

Evidence we're missing

A measurement showing a named shopping agent reads llms.txt and that its presence, or its ordering, changes whether a store is discovered or selected.

Related signals: llms txt priority order, machine readable mirror

A .md mirror is real engineering; merchants should know whether it moves citation or selection, or is only a plausible best practice for now.

Evidence we have

Vendors including Vercel, Cloudflare, and Mintlify report large token savings from serving Markdown to documentation and coding agents (reported vendor examples), and content negotiation is a standard HTTP mechanism (spec-fact).

Evidence we're missing

Evidence that any shopping agent requests Markdown, and any measurement tying a .md mirror to higher citation or selection share for a store.

Related signals: machine readable mirror, facts per token

Merchants routinely pass validation and still lose the pick, so the practical question is whether added depth (shipping, returns, variants, identifiers) changes selection or only unlocks eligibility.

Evidence we have

Google documents the fields a listing needs to be eligible and shown (spec-fact), and Adobe reports most retail product pages score only 66 out of 100 for machine readability (reported). OpenAI now states first-party that product selection draws on structured metadata such as price and product description, and that merchants are ranked on availability, price, quality, and whether they are the maker or primary seller (spec-fact, help.openai.com, updated approx 2026-08-28). Google added six optional conversational feed attributes for AI Mode and reports that in testing, a brand's conversational attributes were incorporated 50% of the time in relevant AI Mode recommendations (spec-fact for the fields, reported for the figure, blog.google, 2026-09-16). Our facts-per-token signal, that fuller completeness then aids selection, remains a labeled inference rather than a measured result.

Evidence we're missing

A controlled test that isolates structured-data completeness and measures its effect on selection share, held apart from the eligibility it also affects.

Related signals: offer markup completeness, facts per token

Protocol readiness is significant engineering, so merchants need to know whether it wins the pick or is purely table stakes for completing a purchase.

Evidence we have

The protocols themselves state that implementing them does not guarantee listings and that each platform runs its own selection process (spec-fact, ACP and UCP docs). Readiness gates whether a purchase can complete, not the comparison that precedes it.

Evidence we're missing

A comparison showing whether protocol-ready stores are selected more often than equally-priced peers that are not ready, holding the offer constant.

Related signals: acp readiness, ucp readiness, payment rail readiness

The finding is counterintuitive and carries real dark-pattern temptation, so it matters whether it holds on production agents and which legitimate endorsement markers an agent actually recognizes.

Evidence we have

ACES reports that agents consistently penalize sponsored tags and reward platform endorsements, with the endorsement lift strongest for Gemini (reported, controlled simulation). A 2026 study (Wadi and Ma, arXiv:2609.17989) reports in its abstract that the sponsored-listing penalty is conditional on whom the agent serves: platform delegation significantly attenuates the penalty and weakens the skepticism disclosure triggers, and stricter terminology lowers but does not close the gap (reported; the paper's point estimates are unverified against its body).

Evidence we're missing

Confirmation that live shopping agents show the same penalty and lift, and a mapping of which real-world endorsement markers (editor's pick, verified seller) they read.

Related signals: endorsement not sponsored, review depth

If advertising spend gradually shapes which products an engine surfaces organically, the separation an engine states today is not something merchants can plan around, and paid and organic strategy stop being independent.

Evidence we have

OpenAI states first-party that ChatGPT product results are selected independently, are not ads, and are not influenced by OpenAI partnerships, and that Sponsored Agents (a limited alpha launched September 2026) are kept distinct from ChatGPT's independent answers (spec-fact, help.openai.com and openai.com, September 2026).

Evidence we're missing

Any independent measurement, over time, of whether a merchant's ad activity correlates with its organic selection share on the same surface. The stated separation is a policy statement, not an audited outcome.

If a strong effect in one layout nearly vanishes in another, merchants cannot treat placement as a fixed lever, and any position-bias finding has to state the layout and model cohort it was measured on.

Evidence we have

A 2026 hotel-listing simulation (Wadi and Ma, arXiv:2608.22697) reports a weak, non-monotonic position effect in a one-dimensional list, where earlier grid-based simulation work reported a strong effect, and finds that raising an agent's reasoning effort further collapses the effect (abstract-confirmed for direction).

Evidence we're missing

A controlled comparison that holds the offer constant across grid and list layouts on the same production agents, to separate the influence of layout from that of model generation.

Related signals: position bias audit

As storefronts expose tools an agent can call instead of parsing a page, the quality of those tool responses could become a selection input, but merchants have no confirmation it matters or how it is weighed.

Evidence we have

WebMCP is live by default on Shopify Liquid storefronts, letting an in-browser agent call tools rather than read the page (spec-fact, Shopify, 2026-08-05), so selection can move from parsing a page to answering a tool call, yet no engine documents tool-call quality as a selection factor.

Evidence we're missing

Any evidence, first-party or measured, that how completely or accurately a storefront answers tool calls changes whether an agent selects or cites it.