What It Does
Agentic Engine Optimization (AEO) is a practitioner discipline, coined by Addy Osmani (Director, Google Cloud AI) in April 2026, for structuring technical documentation and web content so AI coding agents can effectively use it. Where SEO optimizes for human-readable search engine crawlers, AEO optimizes for the distinctive HTTP access patterns of AI coding agents: short-duration sessions (1–2 GET requests), invisibility to client-side analytics, and hard constraints on content size imposed by LLM context windows.
The discipline is framed as a six-layer stack: access control (robots.txt auditing), discovery (llms.txt), capability signaling (skill.md), content formatting (Markdown, front-loaded content), token surfacing (exposing token counts as metadata), and UI affordances (“Copy for AI” buttons). Each layer addresses a specific failure mode where documentation sites built for humans inadvertently block or overwhelm agents.
Key Features
- Token budgeting: Treats token count as a first-class documentation metric. Proposed targets: quick starts <15K tokens, API reference pages <25K, conceptual guides <20K.
- robots.txt auditing: Checks that robots.txt does not unintentionally block AI User-Agent strings (e.g., Claude Code uses
axios/1.8.4, Cursor usesgotfrom sindresorhus). - llms.txt adoption: Placing a structured Markdown index at
/llms.txtso agents can discover documentation entry points without navigating full site trees. - AGENTS.md / skill.md files: Declarative files in repos and at API roots that tell agents what a service does and how to interact with it.
- AI traffic analytics: Tracking referrals from
labs.perplexity.ai,chatgpt.com,claude.ai,copilot.microsoft.com, andgemini.google.comto measure agent-driven traffic. - “Copy for AI” buttons: UI affordances that provide clean Markdown context for users pasting documentation into AI assistant conversations.
- agentic-seo audit tool: Osmani’s lightweight CLI that checks sites for AEO compliance (llms.txt presence, robots.txt agent access, token counts, Markdown availability).
Use Cases
- API documentation teams: Audit existing docs for token bloat and agent-blocking robots.txt rules; add llms.txt and AGENTS.md for discoverability.
- Developer tool vendors: Publish SKILL.md and skill packages so AI coding agents can accurately call your API without hallucinating endpoints.
- Documentation platform builders: Embed token count metadata and “Copy for AI” affordances into documentation toolchains (Mintlify, Docusaurus, etc.).
- Enterprise developer portals: Apply the six-layer stack to internal developer portals so internal AI agents can navigate documentation without human-in-the-loop lookup.
Adoption Level Analysis
Small teams (<20 engineers): Adding llms.txt and AGENTS.md to a repo is low-effort and provides marginal benefit today. “Copy for AI” buttons are worth adding to any documentation site. Token auditing is useful for teams with large documentation sets.
Medium orgs (20–200 engineers): Worth designating documentation standards around token budgets and AGENTS.md. The practices here are mostly lightweight and additive — no significant operational cost.
Enterprise (200+ engineers): Documentation portals serving thousands of developers should take agent access seriously. Token bloat in API reference is a real problem for agents today. MCP server endpoints for structured documentation queries may be more impactful than llms.txt alone.
Alternatives
| Alternative | Key Difference | Prefer when… |
|---|---|---|
| MCP server for docs | Structured, queryable API over HTTP with schema | You want guaranteed agent integration, not speculative llms.txt adoption |
| RAG pipeline over docs | Agent retrieves relevant chunks dynamically | Documentation is too large to fit in context even with chunking |
| Standard SEO + structured data | Overlapping with GEO (Generative Engine Optimization) | Primary goal is AI search citation rather than coding agent use |
Evidence & Sources
- Agentic Engine Optimization — Addy Osmani (original article)
- Webflow launches Webflow AEO product (April 2026) — Manila Times
- World Economic Forum on AEO as emerging marketing discipline (January 2026)
Notes & Caveats
- The term “AEO” is being used in two overlapping but distinct contexts: (1) Osmani’s technical documentation discipline for coding agents, and (2) a broader marketing/SEO concept about optimizing content for AI-powered answer engines. These are different practices sharing an acronym.
- The most controversial recommendation — llms.txt — lacks confirmed adoption by any major LLM inference provider as of April 2026. Google has explicitly stated it does not use it. The file has low cost to implement but its value remains speculative.
- Token budget targets (15K/25K/20K) are Osmani’s heuristics, not derived from empirical research. Context windows are growing (Gemini 3 offers 1M tokens); these limits may be significantly more relaxed within 12–18 months.
- The “1–2 HTTP requests” characterization of agent behavior is an oversimplification — browser-use agents and MCP-integrated agents behave very differently.
- MCP server endpoints for documentation are likely more impactful than static llms.txt files for any vendor with engineering resources to build them.