AI search terms, defined briefly.

Making your content more likely to be cited or mentioned inside answers from generative AI. The term comes from a 2023 research paper.

Source: GEO: Generative Engine Optimization (arXiv)

Writing pages so that systems that answer questions directly — AI search and voice assistants — can lift a clear answer from them.

The umbrella term for making a site findable, readable and usable by AI search tools. GEO and AEO are both used for it.

GEO, AEO and SEO

Google Search’s generative AI features. AI Overviews show a summary above the results; AI Mode answers in a conversation. Both run on Google Search, so a page has to be indexed to appear.

Source: Google Search Central: AI features and your website

A program an AI company uses to fetch web pages. There are three broad kinds: crawlers that collect training data, crawlers that build an AI search index, and fetchers that retrieve a page when a user asks. Decide whether to allow each kind separately.

Which AI crawlers to allow

A file at /robots.txt that tells crawlers which pages they may fetch. It became a standard as RFC 9309 in 2022. Crawlers choose whether to follow it; it does not technically block access.

Which AI crawlers to allowSource: RFC 9309

An XML list of a site’s pages that helps crawlers find them.

Source: sitemaps.org

A Markdown file at the site root that summarises the site and links to its key pages. Proposed in 2024; not a web standard. Whether an AI system reads it depends on the company.

Does llms.txt work?Source: llmstxt.org

Machine-readable facts about a page — the organization, a product, an article, FAQs — written with the schema.org vocabulary and embedded in the HTML. JSON-LD is one way to write it.

Source: schema.org

A browser running JavaScript to build a page’s content. Crawlers that don’t run JavaScript only see the text that is already in the HTML the server returns.

Can AI crawlers read JavaScript?

A web where AI agents research, compare, fill in forms and complete tasks on someone’s behalf. Sites need to let agents in, explain how they work, and expose the actions agents can take.

Commerce where an AI agent finds, compares and buys products for a person.

A specification for connecting AI applications to external tools and data. A site that publishes an MCP server lets agents use its functions as tools.

Source: modelcontextprotocol.io

A specification for AI agents to discover and talk to each other. An agent is described in a published Agent Card.

Source: A2A Protocol

A draft specification that lets AI agents discover how a store sells — how to find products and how to check out.

Source: ucp.dev

A draft specification for declaring actions on a web page, such as forms, as tools an AI agent can use. Being developed in a W3C community group.

Source: WebMCP (W3C Web Machine Learning CG)

Returning Markdown instead of HTML at the same URL when an agent asks with Accept: text/markdown, so it can read the content without the page chrome.

Source: RFC 7763 (text/markdown)