What is AEO? Why your best pages get skipped by AI answers
Answer Engine Optimization (AEO) is the practice of structuring a website so AI answer engines — ChatGPT Search, Perplexity, Google AI Overviews, Gemini, Copilot — can crawl its pages, lift a clean answer out of them, and cite it as the source. SEO optimizes for a ranked list of blue links. AEO optimizes for being the extracted, quoted answer above them.
That distinction sounds academic until you watch a buyer work. They do not scan ten results anymore; they ask "best AI chat assistant for Shopify" and read one synthesized paragraph with two or three citations. Either your page is one of those citations, or the conversation about your category happens without you.
This guide covers what answer engines actually do with your pages, the six properties that make a page citable, and how to check any URL in about fifteen seconds with our free AEO checker — it scores a page 0–100 on the exact criteria below.
How is AEO different from SEO?
AEO is not a replacement for SEO — answer engines lean on search indexes to find candidate pages, so the SEO foundation still matters. The difference is what happens after the fetch. A search engine ranks your whole page. An answer engine reads it, pulls out one passage, and decides whether to attribute a claim to you.
| SEO | AEO | |
|---|---|---|
| Unit of competition | The page — ranked against other pages | The passage — extracted and quoted, or skipped |
| Reader | A crawler building an index for humans to browse | A model composing one answer for one question |
| Winning looks like | Position 1–3 in a list of links | Being the cited source inside the answer |
| Failure mode | Ranking low | Ranking fine — and being unquotable, or unreachable |
| Key surfaces | Keywords, links, Core Web Vitals | Crawler access, answer-first copy, JSON-LD, entity trust |
The failure mode row is the one that stings. A page can rank #1 and still lose in an AI answer: the answer is buried in paragraph nine, the first sentence starts with "We" so the quote has no subject, there is no FAQ schema, or — the most common and least visible failure — a CDN setting is returning 403 to the very crawlers that produce citations.
What does an answer engine actually do with your page?
Every answer engine, whatever its brand, runs the same three-step pipeline. Each step can silently eliminate you.
- Fetch. A crawler or live fetcher requests your page and reads the raw server HTML. Most render little or no JavaScript. If the fetch gets a 403, a bot challenge, or an empty client-rendered shell, you are out at step one.
- Extract. The model looks for a self-contained passage that answers the user's question — a tight paragraph, a table row, an FAQ answer. Pages structured as question-shaped headings with the answer directly beneath give it clean cut points. Walls of prose do not.
- Trust. Before attributing a claim, the model weighs who is making it: a resolvable entity with consistent structured data, profiles, an author, a fresh last-updated date — or an anonymous, undated page. Verifiable beats superlative; models increasingly fact-check "the best" and quote "supports 40 languages".
AEO is simply engineering your pages to survive all three steps. Everything below sorts into that pipeline.
Can AI crawlers even read your site?
This is the highest-impact check and almost nobody runs it. If the crawlers cannot fetch your pages, every hour spent on content and schema is wasted — and the block is usually invisible from your own browser.
Two things block AI crawlers in practice:
- CDN and firewall bot-blocking. Cloudflare and similar services ship a "block AI scrapers and crawlers" toggle. Flip it on — or leave an aggressive bot-management default — and GPTBot gets a 403 while your customers see a perfectly healthy site.
- robots.txt. Some sites blanket-blocked AI agents in 2023–24 to opt out of training and never revisited the decision. That also opts them out of being cited.
It matters because there are two different kinds of AI crawler, and they serve different purposes:
| Type | Agents | What blocking it costs you |
|---|---|---|
| Live-retrieval fetchers | OAI-SearchBot, ChatGPT-User, Perplexity-User, PerplexityBot | Citations. These fetch pages while composing answers — blocking them removes you from AI answers directly. |
| Training crawlers | GPTBot, ClaudeBot, Google-Extended, CCBot | Long-term model knowledge of your brand and category. |
The thirty-second test: curl -A "GPTBot" https://yoursite.com/ and confirm you get HTTP 200 with real HTML, not a challenge page. Our AEO checker runs this probe automatically, plus a robots.txt evaluation for six answer-engine agents.
What does answer-first content look like?
Models extract best from passages that answer the implied question in the first one or two sentences and read correctly when quoted in isolation. Four rules cover most of it:
- Lead with the answer, 40–60 words. The first paragraph under your H1 should fully answer the page's core question — no wind-up, no "in today's fast-moving world". If a model quoted only that paragraph, would the reader be satisfied? (Notice this article opens with a definition, not a story.)
- Name the entity, kill the pronouns. "WisWes supports Shopify, Magento and Shopware" is extractable. "We support all major platforms" is a sentence about nobody. Pronoun openings break attribution — the brand token is what gets associated with the claim.
- Make headings real questions. Buyers type questions into answer engines. An H2 like "How is WisWes priced?" with the answer immediately beneath is a ready-made match; an H2 like "Pricing" is a label the model has to work around.
- Put numbers in sentences. A sentence carrying a concrete figure — "40 languages", "a 14-day trial", "$0.05 per extra conversation" — is far more likely to be lifted verbatim than an adjective. Only where true: models down-rank superlatives they cannot verify.
Tables and lists deserve a special mention: they are the most extractable shapes on the web. Comparison pages that open with a verdict paragraph and a feature table are the single best-performing AEO pattern for commercial queries.
Which structured data actually matters for AEO?
Schema.org JSON-LD is the machine-readable layer answer engines inherit from a decade of search. It is unsexy, it is invisible to visitors, and it is the closest thing AEO has to a cheat code — because most competitors half-implement it.
| Schema | What it does for you |
|---|---|
| FAQPage | Ships ready-made question–answer pairs. The highest-yield block: one page-scoped FAQ with real questions and 40–60 word answers beats one giant sitewide FAQ. |
| Article / Product / SoftwareApplication | Declares what the page is, so the model knows whether it is reading a guide, a product, or an opinion. |
| Organization + sameAs | Resolves who is making the claims. Linking the entity to its LinkedIn, marketplace listings and GitHub is a primary trust signal. |
| datePublished / dateModified | Freshness. On fast-moving topics, answer engines strongly prefer recently-updated sources. Keep dateModified honest — real updates, not timestamp games. |
| Person author | E-E-A-T. For analysis and opinion, a named author with a bio outweighs a faceless org byline. |
| BreadcrumbList | Places the page in your site’s topic hierarchy — cheap to add, quietly useful. |
One warning: broken JSON-LD is ignored entirely. A syntax error in the block is the same as not having it — validate after every change.
What about llms.txt?
Skip it, unless you sell developer tools. No major answer engine has committed to reading llms.txt for retrieval, Google has said outright it will not, and live crawler logs show the AI bots fetching regular HTML pages while walking straight past the file. We wrote the long version in llms.txt is a sign nobody reads. The short version: optimize the surfaces crawlers have publicly agreed to read — HTML, JSON-LD, robots.txt, sitemaps — not the ones you wish they read.
How do you measure whether AEO is working?
AEO wins do not show up in rank trackers. Three measurements cover it:
- AI referral traffic. In GA4, segment referrals from
chatgpt.com,perplexity.ai,gemini.google.com,copilot.microsoft.comandclaude.ai. This is the revenue-side proof: people clicking through from AI answers. - Prompt audits. Periodically ask the major engines your target prompts — "best AI chat for Shopify", "Gorgias alternatives" — and log whether you are named, cited, and described accurately. It is the AEO equivalent of rank tracking, and today nothing automates it credibly.
- Bot hits in server logs. GPTBot, ClaudeBot and PerplexityBot appearing in your access logs is the leading indicator that crawlability is solved — it shows up weeks before citations do.
How AEO-ready are your pages right now?
Everything above is machine-checkable, so we built the machine: the WisWes AEO checker. Paste any URL and it fetches the page exactly the way an answer engine does — raw HTML, no JavaScript — then runs 25 checks across six categories: crawler access (including a live GPTBot probe and robots.txt rules for six AI agents), answer-first content, question-shaped structure, structured data, metadata and extractability, and freshness. You get a 0–100 score and your three highest-impact fixes, in order.
To calibrate it, we pointed the checker at one of the most polished pages on the web: apple.com scores 66/100. Crawler access is a perfect 22/22 — Apple blocks nothing — but the answer layer is thin: the homepage opens with a 6-word paragraph, "Apple" is not named in the first two paragraphs, and there is no FAQPage schema and no dates in the structured data. The best-funded web presence on earth is only two-thirds ready for answer engines. That is how new this discipline is — and how much room there is to be earlier than your competitors.
Run it on your homepage, your best-ranking blog post, and one product page. The gap between how those three score is usually the whole AEO to-do list.
Frequently asked questions
What is AEO in simple terms?
AEO (Answer Engine Optimization) is the practice of structuring your website so AI answer engines — ChatGPT Search, Perplexity, Google AI Overviews, Gemini, Copilot — can crawl your pages, extract a clean answer from them, and cite you as the source. SEO optimizes for a ranked list of links; AEO optimizes for being the quoted answer.
How is AEO different from SEO?
SEO ranks whole pages; AEO extracts passages. A page can rank #1 in Google and still never be quoted by an answer engine because its answer is buried, its headings are labels instead of questions, it has no FAQ schema, or its CDN blocks AI crawlers. AEO builds on an SEO foundation but adds crawler access, answer-first writing and extraction-friendly structure.
Do AI answer engines execute JavaScript?
Mostly no. Live-retrieval fetchers like ChatGPT-User, OAI-SearchBot and PerplexityBot read the server HTML and render little or no JavaScript. If your content only appears after client-side rendering, answer engines see an empty page. Server-side rendering is a hard prerequisite for AEO.
What is the fastest way to improve AEO?
In order of impact: verify AI crawlers get HTTP 200 from your pages (CDN bot-blocking is the most common silent killer), lead every key page with a self-contained 40–60 word answer, phrase H2s as real buyer questions, and add page-scoped FAQPage JSON-LD. Each is checkable in minutes with an AEO scoring tool.
Does llms.txt help with AEO?
For most businesses, no. No major answer engine has committed to reading llms.txt for retrieval, and Google has said it will not support it. The surfaces that actually move citations are the ones crawlers already read: server HTML, schema.org JSON-LD, robots.txt and sitemaps.
The takeaway
- AEO optimizes for being the quoted answer; SEO optimizes for the ranked link. You need both, but they fail differently.
- Answer engines run a three-step pipeline — fetch, extract, trust — and your page can be eliminated silently at any step.
- The most common AEO failure is step one: a CDN or robots.txt blocking AI crawlers. Test it with a GPTBot user-agent before touching content.
- Answer-first copy wins extraction: a self-contained 40–60 word opening, entity names instead of pronouns, question-shaped H2s, tables, and sentences with real numbers.
- JSON-LD — FAQPage, page type, Organization with
sameAs, honest dates, a named author — is the trust layer, and broken JSON-LD counts as none. - Measure with AI referrals in GA4, periodic prompt audits, and bot hits in server logs — not rank trackers.
- Score any page 0–100 in seconds with the free WisWes AEO checker.