AEO checklist for WordPress sites
Short answer
On WordPress, the most common reasons AI assistants cannot use a site are: the “Discourage search engines” box left ticked, robots.txt rules or plugin settings that block AI search crawlers, security plugins or the host’s firewall rejecting those crawlers, and two plugins adding conflicting structured data. Check those first, then improve your key pages so they answer customers’ questions directly.
1. WordPress settings
- Settings → Reading: “Discourage search engines from indexing this site” must be unticked. It is often left on after a site goes live.
- Settings → General: the WordPress Address and Site Address both use
https://and the same host (with or withoutwww, but the same in both). - Settings → Permalinks: use readable addresses (“Post name”), not
?p=123. - Appearance → Customize → Site Identity: set a Site Icon (favicon). It appears next to your site in Google results.
2. robots.txt and crawler access
- Open
yoursite.com/robots.txt. There should be noDisallow: /underUser-agent: *. - Make sure these are not blocked:
OAI-SearchBot,Claude-SearchBot,PerplexityBot,Googlebot,Bingbot. Blocking training crawlers such asGPTBotandClaudeBotis a separate choice that does not remove you from AI search. - Do not block
/wp-content/or/wp-includes/: crawlers need the CSS and scripts to render pages. - Check your SEO plugin’s crawl settings: some include switches to block AI bots. Make sure they block only the crawlers you intend to.
The full list of crawlers is in AI crawlers and robots.txt.
3. Security plugins, firewall and CDN
- In security plugins (Wordfence, Solid Security, All-In-One Security and others), review rate limiting and bot blocking. Rules that block “unknown crawlers” can catch AI search crawlers.
- On Cloudflare, check the bot settings and the AI crawler settings. Cloudflare blocks AI crawlers by default for new domains unless you choose otherwise.
- Ask your host whether their firewall blocks data-centre IP ranges or unusual user agents.
- Test: fetch a page with an AI crawler’s user agent and confirm you get the real page, not an error, a CAPTCHA or a challenge page.
4. Indexing
- Your sitemap loads:
/wp-sitemap.xml(built into WordPress) or/sitemap_index.xml(Yoast, Rank Math). - The sitemap is listed in robots.txt and submitted in Google Search Console and Bing Webmaster Tools. Bing matters for Microsoft Copilot.
- Important pages are not set to
noindexin the SEO plugin’s page settings. - Only one plugin outputs canonical tags and meta descriptions.
5. Structured data
- Fill in the SEO plugin’s organisation (or local business) settings: official name, logo, contact details, social profiles.
- Use one plugin for schema. A theme or page builder adding its own markup on top creates conflicting data.
- Test the home page and a product or service page in Google’s Rich Results Test.
More detail, with examples, in Structured data for AI search.
6. Content
- The home page says, in the first lines, what you do, for whom and where.
- Each main service or product has its own page that answers the questions customers ask: price range, process, timing, areas served.
- The About and Contact pages show who runs the business and how to reach you, with the same details as your Google Business Profile.
- Page builders: check that the main text is in the page’s HTML, not loaded later by scripts. View the page source and search for a sentence from the page.
- Images have meaningful alt text (Media → Library).
7. Speed and stability
- A caching plugin or server cache is on, so pages load quickly for crawlers as well as people.
- The site has no PHP warnings or errors printed into pages or the sitemap.
- SSL certificate renews automatically.
Questions
Where is robots.txt in WordPress?
By default WordPress generates a virtual robots.txt; there is no file to edit. SEO plugins such as Yoast SEO and Rank Math let you edit it from their settings. If a real robots.txt file exists in the site’s root folder, it replaces the virtual one.
Do I need a plugin for llms.txt?
No. llms.txt is optional and Google does not use it. If you want one, some SEO plugins can generate it; otherwise leave it until the more important items on this list are done.
Which SEO plugin is best for AEO?
Any of the established ones works. What matters is using only one, filling in its organisation or local business settings correctly, and checking that its bot-blocking options do not block AI search crawlers.