Nick QuirkSearch isn't only Google anymore. A growing share of your potential customers ask ChatGPT, Claude, or Perplexity — and those assistants decide, in milliseconds, whether your site is worth citing. llms.txt is the emerging standard for making that decision easy.
The spec in one paragraph
Proposed at llmstxt.org, llms.txt is a plain-markdown file at your site root: an H1 with your site name, a one-line blockquote describing it, and sections of links with short descriptions. Think of it as a curated sitemap written for language models — instead of forcing an AI to crawl and guess, you hand it a clean map of what you offer and where.
What it actually does for you
- Better citations. Assistants that fetch your llms.txt get accurate titles and descriptions, so when they cite you, they cite the right page for the right reason.
- Faster comprehension of new content. A fresh page listed in llms.txt is one fetch away from being understood, rather than waiting for a full recrawl.
- Control of the narrative. The descriptions are yours. That's a marketing surface, not just plumbing.
llms.txt vs robots.txt: allies, not rivals
robots.txt says where crawlers may go; llms.txt says what's worth reading. Run both deliberately: many sites allow answer-engine crawlers (they drive citations and traffic) while blocking pure training crawlers. That's a per-bot decision in robots.txt — GPTBot, ClaudeBot, PerplexityBot, Google-Extended, CCBot, and Bytespider all identify themselves.
The gotchas when publishing from WordPress
Two failure modes show up constantly. First, plugins that generate the file dynamically often 404 on managed hosts like WPEngine, whose caching layers bypass WordPress for unknown URLs — a static file written to the webroot is the reliable approach. Second, naive plugins regenerate the file on every post save; since nav menu items are technically posts, saving a large menu can trigger dozens of full regenerations and exhaust PHP memory.
Power Up WP's llms.txt module (in both the free and pro versions) writes a static file with an ownership marker (it will never overwrite a file it didn't create), regenerates at most once per request when content changes, has no page cap, and pairs with per-crawler allow/block controls for the AI bots above.