Jason Burns / jasonburns.co.uk
Available - taking new work Contact

Updated

Updated What changed on
  • Corrected the heading structure. The summary heading was an h3 sitting directly after the h1, which skipped a level and gave screen readers a broken page outline. No wording changed.
llms.txt Proposed spec by Jeremy Howard

What is llms.txt? Plain answer, real example.

The 30-second answer

llms.txt is a proposed markdown file you publish at the root of your site (example.com/llms.txt) that tells AI systems which content matters most and how to read it. Proposed by Jeremy Howard at llmstxt.org in late 2024. Not an official W3C standard. Adoption by major AI engines is mixed. For documentation sites the value is real. For most marketing sites it is cheap insurance, not a major ranking lever.

example.com/ - llms.txt file tree example.com/ |-- llms.txt # required - summary + key URLs |-- llms-full.txt # optional - expanded version |-- robots.txt # complementary, not replaced |-- /services/ |-- ai-consulting.html |-- ai-consulting.md # optional markdown twin |-- geo-consulting.html |-- /insights/ |-- what-is-rag.html |-- what-is-rag.md Yellow = llms.txt-spec files. Teal = optional markdown twins of key HTML pages.
llms.txt sits at the site root alongside robots.txt. Optional markdown twins let crawlers read clean text instead of parsing HTML.

What it looks like

A valid llms.txt is plain markdown. There is one H1 (the site title), a blockquote summary, then sections of named links. Example shape, taken from the spec:

# Site Name > One-paragraph description of what this site is. ## Docs - [Getting Started](https://example.com/docs/start.md): Quick install and first request - [API Reference](https://example.com/docs/api.md): Complete endpoint listing ## Optional - [Background](https://example.com/about.md): Project history and design decisions

The "Optional" section is meant to be skippable. AI systems with limited context budget can stop reading after the main sections.

My own llms.txt - and what changed

This site publishes one at /llms.txt. I added it in early 2026, kept the format minimal, and pointed to the services pages plus the four free tools. The honest answer about what changed: no measurable difference in AI citation rates that I can attribute specifically to llms.txt versus the other GEO work running in parallel. That is consistent with the broader picture - the spec is real, the major engines have not confirmed it as a ranking signal.

What llms.txt is good for

  • Documentation sites. Anthropic publishes one for their developer docs. The format suits a docs site genuinely well because the structure (named sections, ordered priority) maps to how docs are already organised.
  • Sites with clear hierarchy. If you have a content tree that a reader needs to navigate in a specific order, llms.txt is a way to communicate that ordering to an AI consumer.
  • SaaS docs and developer tools. The early adopters concentrate here for a reason - their audience already lives at the intersection of docs and LLM consumption.
Primary sources worth reading direct: llmstxt.org (Jeremy Howard's original proposal), the spec on GitHub (AnswerDotAI/llms-txt), the live directory of sites that have adopted it, and the Search Engine Journal sceptical take for the other side of the argument.

What llms.txt is probably not doing for you

  • Lifting ranking in ChatGPT, Perplexity, Claude or Gemini. None of these have confirmed llms.txt as a citation signal. The schema, the named author, the content depth and the bot access all matter more for citation.
  • Replacing structured data. Schema.org JSON-LD is still doing the heavy lifting for AI extraction. llms.txt does not substitute for it.
  • Replacing your sitemap.xml. Sitemaps still feed traditional search crawlers. llms.txt is for AI consumption, not traditional indexation.

Adoption in 2026: the actual numbers

SE Ranking analysed 300,000 domains in November 2025 and found:

  • 10.13% of domains had an llms.txt file in place. Roughly 1 in 10. A long way from the universal adoption of robots.txt or sitemap.xml.
  • No correlation between having llms.txt and getting cited in AI engine answers. The data shows the file does not currently influence citation frequency.
  • High-performing domains (top traffic) adopted llms.txt at similar rates to the long tail. Not a "winners do this" signal.

The honest read on that data: llms.txt is gaining slow traction with the documentation-site and SaaS-tools end of the market, while doing nothing measurable for AI citation in the broader market. Ahrefs' Patrick Stox wrote the same conclusion in his March 2026 piece - "So is llms.txt actually useful? In my opinion, no, not yet."

So should you publish one anyway?

Yes for docs sites and SaaS tools where the format genuinely maps to your content. The cost is 30 minutes once, the upside is real for developer-audience LLM consumption.

Probably yes for marketing sites - but as low-effort insurance, not as a major lever. Treat it like adding a noindex on staging: takes 30 minutes, removes one small future-proofing risk, do not expect a measurable lift.

No if it is sitting on a sprint backlog displacing higher-impact work. The schema audit, the bot access fix, the named-author setup and the content depth investment all return more per hour than tuning llms.txt.

How to publish one in 30 minutes

  1. Generate a starter file from a tool. Firecrawl's free generator crawls a domain and produces a draft. WordPress sites can use the Yoast or Rank Math feature.
  2. Hand-edit the draft. Cut anything that is not load-bearing. Keep the H1 + summary + 5-15 named links per section.
  3. Publish at the root: https://yoursite.com/llms.txt. Make sure your server serves it as text/plain or text/markdown.
  4. Validate the format at llmstxtvalidator.dev.
  5. Add a link to it from your homepage footer or your robots.txt comments. Not required by the spec but helps human discovery.
// questions I get

Questions about llms.txt.

Is llms.txt an official standard?

No. It is a proposal by Jeremy Howard, published at llmstxt.org in late 2024. It has not been ratified by the W3C or IETF. Adoption by major AI engines is mixed. Treat it as a low-effort signal, not a guaranteed ranking lever.

Where should llms.txt be hosted?

At the root of your domain: example.com/llms.txt. Some implementations allow alternative paths but the root is what most AI systems check first.

Do major AI engines actually use llms.txt?

No major AI vendor has publicly confirmed llms.txt as a ranking or citation signal. Anthropic publishes one for their docs. Some smaller AI tools claim to use it. The practical position: publishing one is cheap insurance, not a guaranteed channel.

Is llms.txt the same as robots.txt?

No. robots.txt restricts which bots can crawl which paths. llms.txt is positive guidance about which content matters most and how to read it. They are complementary, not interchangeable. A site can and should have both.

Should I hand-write or generate my llms.txt?

Generate a starter version from a tool (Firecrawl, Yoast llms.txt, or similar), then hand-edit. Auto-generated llms.txt files often list every page on the site. The value is in curation - tell AI systems which pages actually matter. The hand-edit typically takes thirty minutes.

Jason Burns, independent UK SEO, GEO and AI consultant
Written by

Jason Burns

Independent UK SEO, GEO and AI consultant. 17 years in search. Portfolio includes 3M, BlackRock, Unilever and E.ON. Owner of SEO Moves Ltd since 2014.

More about Jason →
// next step

Want a hand publishing yours?

Send your sitemap or just the homepage URL. I come back with a draft llms.txt you can publish today plus the validation steps. No call needed unless you want one.

Send your URL