llms.txt explained: does your website need one in 2026? (setup guide)
- Aug 6
- 6 min read
llms.txt is a Markdown file you place at the root of your website - for example, seomama.pro/llms.txt - that hands AI models a short, curated map of your most important pages in a format they can read quickly and cheaply. Think of it as a hand-picked reading list for large language models (LLMs) instead of leaving them to crawl your whole site and guess what matters.
Does a typical business website need one in 2026? The honest answer for most small and mid-sized sites is: not yet. As of mid-2026, no major AI provider - OpenAI, Google, Anthropic, or Perplexity - has publicly confirmed that its crawlers read llms.txt, and Google's John Mueller has said server logs show the AI bots do not even request the file (Ahrefs, 2026). The file is close to zero-cost to publish, so a few narrow cases still make sense - but for citation or ranking lift, it currently does nothing verifiable. Below: what the file is, where adoption really stands, how it differs from robots.txt and your sitemap, whether the crawlers use it, and exactly how to set one up.
What this guide covers
What is llms.txt?
The llms.txt standard, documented at llmstxt.org, is a single plain-text file written in Markdown and served from your domain root. The only strictly required element is an H1 with your site or project name. From there you can add an optional one-line blockquote summary, then H2 sections that list curated Markdown links, each with a short human description. A final Optional section holds secondary links a model can safely skip to save context (Answer.AI, 2024).
The philosophy is simple: site authors know best which pages matter, so they curate a clean list instead of forcing a model to crawl everything and infer relevance. Many adopters also publish companion Markdown versions of key pages - a plain-text twin without navigation, ads, or scripts - so the model reads the substance, not the page furniture.
Who created llms.txt, and where does adoption stand in 2026?
llms.txt was proposed by Jeremy Howard, co-founder of Answer.AI (and of fast.ai), on September 3, 2024 (Answer.AI, 2024). The pitch resonated with the developer-tools community almost immediately.
Adoption in 2026 is real but modest. SE Ranking's analysis of 300,000 domains found roughly 10% now carry an llms.txt file, and - tellingly - large high-traffic sites adopt slightly less (around 8%) than mid-traffic sites (around 11%) (OrganiKPI citing SE Ranking, 2026). The heaviest uptake is in software and API documentation: Anthropic, Mintlify, and many developer-doc platforms publish one, which hints at where the file is genuinely useful.
llms.txt vs robots.txt vs sitemap.xml
These three files are often confused, but they do very different jobs:
robots.txt - a permissions file. It tells crawlers what they may not access. It restricts; it never recommends.
sitemap.xml - a discovery file. It lists every URL so search engines can find and index your whole site at scale.
llms.txt - a curation file. It highlights only your best, most relevant pages in clean Markdown with human descriptions, giving a model signal without noise.
The key takeaway: llms.txt replaces neither. Keep robots.txt and your sitemap exactly as they are - llms.txt is purely additive.
Do AI crawlers actually use llms.txt yet?
This is where hype and reality diverge, so here is the balanced picture as of 2026:
Google: John Mueller compared llms.txt to the old keywords meta tag and said none of the AI services have stated they use it - and that server logs show they do not even check for it. Google says AI Overviews and AI Mode rely on traditional SEO signals (Ahrefs, 2026; OrganiKPI, 2026).
OpenAI: GPTBot honors robots.txt, but OpenAI has not said it reads llms.txt (Ahrefs, 2026).
Anthropic: publishes its own llms.txt but has not claimed its crawlers consume the standard (Ahrefs, 2026).
Perplexity and Meta: no documented support (OrganiKPI, 2026).
Two more sobering data points: Ahrefs reports that around 97% of llms.txt files receive zero requests, and one analysis found that removing llms.txt as a variable actually improved AI-citation prediction accuracy - meaning today it adds noise, not signal (Ahrefs, 2026; OrganiKPI, 2026).
That said, the file is not useless. Its clearest current value is with AI coding assistants and developer tools you explicitly point at it - Mueller called it a temporary crutch to save some tokens - and with your own internal RAG pipelines or MCP servers, where you control exactly what the model reads (Ahrefs, 2026).
Does your business site need one?
Use this simple decision test. Skip it for now if:
You run a standard marketing, local-service, or small e-commerce site and your goal is AI citations or higher rankings - the file delivers no verifiable lift today.
Consider adding it if:
You publish developer or API documentation aimed at programmatic or LLM consumers, where the token savings are measurable.
Your CMS or framework auto-generates it - at zero marginal cost, it is pure option value if providers adopt the standard later.
You run internal RAG or AI-agent workflows and want a curated, self-controlled content index.
If citations are your real goal, the levers that move the needle are content structure and authority, not a text file - see our breakdown of what actually matters for service-page SEO in 2026. And if you are still choosing a platform, note that many modern AI website builders now generate files like this for you automatically.
How to set up llms.txt: a step-by-step guide
If you have decided to publish one, it takes only a few minutes:
Audit your best content. List the 10-30 pages that best explain who you are and what you do - core services, key guides, pricing, contact. Skip thin or duplicate pages.
Write it in Markdown. Start with an H1 site name, add an optional one-line blockquote summary, then group curated links under H2 headings, each with a short description.
Add an Optional section. Put lower-priority links under an Optional heading so models can drop them to save context.
(Optional) Create clean .md versions of key pages, so the model reads plain text without navigation or ads.
Upload to your domain root. Save the file as llms.txt so it resolves at yourdomain.com/llms.txt - the same place as robots.txt.
Verify and maintain. Confirm it loads as plain text (HTTP 200), keep robots.txt and sitemap.xml in place, and update the list whenever key pages change. A stale file is worse than none.
Here is a minimal example you can adapt:
# SEO MAMA
> Practical SEO and AI-search optimization for small and mid-sized businesses.
## Core pages
- [SEO Optimization Services](https://www.seomama.pro/services/seo-optimization): What our SEO service includes.
- [Service Page SEO: What Matters in 2026](https://www.seomama.pro/post/service-page-seo-what-matters-in-2026): How to structure service pages for search and AI.
## Guides
- [llms.txt Explained (2026)](https://www.seomama.pro/post/llms-txt-explained-2026): What llms.txt is and whether you need one.
## Optional
- [Home / Contact](https://www.seomama.pro/): Company background and how to reach us.Bottom line for 2026: llms.txt is a low-effort, low-risk file with genuine value for documentation and internal AI tooling - but it is not yet a ranking or citation tactic for the average business website. Publish one if it is free to generate; otherwise invest in content quality, structure, and authority first. If you want help deciding what truly moves your AI and search visibility, our SEO optimization services can audit your site and prioritize the work that pays off.
Frequently asked questions
Does llms.txt help my Google rankings?
No. Google's John Mueller has said llms.txt is not used for Search and compared it to the old keywords meta tag; AI Overviews and AI Mode rely on traditional SEO signals (Ahrefs, 2026).
Do ChatGPT, Claude, or Perplexity read llms.txt?
As of 2026, none of these providers has confirmed its crawlers consume llms.txt, and server-log analyses show most files are never requested (Ahrefs, 2026). Its practical use today is mainly for AI coding tools and internal pipelines you point at the file directly.
Is llms.txt the same as robots.txt?
No. robots.txt restricts what crawlers may access, while llms.txt curates and recommends your best content for LLMs. They serve opposite purposes, so keep both.
Where do I put the llms.txt file?
At the root of your domain, so it resolves at yourdomain.com/llms.txt - the same location as robots.txt.
How many websites use llms.txt in 2026?
Roughly 10% of domains in a 300,000-site sample, with adoption concentrated in developer-documentation sites rather than marketing sites (OrganiKPI citing SE Ranking, 2026).
Will llms.txt matter in the future?
Possibly. It costs almost nothing to publish, so it carries option value if providers adopt it - but there is no verified benefit today, so do not prioritize it over content fundamentals.
