llms.txt Generator & Validator

What is llms.txt and why it exists

llms.txt is a Markdown file served from the site root at /llms.txt that tells a language model, in a few lines, what the site is about and where the important content lives. It was proposed by Jeremy Howard of Answer.AI in September 2024; the spec is published at llmstxt.org. The reasoning is simple: models have a limited context window, and an HTML page is mostly navigation, banners, scripts and markup. A short file with a summary and annotated links saves the model that work and lowers the chance it misreads your site.

Keep its status in mind: llms.txt is a proposed standard, not a protocol that search engines or AI services are bound to follow. What is actually known about adoption is covered in the last section.

llms.txt format: what the spec requires

The spec defines blocks in a fixed order. Only the first one is required.

1. An H1 with the name of the project or site: # Name. It is the first line of the file.
2. A blockquote summary: > … — the gist in 1–3 sentences, with the key facts needed to understand the rest.
3. Optional detail blocks: paragraphs, lists — any Markdown except headings. Use them to explain how to read the file, terms, limitations.
4. H2 sections with “file lists”. Each item is a Markdown link, optionally followed by a colon and a note: - [Name](https://…): what the page covers.

llms.txt example

# Kolosok Bakery

> Craft bakery in Kyiv: sourdough bread, pastries and same-day
> delivery across the city from 8 am to 8 pm.

Orders are taken on the website and by phone. Prices include VAT.

## Catalog

- [Sourdough bread](https://example.com/bread): Ingredients, weight, prices
- [Pastries](https://example.com/pastry): Croissants, buns, seasonal menu

## Delivery and payment

- [Delivery](https://example.com/delivery): Zones, cost and times
- [Payment](https://example.com/payment): Payment methods and refunds

## Optional

- [Our story](https://example.com/about)

The Optional section

A section titled exactly ## Optional has a special meaning: its links can be skipped when a shorter context is needed. Put secondary material there — company history, legal pages, archives.

Markdown versions of pages

The spec also suggests serving a clean Markdown version of a page at the same URL with .md appended (/docs/start.html.md, or index.html.md for URLs without a file name). This is optional, and most sites simply link to their regular HTML pages from llms.txt.

llms.txt vs robots.txt vs sitemap.xml

The key consequence: llms.txt neither blocks nor allows anything. To keep AI crawlers or model training away from your content, use robots.txt — test it with the robots.txt checker or build one with the generator. Conversely, if an AI crawler is blocked in robots.txt, publishing llms.txt does not change that. Your full URL list still belongs in the sitemap; the sitemap checker shows its state.

llms-full.txt: when a second file makes sense

llms-full.txt is a convention popularized by documentation platforms: one file containing the full documentation text rather than just links, so a model gets everything in context without visiting pages. It is not part of the original spec, which describes llms-ctx.txt and llms-ctx-full.txt files built by the llms_txt2ctx tool (the full variant also includes the Optional links).

llms-full.txt is worth it for documentation, APIs and knowledge bases, where the whole text is needed. A store or a service site usually does fine with llms.txt alone. Watch the size: a multi-megabyte file will not fit into many models’ context windows. Our validator checks llms-full.txt more leniently: it requires an H1 and valid links but does not expect every section to be a link list.

How to add llms.txt to your website

The file should open at https://your-domain/llms.txt with a 200 status, UTF-8 encoding and a text/plain or text/markdown content type. The spec also allows a file in a subpath (e.g. /docs/llms.txt), but the root is where clients look first.

WordPress

Upload the file over FTP or your host’s file manager to the folder that contains wp-config.php. WordPress’s default .htaccess does not intercept existing files, so nothing else is needed. Popular SEO plugins (Yoast SEO, Rank Math) added llms.txt generation in 2025 — if you use that feature, don’t also upload a physical file, or it becomes unclear which version is live.

nginx

location = /llms.txt {
    charset utf-8;
    try_files $uri =404;
}

try_files $uri =404 matters for SPAs and sites that route every unknown URL to index.html: without it a missing file returns an HTML page with a 200 status, and clients get markup instead of text. charset utf-8 adds the encoding to Content-Type so non-Latin characters are not garbled.

Apache

AddCharset utf-8 .txt .md

This .htaccess line adds the charset to text files. If mod_rewrite rules send every request to a front controller, make sure they are preceded by RewriteCond %{REQUEST_FILENAME} !-f.

Shopify, Wix, Squarespace and other hosted builders

Not every builder lets you put an arbitrary file in the root. First look for a root file upload or a dedicated llms.txt setting. If there is none, one workaround is a 301 redirect from /llms.txt to a file in the platform’s file storage, provided the platform allows redirects from non-existent paths. Most HTTP clients follow redirects, but serving the file directly is more reliable. Check the live result afterwards: a common builder issue is a “404” HTML page returned with a 200 status.

Common llms.txt mistakes

Write link notes for someone who has never seen the site: what the page contains and when to open it. “Learn more” or a repeat of the title adds nothing. The generator uses meta descriptions as a starting point, but marketing copy from snippets is better replaced with facts.

Do AI systems actually use llms.txt?

• None of OpenAI, Google, Anthropic or Perplexity has officially said that its search or crawlers use llms.txt when choosing sources for answers.
• Google representatives said publicly in 2025 that Google does not use it, and Google requires no special files for AI Overviews — normal indexing rules apply.
• Many companies publish llms.txt for their own docs (Anthropic among them). That shows the file is handy for developers and AI agents that are explicitly pointed at it, not that it is a ranking signal.

The practical takeaway: create llms.txt if you have documentation, an API or a product people use through AI assistants — it takes half an hour. Don’t expect it to bring traffic from ChatGPT or AI Overviews on its own: visibility in AI answers still comes from pages that get indexed, answer real questions and earn links from other sites. To see where your site is missing visibility in regular search, try the SEO Radar.