llms.txt: Google says it does not read it. So who is the file for
Google's own guide says Search ignores llms.txt and that the file neither helps nor harms visibility. Yet Lighthouse in Chrome audits it — hence the confusion. We check who actually publishes the file, who reads it, and when making one is worth it.
Over the past year llms.txt has turned into a mandatory checklist item and a separate line in proposals. The argument is always the same: "so the AI can see you." Google, meanwhile, has stated in its own documentation that Search simply ignores the file. Let us work out what it actually is, who really reads it, and when making one is still worth it.
What the file is
The llms.txt proposal was published by Jeremy Howard in September 2024. The idea is simple and, at first glance, sensible: an HTML page is built for a human — navigation, ads, scripts — and pulling clean text out of it is hard for a model. So let us put a markdown file at the root of the site with a short description of the project and links to the documents that actually matter.
The format is minimal: a first-level heading with the project name — the only required part — a short summary as a blockquote, optional free-form notes, and then lists of links grouped under second-level headings.
An important detail usually left out of proposals: this is not a standard. It is one person's proposal, open to community input. It has no W3C or IETF status, and no search engine is obliged to support it.
What Google says
Google has been about as unambiguous as it gets — in its own guide to optimising for generative AI features on Search.
"You don't need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search… Google Search itself doesn't use them."
"Doing so will neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them."
Google Search Central, guide to optimising for generative AI features on SearchThe wording leaves no room for interpretation: not "we do not use them yet", not "the effect is small", but "ignores them". Which means any offer to "optimise your llms.txt for better Google rankings" is selling something that, by the vendor's own documentation, does not exist.
The same document also dismantles the second popular myth — the one about mandatory markup:
"Structured data isn't required for generative AI search, and there's no special schema.org markup you need to add."
Google Search CentralAnd on the question of what does work, the answer is this: SEO best practices remain relevant because the generative features are rooted in the same core ranking and quality systems as ordinary Search. The requirement for a page is exactly one, and a familiar one: it must be indexed and eligible to be shown with a snippet.
Where the confusion comes from: Chrome audits what Search ignores
Here lies the reason the market believes the opposite. Since May 2026 Lighthouse — the audit tool built into Chrome — has had its "Agentic Browsing" category in the default configuration, and every report now includes an llms.txt check. It looks at whether the file exists, whether it has a first-level heading, whether it is too short, and whether it contains links.
What follows is predictable: a specialist runs Lighthouse, sees a failed check in a Google tool, and concludes that Google requires the file. And the client sees a red mark in the report and asks for it to be cleared.
There is no contradiction here, only different audiences. The Lighthouse category is marked experimental, it has no familiar 0-to-100 score — only a pass ratio — and the tool itself describes llms.txt as a discoverability signal for AI agents, not as a directive for search crawlers. Chrome measures how ready the site is for agents that browse it; Search does not read the file when ranking. Both statements are Google's, and they are about different things.
The practical conclusion: a failed Lighthouse check does not mean an SEO problem. It is a separate experimental metric, and treating it as a ranking factor is a mistake that sells expensively.
Who actually publishes it
The usual argument in favour of the file goes: "all the big AI companies publish it, so it must matter." We decided to check that by hand rather than take it on trust. We requested /llms.txt from documentation addresses on 6 August 2026.
| Address | Response | What is there |
|---|---|---|
docs.anthropic.com/llms.txt | 200 | 58 KB, developer documentation |
docs.perplexity.ai/llms.txt | 200 | 42 KB, Perplexity documentation |
llmstxt.org/llms.txt | 200 | 0.6 KB, the proposal's own site |
platform.openai.com/llms.txt | 404 | no file |
ai.google.dev/llms.txt | 404 | no file |
developers.google.com/llms.txt | 404 | no file |
The picture turned out to be half as impressive as the received wisdom. Anthropic and Perplexity have the file; at the addresses we checked, OpenAI and Google do not. An honest caveat: a company may publish the file at a different path, and its presence on one subdomain says nothing about the others. But the claim that "everyone publishes it" does not survive checking.
Publishing and reading are different things
And that is the central sleight of hand in arguments about llms.txt. The fact that a company puts such a file next to its documentation does not mean its model reads other people's files when answering a user. These are two completely different actions.
Publishing is transparent: the files exist and you can open them and look. Reading is murkier — no major search provider has made a public commitment along the lines of "we use your llms.txt when composing answers". The one unambiguous statement on the subject belongs to Google, and it is a negative one.
Where the file works predictably is in tools you feed it to yourself. Coding agents, documentation assistants, internal helpers: there llms.txt acts as a map telling the tool which pages of your documentation are the important ones. That is exactly why documentation platforms generate it automatically, and exactly why Anthropic and Perplexity have one — in both cases it is developer documentation that agents read.
So should you make one
The answer depends on who you are making it for, and hardly at all on SEO.
What to do instead
Since Google has named outright what does matter, it makes sense to do that instead — especially as the list is short and boring.
Make sure the pages are in the index. The one hard requirement Google states for appearing in generative features: the page must be indexed and eligible to be shown with a snippet. A page outside the index will appear neither in ordinary results nor in an AI answer, and no file at the root of the site will change that.
Do not cut snippets without reason. The nosnippet and max-snippet directives, added at some point against scrapers, exclude the page from generative features as well.
Write what others do not have. In the same guide Google asks for "valuable, non-commodity content" with a viewpoint of its own. That is not a slogan but a direct consequence of how a generative answer is assembled: there is no reason to include a retelling of what is already common knowledge.
None of this looks like news — and that is the point. Generative features are built on the same ranking systems as ordinary search, so the same things work in them.
The one requirement Google actually named. Checking is free.
Frequently asked questions
What is llms.txt in simple terms?
It is a markdown file at the root of a site with a short description of the project and links to its main documents, so that a language model does not have to parse HTML. The proposal was published by Jeremy Howard in September 2024. It is not a standard but an open initiative.
Does llms.txt affect Google rankings?
No. In its own guide Google states that you do not need to create machine readable files, AI text files or Markdown to appear in Search, and that doing so will neither harm nor help visibility, because Google Search ignores them.
Does llms.txt help you get into AI Overviews?
No. Google's generative features are rooted in the same ranking systems as ordinary Search, and there is no separate channel through a file at the root of a site. The requirement for a page is the same: be indexed and eligible to be shown with a snippet.
Who actually uses llms.txt?
What is verifiable is publication: the file exists for Anthropic's and Perplexity's documentation, documentation platforms generate it automatically, and it is read by coding agents and assistants that are given it directly. Major search providers have made no public commitment to using someone else's llms.txt when composing answers.
Can llms.txt harm a site?
The presence of the file in itself, no — Google simply ignores it. Indirect harm is possible: an outdated file pointing at deleted pages misleads the tools that do read it, and the time spent maintaining it by hand brings no measurable return.
Is schema.org markup needed to appear in AI answers?
By Google's documentation, no: structured data is not required for generative AI search, and there is no special markup for it. Markup remains useful for rich results in ordinary search, but that is a different job.
Sources
Google Search Central, guide to optimising for generative AI features on Search: the wording on machine readable files, structured data and the requirements for a page.
llmstxt.org — the llms.txt proposal, first published on 3 September 2024 by Jeremy Howard: format and status of the initiative.
Our own check, 6 August 2026: requests for /llms.txt at six documentation addresses; response codes are given in the table.
