What an llms.txt file is, and whether you actually need one

Start with the part most articles bury: Google does not use llms.txt. Gary Illyes said so at a Search Central event in July 2025 — Google "doesn't support LLMs.txt and isn't planning to" — and nothing since has walked that back. OpenAI and Anthropic have not committed to it either.

So the honest answer to "will an llms.txt file get me into ChatGPT" is no.

That is not the same as the file being useless, and the gap between those two statements is where most of the confusion lives.

What the file is

llms.txt sits at the root of a site — https://example.com/llms.txt — and it is Markdown, not the directive syntax of robots.txt. It is a curated index: the name of the site, a sentence or two saying what the site is, and a list of links with a few words of context each.

# Example Ltd

> We refurbish and sell used lab equipment across the EU. Founded 2009, based in Utrecht.

## Product
- [Centrifuges](https://example.com/centrifuges): 140 models, with service history and warranty terms.
- [Shipping and returns](https://example.com/shipping): timelines and costs per country.

## Company
- [How we grade refurbished equipment](https://example.com/grading): the five-point scale we publish against.

The convention was proposed by Jeremy Howard in September 2024. It has never been through a standards body, no crawler is obliged to fetch it, and the file does not change what any crawler is allowed to access — that is still robots.txt's job, and a page blocked there stays blocked no matter how prominently llms.txt lists it.

Who does read it

Mintlify, Anthropic's own docs, Cloudflare, Stripe and a long tail of developer-tooling companies publish one. The consumers are mostly coding assistants and documentation tools pointed at a specific site by a user — "read the Stripe docs and tell me how refunds work" — rather than the open web crawlers that build a search index.

That is the real use case, and it is a narrow one. If your product has an API, a CLI, or docs that people paste into an assistant, a good llms.txt measurably shortens the path between a developer's question and your correct answer. If you run a dentistry practice, it will do nothing at all.

The thing it is quietly good for

Writing one forces a decision you have probably been avoiding: out of everything you publish, which fifteen pages would you hand to someone who has to understand your business in ninety seconds?

Most sites cannot answer that. The exercise tends to surface two or three pages that should have been merged, one that should have been deleted in 2023, and a category page that is doing the work of a landing page badly. Whether or not a model ever fetches the file, that audit is worth the hour.

If you publish one

Keep it under about 100 links — a file that lists everything is an XML sitemap with extra steps, and sitemaps are better at that. Link to canonical URLs only. Write the descriptions for a stranger, not for an algorithm: "timelines and costs per country" beats "comprehensive shipping information solutions." Serve it as text/plain or text/markdown, and put a date in it so you can tell when it went stale, because it will.

Do not block it in robots.txt. People do this by accident more often than you would think, usually by disallowing a path that happens to catch the root.

What to do instead, if you only have time for one thing

Check which AI crawlers your robots.txt currently allows. A surprising number of sites are blocking OAI-SearchBot or ClaudeBot by accident — inherited from a template, or from a 2023 decision to block training bots that also caught the retrieval bots that surface you in answers. Those are different crawlers with different jobs, and blocking one while meaning the other is the single most common self-inflicted wound we see.

That takes thirty seconds to check and is worth more than any Markdown file.