The llms.txt file (often searched as “llms txt”) is a Markdown text file published at the root of a website to give language models and AI agents a summary of the site and links to its key content. It is an open proposal, not a standard: Google documents that its search engine ignores it, and no major assistant documents using it to decide which sites to cite.
That said, creating one is cheap and does no harm. What gets expensive is believing it fixes something it does not. Below you will find the exact format according to the original proposal, an example adapted to a small business, and the distinction almost no guide makes: what the proposal says versus what providers have confirmed.
| What it is | A Markdown file with a summary of the site and a curated list of links |
|---|---|
| Where it goes | At the root (/llms.txt) or in a subpath (/docs/llms.txt) |
| Status | Open proposal (version 2, August 2026). Not a standard |
| Google Search | Documents that it does not use it: neither helps nor harms |
| Does it block bots? | No. That is the job of robots.txt |
What llms.txt is and where it comes from
The proposal was published by Jeremy Howard in September 2024 at llmstxt.org and revised in August 2026 (version 2). It starts from a real problem: web pages are built for people. HTML wraps information in menus, banners and JavaScript, and a model that visits a page to answer a question has to separate signal from noise within a limited context window.
The proposed solution is a short Markdown file that says “this is who we are and here is what matters”, with links to clean versions of the pages. According to the text, it is meant to be consulted on demand, when an agent needs information while helping a user, rather than for training models. Where it is used most, the author himself says, is software documentation.
The llms.txt file format and an example
The format is strict about order, even though it is Markdown. A file that follows the proposal contains, in this order:
- An H1 title with the name of the project or site. This is the only required part.
- A blockquote (a line starting with
>) with a short summary that gives the context needed to understand the rest. - Optional free text (paragraphs or lists, no headings) with details and guidance on how to interpret the files.
- H2 sections with lists of links. Each item is a Markdown link
[name](url)and, optionally, a colon and a note.
A section named Optional is used, by convention, for secondary links an agent can skip when it needs a shorter context. This is how it would look for a fictional accounting firm:
# Example Accountants
> Accounting firm in Sabadell for freelancers and small businesses: bookkeeping, tax returns and payroll. Service in Catalan and Spanish.
Useful facts: we only work with companies and freelancers based in Catalonia. Published fees do not include VAT.
## Services
- [Bookkeeping for freelancers](https://www.example.com/services/bookkeeping-freelancers.md): what is included and who it is for
- [Payroll and Social Security](https://www.example.com/services/payroll.md): scope of the service and deadlines
## Company
- [Fees](https://www.example.com/fees.md): prices per plan
- [Contact and opening hours](https://www.example.com/contact.md)
## Optional
- [Blog](https://www.example.com/blog/index.md): articles on tax mattersThree details of the current version that many guides have not caught up with:
- The file can live in a subpath and covers the URLs under that path. Where more than one applies, the most specific one is used.
- The proposal recommends offering a Markdown version of each page at the same URL, either appending
.md(page.html.md) or replacing the extension (page.md). Links in llms.txt should point to those clean versions. - To help discovery it suggests standard link relations:
rel="alternate" type="text/markdown"pointing to the Markdown version andrel="describedby"pointing to the llms.txt that covers the page.
The llms-full.txt file many tutorials mention does not appear in the text of the proposal: it is a practice of some tools, not part of the format.
Differences between llms.txt, robots.txt and the XML sitemap
All three sit at the root under a fixed name, but they do different jobs and none replaces another.
| robots.txt | XML sitemap | llms.txt | |
|---|---|---|---|
| What it is for | Telling crawlers what they may crawl | Listing the URLs you want discovered | Summarising the site and pointing to key content |
| Format | Plain-text directives | XML | Markdown in a fixed order |
| Scope | The whole domain | Every indexable page | A hand-picked selection |
| Who documents using it | Google and the crawlers of OpenAI, Anthropic and Perplexity | Google and other search engines | No search engine or assistant, for ranking or citations |
| Does it control access? | Yes (a request, not a barrier) | No | No |
The proposal itself says so: robots.txt states what access is considered acceptable; llms.txt provides context for the allowed content. And a sitemap is no substitute because it lists every page, not model-readable versions or a selection that fits in a context window. For the detail on each, see our guides to robots.txt and the XML sitemap.
Who reads llms.txt: what is confirmed and what is not
This is the delicate part. We reviewed the providers' public documentation on 9 October 2026, and this is what is there.
Confirmed by the provider itself
- Google Search does not use it. Its guide to optimising for generative AI features lists it among the things you can ignore: you do not need to create machine-readable files or Markdown to appear in Search or its AI features, and having one will “neither harm nor help” visibility. It adds that Google may crawl and index the file like any other, with no special treatment.
- Lighthouse, Chrome's auditing tool, has an llms.txt check within its agentic browsing audits. If the file does not exist, the audit is marked “not applicable” because, it says, providing it is optional; it only fails if the server returns an error. It is a technical check, not a ranking signal.
- OpenAI, Anthropic and Perplexity document their crawlers and how to control them with robots.txt. Those pages do not say their bots read the llms.txt of the sites they visit.
- Those same companies publish an llms.txt for their own developer documentation. Publishing one is not the same as consuming everyone else's.
Not confirmed
- That ChatGPT, Claude, Perplexity or Gemini check your llms.txt before answering.
- That having one makes you more likely to be cited.
- That it influences any ranking.
The proposal states that coding agents routinely use these files to find documentation. That is its author's claim and it fits the technical-documentation use case, but it is not a commitment from any search engine.
Does a small business need an llms.txt?
My view, without drama:
- It makes sense if you sell software, have an API or publish technical documentation that developers consult through coding assistants. That is the use case the proposal describes as the main one.
- Go ahead without guilt if your content management system generates it automatically and keeps it up to date. Close to zero cost.
- It is not a priority for a restaurant in Gràcia, a clinic in Tarragona or a law firm in Girona. Other work comes first, and that work does count.
That work is the usual one: pages that can be crawled and indexed, that answer specific questions well, on a site that is a reliable source. Google documents that, to appear as a supporting link in its AI answers, a page must be indexed and eligible to be shown with a snippet; nothing more. We go deeper in our guides to GEO, AEO and SEO and SEO for AI, and it is what we review in our technical SEO service.
If you decide to create one, do it in four steps: pick five to fifteen pages that explain the business, write the summary in two sentences, link to clean versions and test it. The proposal itself suggests the test: give an assistant only the file and ask it questions about your company.
Common llms.txt mistakes
- Using it to block. It is not a robots.txt and has no access directives.
- Dumping every URL into it. That is what the sitemap is for. The value lies in the selection.
- Letting it go stale. A file with old prices or services contradicts the site, and a source that contradicts itself is worse than no source.
- Linking pages you do not want public. The file is public and Google may index it.
- Paying for it as if it were ranking work. If someone sells it as a guarantee of appearing in ChatGPT, ask where the provider documents that.
If you want to know what is really holding back your site's visibility, in search engines and assistants alike, ask for our free initial analysis: we review crawling, indexing and content, and tell you where to start.
Frequently asked questions
Does llms.txt improve Google rankings?
No. Google documents that its search engine does not use these files and that having one neither helps nor harms visibility or rankings.
Is an llms.txt file mandatory?
No. It is a voluntary proposal. Even the Lighthouse audit treats it as optional and does not penalise its absence.
Where does the file go?
At the root of the domain, as https://www.example.com/llms.txt, or in a subpath if you only want to cover part of the site, such as the documentation.
Can it stop AI from using my content?
No. To limit access by AI crawlers you use robots.txt with each bot's name, which is the mechanism the providers document.
Is llms.txt the same as llm.txt?
The name the proposal defines is llms.txt, plural. “llm.txt” is a common slip; a file with that name does not follow the specification.
