llms.txt is a proposed Markdown file that gives compatible AI agents a short guide to a website and links to selected useful pages. It is a resource map—not a crawler-permission file, a replacement for your site’s content, or a proven way to improve search visibility. Google says Google Search, including its generative AI features, does not use it as special markup.
What is llms.txt?
llms.txt is a proposed file that a site owner can place on a website to provide context and point AI agents toward useful pages. Its author, Jeremy Howard, first published the proposal on September 3, 2024; the proposal’s v2 page was modified August 10, 2026. The proposed goal is to make it easier for compatible agents to find relevant, clean information on sites whose pages are primarily designed for people. Read the llms.txt proposal.
As an Amazon Associate I earn from qualifying purchases.
The proposal describes a resource map, not a permission system. It does not replace robots.txt, which addresses crawler preferences, and it does not replace the pages that contain the authoritative information. An agent following the proposal’s intended workflow would read or search the index, follow relevant links, and fetch page details as needed.
Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Clear out junk files and repair common Windows errorsFree Scan →Do AI answer engines read llms.txt?
Google Search and AI features
Google Search Central says Google Search does not use llms.txt or other special AI text or Markdown files for Search, including generative AI features. Google’s guidance is explicit: “Google Search itself doesn’t use them.” It also says keeping an llms.txt file neither helps nor harms visibility or rankings in Google Search. This is Google’s documented policy; it does not establish how other providers behave. Google Search Central’s guidance on llms.txt
#1 Best Overall
Google’s June 2026 Search Central update log clarified that the file is not needed for Google Search and is acceptable to maintain for other services or systems that use it. Google Search Central update log
Other providers
The proposal describes how compatible agents are expected to use the file, but that intended workflow is not proof of adoption by every answer engine. A provider publishing an llms.txt file for its own documentation also does not, by itself, show that its search or answer product fetches these files from arbitrary websites. Do not assume that ChatGPT, Claude, Gemini, Perplexity, or another service reads yours unless that provider documents the behavior for the product in question.
What goes in an llms.txt file?
The proposal specifies a simple order. Only the H1 naming the site or project is required; the other sections are optional. A basic example looks like this:
Free tools Windows power users keep installed
One-click scans. No signup required.
# Acme Documentation
> Documentation for Acme’s API and developer tools.
Start with the setup guide if you are integrating the API.
## Getting started
- [Quickstart](https://example.com/docs/quickstart): Set up a first integration.
- [Authentication](https://example.com/docs/auth): Configure API credentials.
## API reference
- [Endpoints](https://example.com/docs/api): Browse available API operations.The example uses placeholder site names and URLs; replace them with your own actual site details. The format’s components are:
Rank #3
- Keep track of everything from attendance to test scores
- Spiral bound
- Measures 8-1/2" x 11"
- H1: The project or site name. This is the only required section.
- Optional blockquote: A brief description of what the site covers or whom it serves.
- Optional prose: Short context that helps an agent interpret the links. The proposal advises against extra headings outside the H2 sections.
- Optional H2 sections: Categories such as “Getting started,” “API reference,” or “Policies,” each followed by a list of links with concise descriptions.
Select useful entry points rather than trying to reproduce a full sitemap. Prefer canonical URLs. If you provide clean Markdown versions of pages, the proposal suggests making them available by appending or substituting .md in the URL and linking to them where appropriate. Keep descriptions factual, concise, and clear enough to make the destination’s purpose apparent.
How do you publish and advertise the file?
- Choose its scope. Put the file at the site root for broad coverage, or at a path such as
/docs/llms.txtfor a particular section. Under the proposal, a file applies to URLs beneath its path; the most specific applicable file should be used. - Build a curated index. Name the site in an H1, then add optional context and grouped links to relevant, maintained pages.
- Publish it at the chosen path. Make sure the file is publicly accessible and the listed pages work for the people and systems expected to use them.
- Add a discovery link. The proposal recommends
rel="describedby"to point to the applicablellms.txtfile. It recommendsrel="alternate"withtype="text/markdown"to identify a Markdown counterpart for an HTML page. These relationships can be expressed in HTML link elements or HTTPLinkresponse headers. - Keep the index in sync. Review it when pages move, change, or become obsolete so the summaries and destinations remain useful.
The proposal’s suggestions for trying questions against an agent with the file as its starting point are authoring guidance, not evidence that a particular agent will use the file or produce better answers.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Which implementation approach fits your site?
The practical choice is the one that fits your publishing workflow and keeps the index accurate. The proposal names several possible routes, but its examples do not establish current pricing or feature parity.
Recommended Free Tools
| Site setup | Possible approach | What to check |
|---|---|---|
| WordPress | The proposal names Yoast SEO and AIOSEO as plugins with llms.txt generation functionality. | Confirm current support, how generated links are selected, and whether the output stays aligned with your content. |
| Hosted documentation platform | The proposal names Mintlify for llms.txt and Markdown page versions, and GitBook for serving llms.txt on published documentation sites. | Check support for the paths you need, Markdown counterparts, discovery links, and ongoing updates. |
| Custom site or build system | Generate the file from the content source or build process; the proposal also lists code and documentation integrations. | Account for selection, link validation, scoped files, and maintenance whenever the site structure changes. |
These are implementation paths named by the proposal, not endorsements or guarantees of current capabilities. The proposal’s integrations page is the reference for its examples.
Quick Recap
Common mistakes to avoid
- Expecting a ranking boost: Google says the file has no special role in Google Search and does not improve or harm Search visibility.
- Treating it as a crawl-control file: A resource map and crawler preferences serve different purposes.
- Listing every URL: A curated set of useful destinations is more consistent with the proposal’s aim than an unfiltered sitemap.
- Assuming universal adoption: A format proposal or a provider’s own documentation file does not prove that its answer engine reads files from other sites.
- Letting the index go stale: Inaccessible, irrelevant, or outdated links undercut the file’s intended usefulness.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
