To make a Next.js site easier for AI agents to read, publish a concise /llms.txt index, provide Markdown versions of useful pages, and explicitly link the HTML pages to both resources. You can also return Markdown when a client sends Accept: text/markdown, but that is an optional implementation—not a requirement of the llms.txt proposal—and support across clients is not established.
What each format does
These three mechanisms solve different problems. The llms.txt proposal is a community convention for giving agents a short overview and links to more detailed material. It is not a crawler-permission file: use robots.txt for crawler rules.
As an Amazon Associate I earn from qualifying purchases.
| Mechanism | Role | Discovery | Important limitation |
|---|---|---|---|
/llms.txt |
A curated index for a site or a section of one | Conventional path; the applicable index can also be advertised with rel="describedby" |
A proposal and convention, not a ratified standard or permission control |
| Markdown mirror | Clean, page-level content for reading and processing | Advertise it with rel="alternate" type="text/markdown", or link to it from the index |
Requires a publishing or routing approach and ongoing content alignment |
Accept: text/markdown |
Requests a Markdown representation at the same URL | The client sends the header; your server chooses the response | Not part of the proposal’s requirements; universal client and intermediary support is not established |
The proposal by Jeremy Howard was published September 3, 2024 and modified August 10, 2026. Its current format guidance and v2 change notes describe the index, page mirrors, and discovery links.
How to add an llms.txt index to a Next.js site
Write a brief Markdown file that helps a reader or agent choose the right pages, rather than trying to reproduce the entire site. The proposal requires only an H1 naming the project or site; a summary blockquote, explanatory text, and H2 sections of links are optional.
- Create the index. Put it at
/llms.txtfor the whole site, or under a path when it covers only that section. An index applies to pages beneath its path; where several apply, use the most specific one. - Organize links by reader task. Prioritize the most useful canonical documentation, such as setup, API references, and troubleshooting. Link to Markdown mirrors when available.
- Keep the index concise. Its job is navigation. Let the linked pages carry the detail.
- Decide whether to include an Optional section. It is a convention for secondary links, not a required part of the format. In v2, it no longer has its former mechanical role in context-expansion tooling.
A minimal shape could look like this; replace the example names and paths with real pages from your site:
# Example project
A short description of what the project is and where its documentation is.
## Documentation
- [Getting started](/docs/getting-started.md)
- [API reference](/docs/api.md)
## Guides
- [Deployment](/guides/deployment.md)How to serve Markdown versions of Next.js pages
Choose a consistent URL convention and publish clean Markdown for pages where it materially improves readability. The proposal allows a Markdown suffix such as page.html.md or page.md; for directory-style URLs, it gives index.html.md and index.md patterns. The names are flexible—the important part is making the representation predictable and discoverable.
- Keep each Markdown mirror aligned with its corresponding HTML page, including meaningful headings, links, and current information.
- Use static files or Next.js routes according to how your site publishes content; the format guidance does not prescribe one deployment architecture.
- Return an appropriate content type for the representation. Vercel Academy’s Next.js App Router example serves index and full-text files as
text/plain; charset=utf-8and page mirrors astext/markdown; charset=utf-8. It is an implementation example, not a compatibility guarantee for every Next.js version or hosting environment.
How to make both resources discoverable
Do not make agents guess whether a Markdown URL exists. The proposal recommends advertising the page’s Markdown alternative and the index that describes the page. You can express these relationships in HTML or HTTP Link headers.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Repair Windows errors before they cause bigger problems3Fix the driver behind crashes, sound loss and screen glitches<link rel="alternate" type="text/markdown" href="/docs/getting-started.md">
<link rel="describedby" href="/llms.txt">Use the actual mirror URL and the most specific applicable index URL for each page. If a Markdown page has its own relevant index, point to that index rather than assuming the site-root file is always the right one.
Should you support Accept: text/markdown?
You may implement content negotiation so a request carrying Accept: text/markdown receives Markdown at an HTML page’s URL. Treat it as an additional route to the same content, not a substitute for explicit Markdown URLs and discovery links. The reviewed proposal and Next.js implementation guide describe discoverable URLs and route handlers; they do not establish broad support for this header across agents, browsers, or intermediaries.
If you add negotiation, test representative clients and your full delivery path, including CDN and cache behavior. Responses that vary according to the Accept header must be handled so a cached Markdown response is not mistakenly served to a client expecting HTML, or vice versa.
Rank #4
How to check the result
- Request
/llms.txtand confirm it returns the intended index rather than a 404 or server error. - Open a Markdown URL directly and check that the body is clean Markdown and its content type matches your implementation.
- Inspect a representative HTML page for the
alternateanddescribedbyrelations, whether you publish them in HTML or HTTP headers. - If you support content negotiation, request the same page with and without
Accept: text/markdownand verify both response bodies and cache behavior.
Chrome for Developers describes its Lighthouse llms.txt audit as optional and emerging. Its current documentation says a 404 is marked Not Applicable, while a server error during retrieval is flagged. Passing that retrieval check does not demonstrate that agents discover the content, cite it, or produce a business result.
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →What these additions can and cannot establish
A concise index, clean page mirrors, and explicit links give agents more usable ways to find and read your published information. They do not guarantee that a particular agent will fetch it, cite it, or rank the site differently. The proposal and implementation example describe formats and routes, not independently documented traffic, ranking, citation, or agent-usage outcomes.
Quick Recap
Best Value
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
