Skip to content
Blue Nebula

Search

What is llms.txt? What it does, and what it doesn't

llms.txt is a Markdown file that tells AI assistants what your site is and where its key pages are. Here's the format, an example, and who actually reads it.

By Luna R.Published 4 min read

llms.txt is a Markdown file at the root of a website, at /llms.txt, that gives AI assistants a short summary of the site and links to its most useful pages. Jeremy Howard proposed it in September 2024[1]. Google says it doesn't use it, and most files get no AI traffic, so it's a cheap extra rather than a ranking tool.

I publish one on this site and on Allgenda, my own product, and both are generated from the same data as the pages. Here's what goes in the file, the related ideas that grew up around it, and what the evidence says about who reads it.

What goes in an llms.txt file

The proposal defines a fixed order[1]. Only the first part is required:

  1. An H1 with the name of the site or project.
  2. A blockquote with a short summary that explains the rest of the file.
  3. Optional paragraphs with more detail.
  4. Sections under H2 headings, each a list of links. Every link can carry a note after a colon.

A section called "Optional" is, by convention, for secondary pages that an assistant can skip when it needs a shorter read[1]. For a small business, the whole file can be a dozen lines:

md
# Example Bakery

> Example Bakery is a neighborhood bakery in Portland, Oregon, open daily from 7 am, with sourdough, pastries and custom cakes to order.

Custom cakes need three days' notice.

## Menu and ordering

- [Menu](https://example.com/menu): daily breads, pastries and prices
- [Custom cakes](https://example.com/cakes): sizes, flavors and how to order

## Optional

- [Our story](https://example.com/about)

You can see mine at bluenebula.dev/llms.txt, or make one with my free llms.txt generator.

Markdown copies and llms-full.txt

The proposal also suggests offering a clean Markdown version of each page at the same URL with .md added[1]. A model that fetches the Markdown gets the words without the navigation, scripts and layout around them.

llms-full.txt is a separate idea. Mintlify, a documentation platform, started generating it in November 2024 as one file holding the full text of a site's docs[2]. It isn't part of the llms.txt proposal[1]. This site now has both: every case study and post has a .md copy, and /llms-full.txt holds all of them.

Does anything actually read it?

Mostly no, as far as the evidence goes.

Google has been the clearest. In July 2025, Search Engine Land reported that Gary Illyes said Google doesn't support llms.txt and isn't planning to[3]. John Mueller has compared it to the old keywords meta tag, and in May 2026 Search Engine Journal reported that Google's guide to its AI search features lists llms.txt among the things you don't need[4]. I haven't found any statement from OpenAI, Anthropic or Perplexity saying their crawlers use it.

The traffic data points the same way. Ahrefs looked at 137,210 domains in June 2026 and found that 28% publish an llms.txt file, and that 97% of those files received no traffic at all in May 2026[5]. Ahrefs notes its users lean technical, so 28% is likely higher than the web as a whole[5].

There are small signs the other way. One SEO reported in 2025 that his server logs showed OpenAI fetching llms.txt files on a few of his sites[6]. That's one person's logs, and OpenAI hasn't said what it does with them.

Why I still add one

It costs almost nothing when it's built from data the site already has. On this site, a route handler reads the same service list, case studies, posts and FAQ that the pages use, so the file can't fall out of date:

TypeScript
// src/app/llms.txt/route.ts, shortened
export const dynamic = "force-static";

export function GET(): Response {
  const lines = [
    "# Blue Nebula",
    "",
    "> A one-person web development studio in Daly City, CA...",
    "",
    "## Services",
    ...SERVICES.map((s) => `- [${s.name}](${SITE_URL}${s.href}): ${s.summary}`),
  ];
  return new Response(lines.join("\n"), {
    headers: { "content-type": "text/plain; charset=utf-8" },
  });
}

What I wouldn't do is spend time on it before the things AI search demonstrably does use. The first is robots.txt. OpenAI says sites that block its OAI-SearchBot crawler won't be shown in ChatGPT search answers[7], so a rule meant to keep training bots out can take you out of AI search too. My AI crawler robots.txt builder keeps the two apart. After that comes HTML that renders on the server, clear answers near the top of each page, and structured data that matches what's on the page, which I covered in writing for Google's AI Overviews.

How to add one

  1. List the pages someone would need to understand your business: services or products, pricing, docs, contact.
  2. Write a one- or two-sentence summary that says what you do, for whom and where.
  3. Build the file with the generator, or generate it from your CMS or code if your site has one.
  4. Upload it so it opens at yoursite.com/llms.txt, and update it when those pages change.

If you'd rather have it built into your site, along with the Markdown copies, that's the kind of thing I set up in web app development.

Sources

01

Howard, J. (2024, September 3). The /llms.txt file. https://llmstxt.org/

02

Chen, T. (2024, November 20). Simplifying docs for AI with /llms.txt. Mintlify. https://www.mintlify.com/blog/simplifying-docs-with-llms-txt

03

Search Engine Land. (2025, July). Google says normal SEO works for ranking in AI Overviews and llms.txt won't be used. https://searchengineland.com/google-says-normal-seo-works-for-ranking-in-ai-overviews-and-llms-txt-wont-be-used-459422

04

Southern, M. G. (2026, May 20). Google's llms.txt guidance depends on which product you ask. Search Engine Journal. https://www.searchenginejournal.com/googles-llms-txt-guidance-depends-on-which-product-you-ask/575431/

05

Linehan, L., & Guan, X. (2026, June 15). llms.txt study. Ahrefs. https://ahrefs.com/blog/llmstxt-study/

06

Schwartz, B. (2025, July 24). OpenAI crawling llms.txt files. Search Engine Roundtable. https://www.seroundtable.com/openai-crawling-llms-txt-files-39811.html

07

OpenAI. Overview of OpenAI crawlers. Retrieved November 14, 2026, from https://developers.openai.com/api/docs/bots

Written by

Luna R.

Founder and engineer at Blue Nebula, a one-person studio in Daly City, CA, that builds web apps, Shopify stores and AI features. Top Rated Plus on Upwork with 3,700+ hours of client work.

More about me

Have a project in mind?

Tell me what you’re building.

mail@bluenebula.dev
Start a project