---
title: I Checked 319 Indie Blogs for llms.txt. 12 Had One.
canonical: "https://www.rankinghacks.com/llms-txt-in-the-wild/"
pubDate: "2026-07-26T09:00:00.000Z"
updatedDate: "2026-07-26T00:00:00.000Z"
author: Andreas De Rosi
description: "I probed 319 indie blogs for llms.txt. Only 12 had a real one, a 3.9% adoption rate. Full dataset, method, and what it means for AI-search visibility."
categories: [ai-search]
---

The SEO timeline has spent a year arguing about `llms.txt`: is it the new `robots.txt`, a ranking signal for AI answer engines, snake oil, or a nice-to-have? What nobody in that argument seemed to have is a number. So I went and got one.

I took a neutral sample of 319 independent blogs and probed every single one for a `/llms.txt` file. Not a list of sites that already have llms.txt, which would be a rigged 100%, but a general sample of the indie web. Then I opened each hit by hand, because roughly a fifth of the "hits" were fake. The headline: **12 of the 307 reachable blogs had a real llms.txt file. That is 3.9%.** Here is the whole study, the method so you can reproduce it, and the raw data to download.

## How I sampled (the part that makes the number honest)

The trap with any adoption study is the sampling frame. Pull your sample from an llms.txt directory and you "discover" that 90% of sites have one. Pull it from the wrong place and you learn nothing. I needed a frame that was independent of llms.txt entirely.

I used [Kagi Small Web](https://github.com/kagisearch/smallweb), an open list of roughly 35,000 independent personal-blog feeds. It exists to surface human-written indie writing, and it has no relationship to llms.txt at all, which is exactly why it makes a clean denominator. From that list I deduped to hosts (collapsing www and non-www), took a **deterministic even-stride sample of 320 domains** across the alphabetized set so the run is reproducible, and probed `https://<host>/llms.txt` with an http fallback. For any site that had a file, I also checked for the companion `/llms-full.txt`.

One of those 320 turned out not to be an independent blog at all: it was a hosting provider's auto-generated staging subdomain that ships a boilerplate llms.txt by default. I disqualified it, which is exactly the kind of manual check this study is built on, leaving **319 genuine independent blogs**. Everything below is measured against that 319.

The one methodological decision that changed the answer: **I classified every response by its content, not its status code.**

## The soft-404 trap: why status codes lie here

If I had trusted HTTP status codes, this study would have reported roughly 11% adoption, nearly three times too high. That is because a lot of blogs, especially ones on hosted platforms and SPA frameworks, return `200 OK` for *any* path you request and serve their HTML catch-all page. Ask for `/llms.txt` and you get back the homepage with a `200`, not a 404. A status-only crawler counts that as a hit.

Twenty-one of the 319 sites did exactly this. I caught them by requiring the response to actually look like an llms.txt file: non-HTML, Markdown, with an H1 and at least one section or link. Everything else got reclassified as a soft-404.

<figure>
  <img src="/images/posts/llms-txt-in-the-wild/01-adoption-breakdown.png" alt="Horizontal bar chart classifying 319 probed indie blogs: 12 have a real llms.txt file, 21 return an HTML soft-404, 258 return a genuine 404, 16 were blocked or errored, and 12 were unreachable" loading="lazy" />
  <figcaption>The full classification of all 319 blogs. Only <strong>12</strong> real files. The <strong>21</strong> HTML soft-404s are the important detail: they return <code>200 OK</code> with no actual file, so a crawler that trusts status codes would report roughly <strong>33 hits (11%)</strong> instead of the real <strong>12 (3.9%)</strong>. If you run your own adoption study, classify by content or your number is inflated almost 3x.</figcaption>
</figure>

That soft-404 finding is worth as much as the headline rate. Any "X% of sites now have llms.txt" claim you see that was measured with a status-code check is overstating adoption by a wide margin. The web lies about 404s, and llms.txt studies inherit the lie.

## 3.9% is the real base rate

Strip the fakes and the picture is stark. Of 307 reachable blogs, 258 returned a genuine 404, 21 faked it, 16 were blocked or errored, and **12 had a real file**. Adoption among the independent web, as of late July 2026, is about one in twenty-five.

That is not a criticism of llms.txt. It is the useful fact. If you are weighing whether to add one, the competitive reality is that almost none of your peers have. The format is early. Early is where cheap differentiation lives, which I will come back to.

## The 12 that exist agree on nothing

Here is where it gets interesting. You might expect the sliver of sites that adopted a new standard to be the diligent ones who followed the spec. They are not. The 12 files share a filename and almost nothing else.

<figure>
  <img src="/images/posts/llms-txt-in-the-wild/02-file-size-distribution.png" alt="Horizontal bar chart of the 12 llms.txt files by size on a log scale, ranging from 892 bytes for pklavc.com up to 181.8 KB for insufficientscotty.com" loading="lazy" />
  <figcaption>The 12 real files by size, log scale. The range is three orders of magnitude: from a <strong>892-byte</strong> hand-written note (pklavc.com) to a <strong>181.8 KB, 943-link</strong> auto-generated dump (insufficientscotty.com). There is no house style. Median adopter ships about 4.2 KB.</figcaption>
</figure>

The size range spans from under a kilobyte to nearly 200 KB. On one end, hand-curated files that read like a thoughtful table of contents. On the other, machine-generated exports that dump every URL on the site into a single wall of links, which defeats the point of a curated map. The median adopter sits at a sensible 4.2 KB, 5 sections, 18 links, but the median hides how bimodal this is.

<figure>
  <img src="/images/posts/llms-txt-in-the-wild/03-sections-vs-links.png" alt="Scatter plot of the 12 llms.txt files, x-axis number of sections from 2 to 8, y-axis number of links on a symlog scale from 0 to over 900, showing no correlation and huge spread" loading="lazy" />
  <figcaption>Sections vs links for each adopter. No agreed shape at all: two blogs (neilkakkar, pklavc) list <strong>zero</strong> links and use the file as a pure summary; two others dump <strong>300 to 943</strong> links. Section counts cluster loosely at 3 to 7 but the link counts scatter across three orders of magnitude. Nobody has settled what an llms.txt is <em>for</em> yet.</figcaption>
</figure>

That scatter is the real story of the format in 2026. The spec suggests a short, curated set of links with a summary at the top. In practice, adopters are split between people using it as a hand-written elevator pitch and tools using it as a sitemap-with-Markdown. Both call the file `llms.txt`. An AI retriever reading them gets two completely different kinds of document.

## Even the adopters skip half the spec

The llms.txt convention recommends two things beyond the link list: a `>` blockquote summary right after the H1, so a retriever gets a one-paragraph description of the site, and a companion `/llms-full.txt` containing the actual expanded content. Neither is universal even among the 12.

<figure>
  <img src="/images/posts/llms-txt-in-the-wild/04-format-features.png" alt="Bar chart showing that 10 of 12 llms.txt adopters include a summary blockquote and only 4 of 12 also ship a companion llms-full.txt file" loading="lazy" />
  <figcaption>Feature adoption inside the adopter group. <strong>10 of 12</strong> include the recommended summary blockquote, which is the easy, high-value part. Only <strong>4 of 12</strong> ship the companion <code>/llms-full.txt</code>. So "has an llms.txt" and "has a well-formed llms.txt" are not the same population, and the second one is smaller still.</figcaption>
</figure>

The summary blockquote is the single highest-leverage line in the file, one sentence that tells an answer engine what your site is, and two of the twelve skip it. The full-text companion is rarer, which is defensible since it is more work, but it means most of these files point at pages without giving retrievers a clean fetch target. The takeaway is that "well-formed" is a smaller club than "adopted," and being in the smaller club is nearly free.

Who is in the club? Overwhelmingly developers and technologists: API Evangelist, sdan.io, Neil Kakkar, Gaurav Trivedi. The people shipping llms.txt today are the ones who read the spec the week it dropped. It has not reached the general indie publisher yet.

## What a 4% base rate actually means for you

I run this site as a live experiment, and I ship [my own llms.txt](/llms.txt), so I am not neutral about whether this matters. But the data points somewhere specific, and it is not "everyone must ship one immediately or lose."

It points here: **a well-formed llms.txt is a cheap, uncrowded signal, not a decisive one.** Nobody has demonstrated it moves rankings, and RankingHacks has [argued at length that treating it as a ranking factor is snake oil](/llms-txt-does-it-work/). What the 4% base rate tells you is different. It tells you that the discovery surface for AI answer engines is nearly empty on the indie web, that the cost of doing it right is one afternoon, and that "right" mostly means the parts most adopters skip: a real summary blockquote, a curated rather than dumped link list, and a companion full-text file. You are not competing against a crowded field. You are competing against a field that mostly has not shown up, and among those who have, half did it badly.

That is the honest case for llms.txt in 2026. Not a ranking hack. A cheap, low-competition way to be legible to the machines that increasingly sit between your writing and its readers, at a moment when your peers overwhelmingly are not.

## The data, and how to reproduce it

The point of a study is that you can check it. [Download the full dataset (CSV)](/downloads/llms-txt-in-the-wild.csv). It has one row per probed host with the classification, HTTP status, file size, line count, section count, link count, and whether a summary blockquote was present. If you want to run your own version: take a frame that is independent of llms.txt (I used Kagi Small Web), sample deterministically so it reproduces, probe `/llms.txt`, and above all **classify by content, not status code**, or your adoption number will be inflated by soft-404s.

The honest caveats. This is a base-rate snapshot from a 319-site sample taken in late July 2026, not a trend line, and 12 adopters is a small n, so I report shape and range rather than fitting curves through it. Adoption is climbing and this number will look low in a year. The value here is the method and the current reading: content-classified, neutrally sampled, roughly 4% and messy.

If it says anything durable, it is that the gap between "people arguing about llms.txt" and "people who have actually shipped a clean one" is enormous. That gap is the opportunity.

Related reading: [does llms.txt actually work](/llms-txt-does-it-work/), [how I GEO-audited my own site](/geo-audit-own-site/), [what happened when I GEO-optimized RankingHacks](/we-geo-optimized-our-own-site/), and [how I track whether AI engines cite me](/track-ai-citations-chatgpt-perplexity/).