Skip to content

Commit 6355b1f

Browse files
committed
blog: how I raised the docs agent readiness score from 85 to 99
Adds a post walking through the agent-readiness work merged in #8087: the llms.txt split, per-page directives, Markdown parity, the llms-full.txt link fixes, skill.md, and MCP discovery, plus a process readers can follow on their own docs. Includes the two checks that still fail (Markdown Content Parity and MCP Server Discoverable) rather than presenting the work as complete.
1 parent b6806c8 commit 6355b1f

3 files changed

Lines changed: 248 additions & 0 deletions

File tree

  • apps/blog
Lines changed: 162 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,162 @@
1+
---
2+
title: "How I Raised the Prisma Docs Agent Readiness Score from 85 to 99"
3+
slug: "docs-agent-readiness-score-85-to-99"
4+
date: "2026-07-21"
5+
authors:
6+
- "Ankur Datta"
7+
metaTitle: "Agent Readiness Score: Prisma Docs from 85 to 99"
8+
metaDescription: "How I fixed a truncated llms.txt, HTML and Markdown drift, a broken content feed, and missing skill.md and MCP discovery in the Prisma docs, and how to audit yours."
9+
heroImagePath: "/docs-agent-readiness-score-85-to-99/imgs/hero.svg"
10+
metaImagePath: "/docs-agent-readiness-score-85-to-99/imgs/meta.png"
11+
heroImageAlt: "An agent readiness score moving from 85 to 99"
12+
tags:
13+
- "ai"
14+
- "education"
15+
---
16+
17+
A coding agent fetches a Prisma docs page mid-task, on behalf of someone who never sees it. If the page it gets back is truncated, or its links resolve to the wrong path, nothing errors. The agent answers from what it got.
18+
19+
I ran the [Prisma docs](https://www.prisma.io/docs) through [Mintlify's Agent Readiness Score](https://www.mintlify.com/score/prisma), an audit of 30 checks that measures whether a coding agent can discover, fetch, and parse a documentation site. The report came back at **85 out of 100**, a B. Today it shows **99 out of 100**, an A. Here is what was failing, what I changed, and what is still not fixed.
20+
21+
## Why documentation has to work for agents
22+
23+
A human reader can recover from a bad docs site. They scroll past the cookie banner, notice the version switcher, and open a second tab when a link breaks.
24+
25+
An agent does none of that. It makes one HTTP request, takes what comes back, and moves on. If the index file is cut off partway through, the agent loses everything after the cut and cannot tell. If a link resolves to the wrong path, it follows the link and reads the wrong page. If the Markdown version of a page is missing a paragraph the HTML has, it answers from the shorter version.
26+
27+
None of those failures produce an error. They produce a confident, wrong answer in someone's editor, attributed to your docs.
28+
29+
## What the score found at 85
30+
31+
Failures landed in five areas: content discoverability, Markdown parity, full content discoverability, agent skills, and MCP server discovery.
32+
33+
The largest problem was our own `llms.txt`. The convention is that this file is the entry point: an agent fetches it to learn what the site contains. Ours listed all 631 pages inline, which came to about 116,000 characters. Mintlify fails an `llms.txt` over 50,000 characters, so we were more than double the limit, and the tail of the file was past the point where common agent clients stop reading a text feed.
34+
35+
A page fetched on its own gave the agent no pointer back to the index.
36+
37+
The Markdown build dropped the page description that the HTML rendered, so the `.md` an agent read was missing a summary the human page showed.
38+
39+
The full-content feed, `llms-full.txt`, was 7 MB and contained roughly 2,500 root-relative links. Those lost their `/docs` prefix when a client resolved them against the feed's own URL, so following one landed on a page that did not exist.
40+
41+
Two checks scored zero because the documents did not exist at all: no agent skill definition, and no discovery document for the MCP server that already runs at `mcp.prisma.io/mcp`.
42+
43+
## The changes I made
44+
45+
Each fix targets one way an agent fails. The first column is the failure, the second is what shipped, the third is what it buys an agent.
46+
47+
| Problem | Change | Why it helps agents |
48+
| --- | --- | --- |
49+
| `llms.txt` was 116k characters and got truncated | Moved the page list into per-area indexes at `/docs/llms/<area>.txt` and left the root as a table of contents | The whole index survives the fetch, and an agent can pull only the area it needs |
50+
| A page fetched on its own had no route back to the index | Added an `llms.txt` directive to every page, as a hidden element in the HTML and a blockquote in the Markdown | An agent that lands on one page can find the rest of the docs |
51+
| The Markdown build dropped the page description | Included the description in the Markdown output, and tagged HTML-only chrome with `data-markdown-ignore` | The `.md` an agent reads carries the same summary the HTML page shows |
52+
| `llms-full.txt` had about 2,500 links that resolved to the wrong path | Rewrote in-body links to absolute URLs | Links an agent follows from the feed land on the page they name |
53+
| `llms-full.txt` was 7 MB | Dropped legacy ORM v6 and deprecated product pages, taking the feed to 4.5 MB and absorbing the bytes the absolute links added | A feed an agent can hold without truncating it |
54+
| No agent skill existed | Published skill documents at `/skill.md` and `/docs/skill.md`, with a listing at `/.well-known/agent-skills` | An agent can load the Prisma setup workflow without inferring it from prose |
55+
| Our MCP server was not discoverable | Served discovery documents at `/.well-known/mcp` and `/.well-known/mcp.json` | Nothing yet, as far as I can measure. See the honest accounting below |
56+
57+
Dropping v6 removes content rather than fixing it. Those 191 pages are still fetchable individually by appending `.md` to their URL. They are out of the concatenated feed, not out of the docs.
58+
59+
One clarification on `data-markdown-ignore`, because the name oversells it. Nothing in our pipeline reads that attribute. The `.md` output is built from MDX source, not scraped from rendered HTML, so the copy buttons and the "Edit on GitHub" footer were never in the Markdown to begin with. The attribute is the remedy Mintlify's own parity check suggests: it marks those elements in the HTML so the checker can tell they are chrome rather than content the Markdown is missing.
60+
61+
The root index is now a table of contents. Fetching [www.prisma.io/docs/llms.txt](https://www.prisma.io/docs/llms.txt) gives common queries, then one line per area. Three of the twelve lines:
62+
63+
```markdown
64+
## Product Area Indexes
65+
66+
- [`Prisma ORM`](https://www.prisma.io/docs/llms/orm.txt): Current Prisma ORM docs: setup, schema modeling, Prisma Client, migrations, and references (excludes legacy v6 and Prisma Next).
67+
- [`Prisma ORM v6 (legacy)`](https://www.prisma.io/docs/llms/orm-v6.txt): Legacy Prisma ORM v6 documentation, maintained for backwards compatibility only. Prefer the current Prisma ORM section for new work.
68+
# … Prisma Next, then …
69+
- [`Prisma Postgres`](https://www.prisma.io/docs/llms/postgres.txt): Prisma Postgres setup, connection strings, local development, operations, and guides.
70+
```
71+
72+
Each area index holds its own page list and stays under a 50 KB budget that CI enforces. The largest, `orm-v6.txt`, sits at 37 KB.
73+
74+
The page directive is the other change small enough to copy. It sits at the top of every Markdown page, right after the H1. Fetch [www.prisma.io/docs/postgres.md](https://www.prisma.io/docs/postgres.md) and the first lines are:
75+
76+
```markdown
77+
# Prisma Postgres (/docs/postgres)
78+
79+
> For the complete Prisma documentation index, see [llms.txt](https://www.prisma.io/docs/llms.txt). A markdown version of any docs page is available by appending `.md` to its URL.
80+
```
81+
82+
MCP discovery is a small JSON document at a well-known path, served by a route handler. `curl https://www.prisma.io/.well-known/mcp` returns:
83+
84+
```json
85+
{
86+
"version": "1.0.0",
87+
"transport": "http",
88+
"url": "https://mcp.prisma.io/mcp",
89+
"servers": [
90+
{
91+
"name": "prisma",
92+
"url": "https://mcp.prisma.io/mcp",
93+
"transport": "http",
94+
"authentication": "oauth"
95+
}
96+
]
97+
}
98+
```
99+
100+
The server behind that document is the same [Prisma MCP server](/search-prisma-docs-from-your-coding-agent) we already ship. Nothing about it changed.
101+
102+
## Where the score landed, including what did not work
103+
104+
The report now shows **99 out of 100**, a grade A. Of the 30 checks, 24 pass, 2 fail, 3 are skipped as not applicable, and 1 warns. The components are weighted, which is why two failures cost a single point rather than a grade.
105+
106+
The report is published as Markdown at [mintlify.com/score/prisma.md](https://www.mintlify.com/score/prisma.md), so the per-check detail below is quotable rather than something you have to take from me.
107+
108+
Both remaining failures are worth naming, because one of them is a fix in this post that did not take.
109+
110+
**Markdown Content Parity still fails.** This is the check my description fix was aimed at. The report reads: "1 of 15 pages have substantive content differences between markdown and HTML (avg 8% missing)". One sampled page in fifteen, and it is enough to fail the check. I have not yet found which page or what it is missing.
111+
112+
**MCP Server Discoverable still fails**, with "No MCP server was discovered at the expected endpoints." Both discovery documents return the JSON above at both paths. Whatever endpoints the checker probes, these are not them. I published those documents because an audit asked for them, and I have no evidence yet that anything reads that path. That row in the table is a bet, not a result.
113+
114+
There is also a warning I did not clear. The `llms.txt` directive is found in the HTML of all 15 sampled pages, but the checker wants it near the top and ours sits past the halfway mark. The Markdown half passes cleanly. Content discoverability is 6 of 7.
115+
116+
The measurements behind the changes are live. As of July 2026:
117+
118+
```bash
119+
curl -s https://www.prisma.io/docs/llms.txt | wc -c # 7031
120+
curl -s https://www.prisma.io/docs/llms-full.txt | wc -c # 4517273
121+
curl -s https://www.prisma.io/docs/skill.md | sed -n 2p # name: prisma
122+
```
123+
124+
I also added a CI check, [`lint-agent-ready.ts`](https://github.com/prisma/web/blob/main/apps/docs/scripts/lint-agent-ready.ts), that runs on every pull request into `main` as `pnpm --filter docs lint:agent-ready`. It runs ten assertions: per-file size budgets, every page reachable from the index, the directive present on every page, common-query links resolving, excluded pages staying out of the full feed, and the skill and MCP documents keeping their required fields.
125+
126+
That guard matters more than the score. The audit told me where the docs stood in July; it does not watch the repository. Without something enforcing the invariants on every pull request, the next few hundred pages of content would have undone the work quietly.
127+
128+
## How to do this for your own docs
129+
130+
Start by running your docs through the [Agent Readiness Score](https://www.mintlify.com/score). It crawls the site and reports per-check results, so you get concrete failures instead of a general sense that things could be better.
131+
132+
Read the failed checks before you change anything, and group them by cause rather than by check. Our five failures came down to one file being too large, two documents not existing, link resolution being wrong, and the Markdown build dropping a field. That is fewer changes than a 30-item list suggests.
133+
134+
Then work through the four areas:
135+
136+
- **Discoverability.** Keep `llms.txt` small enough to survive a fetch. If yours lists every page inline, move the page lists into per-area indexes and leave the root as a table of contents pointing at them.
137+
- **Content parity.** Fetch a few of your `.md` pages and diff them against the rendered HTML. Anything present in one and missing from the other is a divergence an agent inherits. The usual culprit is a frontmatter field the Markdown build never carries through.
138+
- **Metadata.** Publish the documents agents look for: a skill definition at `/skill.md`, and an MCP discovery document at `/.well-known/mcp` if you run a server. A skill needs `name`, `description`, `license`, `compatibility`, `metadata`, and `allowed-tools` in its frontmatter. Ours is readable at [prisma.io/docs/skill.md](https://www.prisma.io/docs/skill.md) if you want a worked example.
139+
- **Access.** Follow the links in your own feeds and confirm they resolve. Root-relative links that work inside a page break when a client resolves them against a feed URL at a different depth, which is exactly how our 2,500 links went wrong.
140+
141+
Rerun the score when you are done and confirm the checks you targeted actually flipped. Two of mine did not, which I would not have known without rerunning.
142+
143+
Then put the invariants into CI. Size budgets, reachability, and the directive are cheap to assert and catch the regressions that content growth actually causes.
144+
145+
The endpoints are all public if you want to see the finished shape: [llms.txt](https://www.prisma.io/docs/llms.txt), [skill.md](https://www.prisma.io/docs/skill.md), and [the MCP discovery document](https://www.prisma.io/.well-known/mcp). Point your agent at them and see what it does.
146+
147+
## Frequently asked questions
148+
149+
<Accordions type="single">
150+
<Accordion title="What is an agent readiness score?">
151+
An agent readiness score measures how well a documentation site serves AI coding agents rather than human readers. Mintlify's version runs 30 weighted checks across categories including content discoverability, Markdown availability, page size and truncation risk, content structure, URL stability, agent skills, and MCP server discovery, then reports a score out of 100 with a letter grade.
152+
</Accordion>
153+
<Accordion title="How large can an llms.txt file be before agents truncate it?">
154+
Mintlify's Agent Readiness Score fails an `llms.txt` over 50,000 characters. Prisma's reached about 116,000 characters in mid-2026, more than double that limit. Prisma now holds the root index to a 50 KB budget enforced in CI, and the file serves at roughly 7,000 characters.
155+
</Accordion>
156+
<Accordion title="What is the difference between llms.txt and llms-full.txt?">
157+
`llms.txt` is an index: a short file listing what the documentation contains, with links. `llms-full.txt` is the content itself, the whole documentation concatenated into one machine-readable file. An agent fetches the index to navigate and the full feed when it wants everything at once. As of July 2026 the Prisma index is about 7 KB and the full feed is about 4.5 MB.
158+
</Accordion>
159+
<Accordion title="Where do agents look for an MCP server?">
160+
Audits and clients that discover MCP servers by convention request `/.well-known/mcp` or `/.well-known/mcp.json` on the site's root domain. Prisma serves a JSON document at both paths naming `https://mcp.prisma.io/mcp`, its HTTP transport, and OAuth authentication. Publishing those documents did not flip Mintlify's MCP server check, which still scored as failing in July 2026, so treat the path as a convention worth satisfying rather than a solved problem.
161+
</Accordion>
162+
</Accordions>

apps/blog/public/docs-agent-readiness-score-85-to-99/imgs/hero.svg

Lines changed: 86 additions & 0 deletions
Loading
277 KB
Loading

0 commit comments

Comments
 (0)