Skip to content

Commit c98100a

Browse files
ankur-archclaude
ankur-arch
and
claude
authored
fix(agent-score): directive placement, markdown parity, MCP endpoints (#8099)
* fix(agent-score): directive placement, markdown parity, MCP endpoints Fixes the three regressing checks in the Mintlify/afdocs agent-readiness audit and adds guards so they stay fixed: Content Discoverability (directive buried past 50% of the HTML): - Move the hidden llms.txt directive to the first child of <body> in the docs root layout; the sidebar markup previously pushed it to 30-88% of the body. Now sits at 0-1% on every page. Markdown Content Parity (pages failing at 52-90% missing): - Restore heading markers in getLLMText output from the page toc — fumadocs' processed markdown emits headings as bare "Text [#anchor]" lines, so "## 1. Set up your project" degraded to a stripped list item on the markdown side. - Convert <details>/<summary> to a bold summary + dedented body; the serialized 2-space indent broke the code fences inside. - Wrap the OpenAPI explorer (APIPage) in data-markdown-ignore; the interactive reference has no markdown equivalent and the .md already carries the generated API summary. - Unescape remark escapes (\_, \{, \}) outside code so prose like snake_case matches the rendered HTML. MCP Server Discoverable (no server at expected endpoints): - New /docs/mcp route proxies MCP protocol traffic to mcp.prisma.io/mcp (browser GETs redirect to the /mcp marketing page). - Header-matched beforeFiles rewrites on the site route MCP traffic on www.prisma.io/mcp to the real server; browsers still get the page. Guards: lint-agent-ready now checks directive-first-in-body, heading markers across all pages, <details> leakage, the APIPage parity wrapper, and both MCP endpoints. Skill docs updated with the new invariants and an afdocs reproduction playbook. Verified with the afdocs CLI against a local production build: directive + parity + markdown-url + content-negotiation all pass on the 20 pages that previously failed or warned (avg 1% missing, was avg 8%). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(agent-score): address CodeRabbit review - Hide the llms.txt directive from screen readers and the tab order (aria-hidden + tabIndex={-1}); audits read raw HTML, not the a11y tree. - Bound the /docs/mcp proxy: reject request bodies over 1 MiB (413, also for chunked bodies without Content-Length) and abort the upstream fetch if headers take more than 30s, while keeping SSE bodies streaming. - lint-agent-ready now requires all three header-matched /mcp rewrites (accept, content-type, mcp-session-id), not just one. - Pin the afdocs version in the skill's reproduction commands. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(agent-score): scope MCP rewrite guard to single entries The per-key regex could pair `source: "/mcp"` from one rewrite with a header key from a neighbouring entry (and ast-grep flagged the variable RegExp). Split the config into per-rewrite chunks at each `source:` and require the header key and MCP destination together in the same chunk, using static string matching. Verified the guard now fails when an entry is dropped or a destination is changed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
1 parent 04f5a0f commit c98100a

9 files changed

Lines changed: 479 additions & 54 deletions

File tree

‎.claude/skills/docs-agent-ready/SKILL.md‎

Lines changed: 22 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ name: docs-agent-ready
33
description: Use when adding a new docs section or product area, editing llms.ts / the llms.txt or llms/[...slug] / llms-full.txt routes / get-llm-text / skill.md / .well-known endpoints, or working on the "agent score", "llms.txt", or anything "agent-ready" in the docs and site apps. Explains the invariants the Mintlify agent-readiness audit measures and how to hold them.
44
metadata:
55
author: Prisma
6-
version: "2026.7.21"
6+
version: "2026.7.24"
77
---
88

99
# Docs agent-readiness
@@ -15,8 +15,10 @@ Keep Prisma's docs machine-readable so the Mintlify **agent-score** audit does n
1515
- **Root `llms.txt` < 50k bytes** (warn at 35k). It links to per-area section indexes, not every page.
1616
- **Each section index < 50k bytes** (warn at 40k). Over budget means split the section.
1717
- **Every page is reachable** — each `filterPagesForLLMsIndex` page appears in a section file or the root "Other pages" list. The guard asserts against the generated content, not just membership.
18-
- **Directives in HTML + Markdown** — every page's Markdown (`getLLMText`) starts with the hidden `llms.txt` directive blockquote; the HTML page keeps a hidden directive as its first child.
19-
- **HTML/Markdown parity** via `data-markdown-ignore` on human-only chrome so the Markdown mirrors the page.
18+
- **Directives in HTML + Markdown** — every page's Markdown (`getLLMText`) starts with the hidden `llms.txt` directive blockquote; the HTML keeps a hidden directive as the **first child of `<body>` in the root layout** (`apps/docs/src/app/layout.tsx`), NOT inside the page component. Audits measure the directive's byte position in the body and warn when it sits past 50%, which is where it lands if rendered after the sidebar markup.
19+
- **HTML/Markdown parity** via `data-markdown-ignore` on human-only chrome so the Markdown mirrors the page. The OpenAPI explorer (`APIPage` wrapper in `src/components/api-page.tsx`) carries `data-markdown-ignore` because the interactive reference has no markdown equivalent — the `.md` serves the generated API summary instead.
20+
- **Markdown keeps real headings** — fumadocs' processed output emits headings as bare `Text [#anchor]` lines; `getLLMText` restores `##` markers from the page toc (`restoreHeadingMarkers` in `llm-markdown.ts`). Without them, parity checkers strip list-like heading text ("## 1. Set up …") and agents see prose instead of structure.
21+
- **`<details>` blocks are converted** to a bold summary line + dedented body (`formatDetails` in `llm-markdown.ts`); serialized `<details>` children are 2-space indented, which silently breaks the code fences inside for markdown consumers.
2022
- **`llms-full.txt` excludes** legacy `/orm/v6` and the Accelerate/Optimize products (`getLLMsFullPages`).
2123
- **Skill + MCP endpoints live at BOTH roots**: `www.prisma.io` (apps/site) and `/docs` (apps/docs).
2224

@@ -32,6 +34,8 @@ Keep Prisma's docs machine-readable so the Mintlify **agent-score** audit does n
3234
| `/docs/.well-known/mcp[.json]` | `apps/docs/src/lib/mcp-discovery.ts` |
3335
| `/skill.md`, `/.well-known/agent-skills/*` | `apps/site/src/lib/agent-skills.ts` (`buildSkillMarkdown`) |
3436
| `/.well-known/mcp*` (site) | `apps/site/src/lib/agent-skills.ts` (`buildMcpDiscovery`, server cards) |
37+
| `/docs/mcp` (MCP proxy) | `apps/docs/src/app/mcp/route.ts` → proxies protocol traffic to `mcp.prisma.io/mcp` |
38+
| `/mcp` (site, MCP traffic) | header-matched `beforeFiles` rewrites in `apps/site/next.config.mjs` (browser GETs still get the marketing page) |
3539

3640
The route handlers are thin wrappers: shared builders in `llms.ts` are the single source of truth, so the guard measures exactly what the routes serve.
3741

@@ -41,18 +45,32 @@ The route handlers are thin wrappers: shared builders in `llms.ts` are the singl
4145

4246
**(b) Section over budget.** When a section fails/warns on size, split it into two sections in `llmsSections` (narrower `prefixes`, or carve a sub-tree out with a new slug). Re-run the guard.
4347

44-
**(c) Changing page chrome** in `apps/docs/src/app/(docs)/(default)/[[...slug]]/page.tsx`: keep the hidden `llms.txt` directive as the first child, and put `data-markdown-ignore` on any human-only chrome (banners, nav, badges) so it stays out of the Markdown.
48+
**(c) Changing page chrome.** The hidden `llms.txt` directive lives in `apps/docs/src/app/layout.tsx` as the first child of `<body>` — keep it there (before `<Banner`), never move it into the page component where the sidebar markup would push it past 50% of the HTML. In `[[...slug]]/page.tsx`, put `data-markdown-ignore` on any human-only chrome (banners, nav, badges) so it stays out of the parity comparison. New interactive/human-only MDX components should get `data-markdown-ignore` on their wrapper plus a markdown fallback in `normalizeProcessedMarkdown` (`llm-markdown.ts`), following `APIPage`/`formatApiPage`.
4549

4650
**(d) Changing the CLI workflow or MCP tools** in docs content: update the skill copy in `apps/site/src/lib/agent-skills.ts` AND `apps/docs/src/lib/agent-skill.ts` — they quote real commands and tool names. Keep them in sync with the Prisma Postgres quickstart and `content/docs/ai/tools/mcp-server.mdx`. The `commonQueries` links in `llms.ts` must point to existing pages (the guard fails on stale links).
4751

4852
**(e) Verification.**
4953

5054
```bash
5155
pnpm --filter docs lint:agent-ready # all invariants + size table
56+
pnpm --filter docs test:llm-markdown # markdown pipeline fidelity snapshots
5257
pnpm --filter docs types:check # types
5358
curl -s https://www.prisma.io/docs/llms.txt | head
5459
curl -s https://www.prisma.io/docs/skill.md | head
5560
curl -s https://www.prisma.io/.well-known/mcp
5661
```
5762

5863
The guard prints a size table with per-file headroom so reviewers see how close each file is to its budget.
64+
65+
**(f) Reproducing the audit.** The audit is the `afdocs` npm CLI (https://afdocs.dev). To reproduce a report locally:
66+
67+
```bash
68+
# version pinned against supply-chain surprises — bump deliberately
69+
npx afdocs@0.18.7 check https://www.prisma.io/docs --sampling deterministic -v
70+
# parity needs its upstream checks in the same run:
71+
npx afdocs@0.18.7 check https://www.prisma.io/docs \
72+
--checks markdown-url-support,content-negotiation,markdown-content-parity \
73+
--sampling deterministic --format json -v
74+
```
75+
76+
Audit gotchas encoded in the invariants above: the HTML directive check needs an `<a href*="/llms.txt">` within the first 10% of the (nav/script/style-stripped) `<body>` on sampled pages and warns when every match sits past 50%; the parity check compares HTML text segments against the `.md`, strips `data-markdown-ignore` elements from the HTML side, and only treats a fenced code block as protected when the fence starts at column 0 — which is why `<details>` bodies must be dedented and headings must keep their `#` markers. The separate "MCP Server Discoverable" check probes `<origin>/mcp` with an MCP initialize request (discovery documents alone do not count), which is what the `/docs/mcp` proxy route and the site `/mcp` rewrites are for.

‎apps/docs/scripts/lint-agent-ready.ts‎

Lines changed: 153 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -146,6 +146,8 @@ if (unmatched.length > CATCHALL_WARN) {
146146
// fast enough to cover the full set.
147147
const directiveFailures: string[] = [];
148148
const missingDescription: string[] = [];
149+
const headingMarkerFailures: string[] = [];
150+
const detailsLeaks: string[] = [];
149151
for (const page of indexPages) {
150152
let text: string;
151153
try {
@@ -168,6 +170,27 @@ for (const page of indexPages) {
168170
if (description && !text.includes(description)) {
169171
missingDescription.push(page.url);
170172
}
173+
174+
// Heading markers: the processed markdown emits headings as bare
175+
// "Text [#anchor]" lines; getLLMText restores the `#` markers from the toc.
176+
// If a toc anchor appears in the output, the line carrying it must be a real
177+
// markdown heading — otherwise agents see prose and the afdocs parity check
178+
// strips list-like heading text ("## 1. Set up …") on the markdown side.
179+
for (const item of page.data.toc ?? []) {
180+
if (typeof item.url !== "string" || !item.url.startsWith("#")) continue;
181+
const anchorRef = `[${item.url}]`;
182+
const anchorLine = lines.find((line) => line.includes(anchorRef));
183+
if (anchorLine !== undefined && !/^#{1,6} /.test(anchorLine)) {
184+
headingMarkerFailures.push(`${page.url} (${item.url})`);
185+
}
186+
}
187+
188+
// <details> blocks must be converted to plain markdown (formatDetails in
189+
// llm-markdown.ts); a leaked <details> means its body is still 2-space
190+
// indented, which breaks code fences for markdown consumers.
191+
if (text.includes("<details")) {
192+
detailsLeaks.push(page.url);
193+
}
171194
}
172195

173196
if (directiveFailures.length > 0) {
@@ -192,39 +215,88 @@ if (missingDescription.length > 0) {
192215
pass("Description in markdown", "all frontmatter descriptions present in markdown");
193216
}
194217

218+
if (headingMarkerFailures.length > 0) {
219+
fail(
220+
"Heading markers restored",
221+
`${headingMarkerFailures.length} toc heading(s) rendered without markdown markers (restoreHeadingMarkers in llm-markdown.ts regressed):\n ${headingMarkerFailures
222+
.slice(0, 10)
223+
.join("\n ")}`,
224+
);
225+
} else {
226+
pass("Heading markers restored", "all toc anchors in markdown output sit on real headings");
227+
}
228+
229+
if (detailsLeaks.length > 0) {
230+
fail(
231+
"No <details> leakage",
232+
`${detailsLeaks.length} page(s) leak raw <details> into markdown (formatDetails in llm-markdown.ts regressed):\n ${detailsLeaks
233+
.slice(0, 10)
234+
.join("\n ")}`,
235+
);
236+
} else {
237+
pass("No <details> leakage", "all <details> blocks converted to plain markdown");
238+
}
239+
195240
// ── Check 5b: HTML surface source guard ──────────────────────────────────────
196-
// The rendered HTML page carries the same directive via a hidden element.
197-
// Rendering React in this script is not worth it; instead guard at the source
198-
// level that the docs page component emits a hidden element referencing llms.txt
199-
// BEFORE it renders <DocsPage. This is a source-level guard, not a render test.
200-
const docsPagePath = join(
201-
scriptDir,
202-
"..",
203-
"src",
204-
"app",
205-
"(docs)",
206-
"(default)",
207-
"[[...slug]]",
208-
"page.tsx",
209-
);
241+
// The rendered HTML page carries the same directive via a hidden element that
242+
// must be the FIRST child of <body> in the ROOT LAYOUT — not inside the page
243+
// component. Agent-readiness audits (afdocs "llms-txt-directive-html") measure
244+
// the directive's byte position within the body and warn when it sits past 50%,
245+
// which is where it lands if it renders inside the content area after the
246+
// sidebar markup. Rendering React in this script is not worth it; instead guard
247+
// at the source level that layout.tsx links llms.txt between <body> and the
248+
// first real child (<Banner). This is a source-level guard, not a render test.
249+
const docsLayoutPath = join(scriptDir, "..", "src", "app", "layout.tsx");
210250
try {
211-
const docsPageSource = readFileSync(docsPagePath, "utf8");
212-
const docsPageRenderIndex = docsPageSource.indexOf("<DocsPage");
213-
const llmsRefIndex = docsPageSource.indexOf("llms.txt");
214-
if (docsPageRenderIndex === -1) {
215-
fail("HTML directive source guard", `<DocsPage not found in ${docsPagePath}`);
251+
const layoutSource = readFileSync(docsLayoutPath, "utf8");
252+
const bodyIndex = layoutSource.indexOf("<body");
253+
const llmsRefIndex = layoutSource.indexOf('href="https://www.prisma.io/docs/llms.txt"');
254+
const bannerIndex = layoutSource.indexOf("<Banner");
255+
const ignoreIndex = layoutSource.indexOf("data-markdown-ignore");
256+
if (bodyIndex === -1) {
257+
fail("HTML directive source guard", `<body not found in ${docsLayoutPath}`);
216258
} else if (llmsRefIndex === -1) {
217-
fail("HTML directive source guard", `page.tsx does not reference llms.txt`);
218-
} else if (llmsRefIndex >= docsPageRenderIndex) {
259+
fail("HTML directive source guard", "layout.tsx does not link llms.txt");
260+
} else if (llmsRefIndex < bodyIndex) {
261+
fail(
262+
"HTML directive source guard",
263+
"layout.tsx links llms.txt before <body>; the hidden directive must be the first child of <body>",
264+
);
265+
} else if (bannerIndex !== -1 && llmsRefIndex > bannerIndex) {
266+
fail(
267+
"HTML directive source guard",
268+
"layout.tsx links llms.txt after <Banner; the hidden directive must be the first child of <body> so audits find it near the top of the HTML",
269+
);
270+
} else if (ignoreIndex === -1 || ignoreIndex > llmsRefIndex) {
219271
fail(
220272
"HTML directive source guard",
221-
"page.tsx references llms.txt only after <DocsPage; the hidden directive must precede it",
273+
"the hidden directive in layout.tsx must carry data-markdown-ignore so it stays out of the HTML/markdown parity comparison",
222274
);
223275
} else {
224-
pass("HTML directive source guard", "hidden llms.txt directive precedes <DocsPage");
276+
pass("HTML directive source guard", "hidden llms.txt directive is the first child of <body>");
225277
}
226278
} catch (error) {
227-
fail("HTML directive source guard", `could not read ${docsPagePath}: ${String(error)}`);
279+
fail("HTML directive source guard", `could not read ${docsLayoutPath}: ${String(error)}`);
280+
}
281+
282+
// ── Check 5c: APIPage parity guard ───────────────────────────────────────────
283+
// The interactive OpenAPI explorer on /management-api/endpoints/* has no
284+
// markdown equivalent (per-language code samples, auth widgets); the wrapper in
285+
// api-page.tsx must carry data-markdown-ignore so parity checkers compare only
286+
// the generated markdown API reference.
287+
const apiPagePath = join(scriptDir, "..", "src", "components", "api-page.tsx");
288+
try {
289+
const apiPageSource = readFileSync(apiPagePath, "utf8");
290+
if (!apiPageSource.includes("data-markdown-ignore")) {
291+
fail(
292+
"APIPage parity guard",
293+
"api-page.tsx no longer wraps the OpenAPI explorer in data-markdown-ignore; management-api endpoint pages will fail markdown/HTML parity",
294+
);
295+
} else {
296+
pass("APIPage parity guard", "OpenAPI explorer is excluded from parity comparison");
297+
}
298+
} catch (error) {
299+
fail("APIPage parity guard", `could not read ${apiPagePath}: ${String(error)}`);
228300
}
229301

230302
// ── Check 6: common queries resolve to existing pages ────────────────────────
@@ -369,6 +441,63 @@ if (mcpErrors.length > 0) {
369441
);
370442
}
371443

444+
// ── Check 9b: MCP protocol endpoints (docs + site) ───────────────────────────
445+
// Discovery documents alone do not satisfy "MCP server discoverable" audits —
446+
// they probe the conventional `<origin>/mcp` endpoints with an MCP initialize
447+
// request. /docs/mcp is a proxy route in this app; www.prisma.io/mcp is a
448+
// marketing page, so MCP traffic there is routed by header-matched rewrites in
449+
// the site next.config. Guard both at the source level.
450+
const mcpEndpointErrors: string[] = [];
451+
const docsMcpRoutePath = join(scriptDir, "..", "src", "app", "mcp", "route.ts");
452+
try {
453+
const routeSource = readFileSync(docsMcpRoutePath, "utf8");
454+
if (!routeSource.includes(`"${MCP_URL}"`)) {
455+
mcpEndpointErrors.push(`docs: src/app/mcp/route.ts does not proxy to "${MCP_URL}"`);
456+
}
457+
for (const handler of ["GET", "POST", "DELETE"]) {
458+
if (!new RegExp(`export (async )?function ${handler}\\b`).test(routeSource)) {
459+
mcpEndpointErrors.push(`docs: src/app/mcp/route.ts is missing the ${handler} handler`);
460+
}
461+
}
462+
} catch (error) {
463+
mcpEndpointErrors.push(`docs: could not read ${docsMcpRoutePath}: ${String(error)}`);
464+
}
465+
466+
const siteNextConfigPath = join(scriptDir, "..", "..", "site", "next.config.mjs");
467+
try {
468+
const siteConfigSource = readFileSync(siteNextConfigPath, "utf8");
469+
// All three header conditions are needed to cover the MCP Streamable HTTP
470+
// transport: POST messages (content-type), the server event stream (accept),
471+
// and session teardown (mcp-session-id). Split the config into per-rewrite
472+
// chunks (each starts at its `source:` and ends before the next one) so a
473+
// header key and the MCP destination must appear together in the SAME
474+
// rewrite entry — matching across neighbouring entries would let a dropped
475+
// header condition slip through.
476+
const mcpRewriteChunks = siteConfigSource
477+
.split(/(?=source: ")/)
478+
.filter((chunk) => chunk.startsWith('source: "/mcp"'));
479+
const requiredMcpHeaderKeys = ["accept", "content-type", "mcp-session-id"];
480+
const missingMcpHeaderRewrites = requiredMcpHeaderKeys.filter(
481+
(key) =>
482+
!mcpRewriteChunks.some(
483+
(chunk) => chunk.includes(`key: "${key}"`) && chunk.includes(`destination: "${MCP_URL}"`),
484+
),
485+
);
486+
if (missingMcpHeaderRewrites.length > 0) {
487+
mcpEndpointErrors.push(
488+
`site: next.config.mjs is missing a /mcp rewrite to "${MCP_URL}" for header key(s): ${missingMcpHeaderRewrites.join(", ")}`,
489+
);
490+
}
491+
} catch (error) {
492+
mcpEndpointErrors.push(`site: could not read ${siteNextConfigPath}: ${String(error)}`);
493+
}
494+
495+
if (mcpEndpointErrors.length > 0) {
496+
fail("MCP protocol endpoints", `\n ${mcpEndpointErrors.join("\n ")}`);
497+
} else {
498+
pass("MCP protocol endpoints", "/docs/mcp proxy route + site /mcp rewrites present");
499+
}
500+
372501
// ── Check 10: placeholder protectors are collision-safe ──────────────────────
373502
// Regression coverage for the protect/restore pipeline shared by llm-markdown.ts
374503
// and get-llm-text.ts. Adversarial inputs embed text that LOOKS like the internal

‎apps/docs/src/app/(docs)/(default)/[[...slug]]/page.tsx‎

Lines changed: 3 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -31,29 +31,11 @@ export default async function Page({ params }: { params: Promise<PageParams> })
3131
const aiPromptSlug = (page.data as { aiPrompt?: string }).aiPrompt;
3232
const promptContent = aiPromptSlug ? await getPromptContent(aiPromptSlug) : null;
3333

34-
const pageMarkdownUrl = `https://www.prisma.io${withDocsBasePath(page.url)}.md`;
35-
3634
return (
3735
<>
38-
<div
39-
data-markdown-ignore
40-
style={{
41-
position: "absolute",
42-
width: 1,
43-
height: 1,
44-
padding: 0,
45-
margin: -1,
46-
overflow: "hidden",
47-
clip: "rect(0, 0, 0, 0)",
48-
whiteSpace: "nowrap",
49-
border: 0,
50-
}}
51-
>
52-
For the complete Prisma documentation index optimized for AI agents, see{" "}
53-
<a href="https://www.prisma.io/docs/llms.txt">https://www.prisma.io/docs/llms.txt</a>. A
54-
markdown version of this page is available at{" "}
55-
<a href={pageMarkdownUrl}>{pageMarkdownUrl}</a> (append <code>.md</code> to any docs URL).
56-
</div>
36+
{/* The hidden llms.txt directive for AI agents lives in the root layout
37+
(src/app/layout.tsx) as the first child of <body> — agent-readiness
38+
audits require it near the top of the HTML, before the sidebar. */}
5739
<TechArticleSchema page={page} />
5840
<BreadcrumbSchema page={page} />
5941
<DocsPage

0 commit comments

Comments
 (0)