feat: improve AcceptMarkdown support#209
Conversation
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
superwall-docs-staging | 5cff604 | Commit Preview URL Branch Preview URL |
Jun 11 2026, 09:07 PM |
dcrawbuck
changed the title
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
dcrawbuck
force-pushed
the
duncan/sw-5396-docs-full-acceptmarkdowncom-support
branch
from
35ee720 to
3bf524c
Compare
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3bf524c276
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| return buildMarkdownRouteResponse(includeBody ? body : null, slugs); | ||
| } | ||
|
|
||
| export const Route = createFileRoute("/{$}.md")({ |
There was a problem hiding this comment.
When the alternate URL is for a nested doc such as /docs/ios/quickstart/install.md, this route will not handle it: the generated route tree for this file is /{$}.md, i.e. a single path segment with a .md suffix, while TanStack splats are the /$ segment captured in params._splat. Since shouldBypassLLMRewrite now skips all .md paths, those nested alternates fall through to the normal docs splat as ios/quickstart/install.md and 404 instead of returning the page markdown. Please make the .md/.mdx routes catch nested paths before the suffix, or stop bypassing the existing markdown rewrite for direct suffix URLs.
Useful? React with 👍 / 👎.
| rehypeCodeOptions: isDevelopment ? false : undefined, | ||
| remarkPlugins: (existing) => [ | ||
| remarkImagePaths, | ||
| remarkMarkdownImageMap, |
There was a problem hiding this comment.
For pages whose images come from <include>d shared MDX, this plugin runs before remarkInclude, so those included image nodes are not added to markdownImageMap even though they are present in the processed markdown later. For example, content/docs/ios/guides/advanced/observer-mode.mdx includes content/shared/observer-mode.mdx, which contains /images/om_transactions.png; that page's markdown alternate can be left with an unresolved __img... placeholder instead of the restored image URL. Move the map collection after include expansion or otherwise collect images from included content.
Useful? React with 👍 / 👎.
| headers: { | ||
| Vary: "Accept", | ||
| }, |
There was a problem hiding this comment.
When a browser on another origin fetches a canonical docs URL with Accept: text/markdown, this 307 is part of the CORS redirect chain but only includes Vary. The final markdown route sets Access-Control-Allow-Origin: *, yet the browser can block before following it because the redirect response itself lacks that header. Add the same CORS header to the negotiation redirect so cross-origin consumers can use the advertised AcceptMarkdown URL, not just the direct .md URL.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 04d16ec766
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| }; | ||
|
|
||
| const htmlImagePlaceholderPattern = /<img\b([^>]*?)\bsrc=["']__img(\d+)["']([^>]*)\/?>/g; | ||
| const markdownImagePlaceholderPattern = /(!\[[^\]]*]\(\s*)<?__img(\d+)>?(\s*\))/g; |
There was a problem hiding this comment.
When a markdown image includes a title, the processed placeholder can be emitted as , but this pattern only matches placeholders followed immediately by ), so those images are left as __img… in the .md alternate. Several existing docs use titled images, including content/docs/ios/quickstart/install.mdx, so those pages can expose unresolved placeholders instead of usable image URLs.
Useful? React with 👍 / 👎.
dcrawbuck
force-pushed
the
duncan/sw-5396-docs-full-acceptmarkdowncom-support
branch
from
0f9a1a7 to
da40466
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit da40466. Configure here.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: da4046622d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const placeholder = getImagePlaceholder(node); | ||
| const url = getImageUrl(node); | ||
| if (placeholder && url) imageMap[placeholder] = url; |
There was a problem hiding this comment.
For MDX <img> nodes this condition can never add an entry: with a real src, getImagePlaceholder() is undefined, and with a __imgN placeholder, getImageUrl() returns undefined. Pages such as content/docs/agents/create-an-agent.mdx use Frame-wrapped JSX images, so when the processed markdown contains <img src="__imgN">, restoreImagePlaceholders() has no URL for that placeholder and the .md alternate can expose unresolved image placeholders instead of usable image links. Collect the JSX image URLs under the placeholder index order rather than requiring both values on the same node.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5cff604ed6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| const directRewrite = rewriteLLMMarkdown(url.pathname) || rewriteLLMMdx(url.pathname); | ||
| if (directRewrite) return directRewrite; | ||
| if (negotiateDocsRepresentation(request) !== "text/markdown") return undefined; |
There was a problem hiding this comment.
With this negotiation gate, a normal curl -sL or LLM crawler following the bare page URLs emitted in llms.txt no longer gets the markdown redirect, because curl sends Accept: */* and the new negotiator defaults wildcards to HTML. That makes the generated llms summaries' page links return HTML unless callers know to add an Accept: text/markdown header, so either emit .md links in the summaries or update the advertised fetch path/header before relying on Accept negotiation here.
Useful? React with 👍 / 👎.
1 participant
This adds AcceptMarkdown-compliant content negotiation for docs routes, including Fumadocs-based q-value negotiation,
Vary: Accept, alternate/canonicalLinkheaders, and 406 responses for unsupported media types.It adds Markdown alternate URL helpers plus root and nested
/llms.mdx/docsroutes so/docs/,.md,.mdx, and canonical docs URLs all resolve to Markdown where appropriate.It switches processed Markdown generation to Fumadocs placeholder rendering for common MDX components, making Markdown cleaner for agents while keeping HTML pages unchanged.
It adds tests for negotiation and rewrite behavior; local validation covered
bun test,bun run build:cf, curl checks against local dev, and agent-browser checks.Note
Medium Risk
Touches global request middleware and broad docs URL routing; behavior changes for clients using
Acceptheaders or oldllms-*.txtpaths (aliases remain but discovery shifts).Overview
Adds AcceptMarkdown-style docs serving: middleware negotiates
Accept(including q-values and legacytext/plain/text/x-markdown), redirects canonical HTML URLs to.mdwhen Markdown wins, returns 406 for unsupported explicit types on known doc paths, and attachesVary: AcceptplusLink: rel="alternate"for Markdown on HTML responses.Introduces stable
.md/.mdxroutes (including SDK selector paths), sharedmarkdown-route/markdown-alternatehelpers, and HTMLrel="alternate"links on doc pages. Processed Markdown for agents is improved via Fumadocs MDX placeholders (Callouts, Tabs, etc.), amarkdownImageMapremark plugin, and absolute image URLs.LLMS URLs move to section paths like
/docs/ios/llms.txt(with legacy root aliases kept); Agents is added as an LLMS section. vibe-coding docs document.mdsuffixes and the newllms.txtlink table.Reviewed by Cursor Bugbot for commit 5cff604. Bugbot is set up for automated code reviews on this repo. Configure here.