docs: replace starlight-llms-txt with custom llms.txt/agents.txt pointing to .github/aw/*.md#38630
Conversation
…s.txt endpoints - Remove starlight-llms-txt package and plugin config - Add src/pages/llms.txt.ts and agents.txt.ts Astro endpoints that list all .github/aw/*.md agent-optimised prompts with raw GitHub URLs - Add src/pages/_aw-prompts.ts shared helper for reading aw prompt files - Update public/.well-known/ai.txt to reference agents.txt Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds custom plain-text endpoints for agent/LLM prompt discovery and removes the starlight-llms-txt integration in favor of generating prompt listings directly from .github/aw/*.md.
Changes:
- Added
llms.txtandagents.txtAstro API routes that render prompt links and descriptions. - Introduced a shared helper to read
.github/awprompt files and extractdescriptionfrom frontmatter. - Updated
.well-known/ai.txtand a docs page label; removed the Starlightllms-txtplugin and dependency.
Show a summary per file
| File | Description |
|---|---|
| docs/src/pages/llms.txt.ts | New endpoint to output an LLM-oriented prompt index. |
| docs/src/pages/agents.txt.ts | New endpoint to output an agent-oriented prompt index. |
| docs/src/pages/_aw-prompts.ts | Shared filesystem helper to list prompts + parse frontmatter descriptions. |
| docs/src/content/docs/agent-factory-status.mdx | Updated workflow display name text. |
| docs/public/.well-known/ai.txt | Points AI crawlers to the new agents index endpoint. |
| docs/package.json | Removes starlight-llms-txt dependency. |
| docs/astro.config.mjs | Removes starlight-llms-txt integration configuration. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Files not reviewed (1)
- docs/package-lock.json: Language not supported
- Files reviewed: 7/8 changed files
- Comments generated: 4
| export const GET: APIRoute = () => { | ||
| const prompts = getAwPrompts(); | ||
|
|
||
| const lines = [ | ||
| '# GitHub Agentic Workflows', | ||
| '', | ||
| '> Agent-optimised prompt files for GitHub Agentic Workflows (gh-aw).', | ||
| '> These markdown files are designed for AI agents and LLMs, not human readers.', | ||
| '', | ||
| '## Agent Prompts', | ||
| '', | ||
| ...prompts.map(({ file, description, rawUrl }) => { | ||
| const label = file.replace(/\.md$/, ''); | ||
| return description | ||
| ? `- [${label}](${rawUrl}): ${description}` | ||
| : `- [${label}](${rawUrl})`; | ||
| }), | ||
| ]; |
| export const GET: APIRoute = () => { | ||
| const prompts = getAwPrompts(); | ||
|
|
||
| const lines = [ | ||
| '# GitHub Agentic Workflows – Agent Prompts', | ||
| '', | ||
| '> Agent-optimised prompt files for GitHub Agentic Workflows (gh-aw).', | ||
| '> Use these files to ground AI agents working with gh-aw workflows.', | ||
| '', | ||
| '## Prompts', | ||
| '', | ||
| ...prompts.map(({ file, description, rawUrl }) => { | ||
| const label = file.replace(/\.md$/, ''); | ||
| return description | ||
| ? `- [${label}](${rawUrl}): ${description}` | ||
| : `- [${label}](${rawUrl})`; | ||
| }), | ||
| ]; |
| export function getAwPrompts(): AwPrompt[] { | ||
| // process.cwd() is the docs/ directory during `astro build` | ||
| const awDir = join(process.cwd(), '../.github/aw'); | ||
| return readdirSync(awDir) | ||
| .filter((f) => f.endsWith('.md')) | ||
| .sort() | ||
| .map((file) => { | ||
| const content = readFileSync(join(awDir, file), 'utf-8'); | ||
| return { | ||
| file, | ||
| description: parseFrontmatterDescription(content), | ||
| rawUrl: `${RAW_BASE}/${file}`, | ||
| }; | ||
| }); | ||
| } |
| const lines = [ | ||
| '# GitHub Agentic Workflows – Agent Prompts', | ||
| '', | ||
| '> Agent-optimised prompt files for GitHub Agentic Workflows (gh-aw).', | ||
| '> Use these files to ground AI agents working with gh-aw workflows.', | ||
| '', | ||
| '## Prompts', | ||
| '', | ||
| ...prompts.map(({ file, description, rawUrl }) => { | ||
| const label = file.replace(/\.md$/, ''); | ||
| return description | ||
| ? `- [${label}](${rawUrl}): ${description}` | ||
| : `- [${label}](${rawUrl})`; | ||
| }), | ||
| ]; |
|
🧠 Matt Pocock Skills Reviewer was skipped during the skills-based review. |
|
✅ PR Code Quality Reviewer completed the code quality review. |
|
🧪 Test Quality Sentinel completed test quality analysis. No test files were added or modified in PR #38630. The PR is a documentation-only change: replacing starlight-llms-txt with custom llms.txt/agents.txt TypeScript pages. Test Quality Sentinel skipped. |
|
✅ Design Decision Gate 🏗️ completed the design decision gate check. No ADR enforcement needed: PR #38630 does not have the 'implementation' label (has_implementation_label=false) and has 0 new lines of code in business logic directories, well below the 100-line threshold (requires_adr_by_default_volume=false). |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
REQUEST_CHANGES — two correctness holes and two latent defects in the new custom llms.txt/agents.txt generator.
🔍 Blocking issues summary
Blocking
-
Shallow directory scan omits existing files (
_aw-prompts.ts:28):readdirSyncis non-recursive..github/aw/runbooks/workflow-health.mdalready exists and is silently excluded from the generated index. Whether subdirectory content should be included or excluded must be an explicit decision, not an accident. -
process.cwd()fragile path resolution (_aw-prompts.ts:27): Breaks silently ifastro buildis called from any directory other thandocs/. The ESM-idiomatic fix usingimport.meta.urlis a one-liner.
Non-blocking but worth fixing before the index is widely consumed
-
RAW_BASEhardcoded tomain(_aw-prompts.ts:8): PR preview builds generate links that 404 until merge.GITHUB_SHAenv var at build time fixes this. -
YAML-quoted
description:values leak surrounding quotes (_aw-prompts.ts:21): Latent defect; safe today but will silently corrupt output for the first file that usesdescription: "...".
🔎 Code quality review by PR Code Quality Reviewer · ⌖ 13.4 AIC
|
|
||
| export function getAwPrompts(): AwPrompt[] { | ||
| // process.cwd() is the docs/ directory during `astro build` | ||
| const awDir = join(process.cwd(), '../.github/aw'); |
There was a problem hiding this comment.
Fragile CWD assumption: path resolution via process.cwd() will silently produce a wrong awDir (and crash the build) if astro build is ever invoked from a directory other than docs/.
💡 Suggested fix
Use import.meta.url to derive the path relative to the source file, which is immune to whatever the working directory happens to be:
import { fileURLToPath } from "node:url";
// ...
const awDir = fileURLToPath(new URL("../../../.github/aw", import.meta.url));_aw-prompts.ts lives at docs/src/pages/, so three ../ steps land at the repo root and .github/aw is correct from there. This approach survives astro build being called from any working directory (e.g., repo root in a monorepo script or IDE task runner).
The comment documenting the assumption is a smell — if the code were robust, the comment would be unnecessary.
| const match = content.match(/^---[\r\n]+([\s\S]*?)[\r\n]+---/); | ||
| if (!match) return ''; | ||
| // Simple key extraction – avoids pulling in a YAML parser at this layer | ||
| const descMatch = match[1].match(/^description:\s*(.+)$/m); |
There was a problem hiding this comment.
YAML-quoted description values will leak surrounding quotes into output: the regex captures the raw YAML value including any wrapping quote characters, so a description like description: "My prompt" renders as "My prompt" (with literal quotes) in the generated text files.
💡 Suggested fix
Strip wrapping single/double quotes after trimming:
const raw = descMatch[1].trim();
return raw.replace(/^(['"])(.*)\1$/, '$2');All current .github/aw/*.md files use unquoted descriptions, so this is a latent defect. The first file that uses description: "My prompt" will silently publish the quotes verbatim in every AI crawler index.
| export function getAwPrompts(): AwPrompt[] { | ||
| // process.cwd() is the docs/ directory during `astro build` | ||
| const awDir = join(process.cwd(), '../.github/aw'); | ||
| return readdirSync(awDir) |
There was a problem hiding this comment.
readdirSync is shallow: existing .md files in subdirectories are silently omitted from the index: .github/aw/runbooks/ currently contains workflow-health.md (an actual operational guide), and .github/aw/imports/ may accumulate .md files over time. None of these will ever appear in llms.txt or agents.txt.
💡 Details
The intent seems to be indexing agent prompt files, and it is genuinely unclear whether runbooks/ and imports/ subdirectory content should be included. The problem is that the code has no explicit exclusion logic — it just accidentally excludes them because readdirSync is non-recursive and the .filter(f => f.endsWith('.md')) test only sees direct children.
Two options:
Option A – explicit shallow-only intention (add a comment):
// Intentionally shallow: only root-level prompt files, not subdirectories (runbooks/, imports/)
const awDir = ...;
return readdirSync(awDir)
...Option B – recursive inclusion:
import { readdirSync, statSync, readFileSync } from 'node:fs';
function findMdFiles(dir: string): string[] {
return readdirSync(dir).flatMap((f) => {
const full = join(dir, f);
return statSync(full).isDirectory()
? findMdFiles(full).map((sub) => join(f, sub))
: f.endsWith('.md') ? [f] : [];
});
}The current code is a silent correctness hole: new files added to subdirectories will never appear in any published index without a code change.
| import { readdirSync, readFileSync } from 'node:fs'; | ||
| import { join } from 'node:path'; | ||
|
|
||
| export const RAW_BASE = |
There was a problem hiding this comment.
RAW_BASE is hardcoded to the main branch: raw URLs will be stale/broken during branch builds and PR previews where new .github/aw/*.md files are added but not yet merged to main.
💡 Details
The constant https://raw.githubusercontent.com/github/gh-aw/main/.github/aw is baked in. A PR that adds a new prompt file will generate a llms.txt entry with a raw.githubusercontent.com/.../main/... URL that 404s until the PR merges. Any agent or tool consuming a preview deploy of the docs will follow a dead link.
Consider injecting the commit SHA at build time:
// Picks up GITHUB_SHA in CI, falls back to 'main' for local builds
const REF = process.env.GITHUB_SHA ?? 'main';
export const RAW_BASE =
`https://raw.githubusercontent.com/github/gh-aw/${REF}/.github/aw`;This ensures the generated index always points to the exact commit that was used to build it.
starlight-llms-txtwas generatingllms.txtfrom human-facing doc pages. The.github/aw/*.mdfiles are the actual agent-optimised prompts and are the right source for LLM/agent consumption.Changes
starlight-llms-txtdep and its plugin block fromastro.config.mjssrc/pages/_aw-prompts.ts— build-time helper that scans.github/aw/*.md, extractsdescriptionfrontmatter, and returnsraw.githubusercontent.comURLssrc/pages/llms.txt.tsandsrc/pages/agents.txt.ts— Astro static endpoints emitting llmstxt.org-format files listing every prompt file with its description.well-known/ai.txt— addAgents-txt:entry, dropLlms-full-txt:(no longer generated)Generated output per entry: