Skip to content

Commit a8c0309

Browse files
authored
chore(ai): default release-content backfill to OpenRouter DeepSeek (#2217)
Match production summarize routing for scripts/generate-release-content.ts. Anthropic Batches stays explicit via --anthropic-batch; Haiku realtime via --anthropic. Update the generating-release-content skill, and add .gitattributes to pin text files to LF.
1 parent 5de98f9 commit a8c0309

3 files changed

Lines changed: 243 additions & 75 deletions

File tree

.claude/skills/generating-release-content/SKILL.md

Lines changed: 27 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -3,10 +3,12 @@ name: generating-release-content
33
description: >
44
Generate the AI fields on releases — `title_generated`, `title_short`,
55
`summary`, and `composition` — for one release, a window, or a backfill,
6-
via Claude Code sub-agents and direct SDK calls. Use when iterating on
7-
prompts, running provider/model experiments, or filling in a
8-
small-to-medium backfill outside the production ingest path or the
9-
Batches API. (Managed agents can spawn sub-agents now too; the dispatch
6+
via Claude Code sub-agents and direct SDK calls (OpenRouter/DeepSeek to
7+
match production summarize). Use when iterating on prompts, running
8+
provider/model experiments, or filling in a small-to-medium backfill
9+
outside the production ingest path. Large backfills: prefer
10+
scripts/generate-release-content.ts (OpenRouter by default; Anthropic
11+
Batches via --anthropic-batch). (Managed agents can spawn sub-agents now too; the dispatch
1012
scaffolding for this skill just isn't wired up on that side yet.)
1113
---
1214

@@ -25,15 +27,15 @@ The fields are the same whether you're generating for the first time or rewritin
2527

2628
## Prompt is canonical in code, not in this skill
2729

28-
The system prompt and all parsing rules live in `packages/ai/src/release-content.ts`. Read it directly; do **not** paraphrase it back into the user message. The same module is consumed by the ingest worker (`workers/api/src/workflows/poll-and-fetch.ts`), the Batches script (`scripts/generate-release-content.ts`), and this skill, so any drift between local-agent output and ingest-time output starts there.
30+
The system prompt and all parsing rules live in `packages/ai/src/release-content.ts`. Read it directly; do **not** paraphrase it back into the user message. The same module is consumed by the ingest worker (`workers/api/src/workflows/poll-and-fetch.ts`), the backfill script (`scripts/generate-release-content.ts`, OpenRouter by default), and this skill, so any drift between local-agent output and ingest-time output starts there.
2931

3032
Re-export from that file:
3133

3234
- `SYSTEM_PROMPT` — pass verbatim as the system message, with `cache_control: { type: "ephemeral" }`
3335
- `buildReleaseBlock(input)` — build the user message body from a `SummarizeReleaseInput` (org slug, source name, product name, title, version, url, content)
3436
- `isEmptyContent(body)` — short-circuit boilerplate-only bodies; skip the model and write NULLs
3537
- `parseReleaseContent(text)` — pull `<title>`, `<title_short>`, `<summary>`, `<composition>` (plus `<breaking>`/`<migration>`, which this skill's write paths drop — see above) out of a response
36-
- `MODEL` (`claude-haiku-4-5`), `MAX_OUTPUT_TOKENS` (420 — 280 pre-#1696 cap + buffer for the breaking/migration tags), `MAX_BODY_CHARS` (8000)
38+
- `MODEL` (`claude-haiku-4-5` — Anthropic fallback / `--anthropic*` paths only), `MAX_OUTPUT_TOKENS` (420 — 280 pre-#1696 cap + buffer for the breaking/migration tags), `MAX_BODY_CHARS` (8000). Live summarize + the backfill script default to OpenRouter `~deepseek/deepseek-v4-flash-latest` (`SUMMARIZE_MODEL` / `RELEASE_CONTENT_MODEL`).
3739

3840
For experiments that change the prompt, edit `SYSTEM_PROMPT` in place on a branch and run this skill against a small org — that's exactly what the upstream module exists for.
3941

@@ -46,7 +48,7 @@ For experiments that change the prompt, edit `SYSTEM_PROMPT` in place on a branc
4648

4749
When **not** to use this:
4850

49-
- **Large backfills (>200 rows).** Prefer `bun scripts/generate-release-content.ts --orgs=… --since=… --apply`. The Batches API gives a 50% discount and runs offline; local sub-agents pay full price and consume your session's token budget.
51+
- **Large backfills (>200 rows).** Prefer `bun scripts/generate-release-content.ts --orgs=… --since=… --apply` (OpenRouter DeepSeek realtime by default, same family as prod summarize). For a deliberate Anthropic Batches run (50% Haiku discount, up to ~24h), add `--anthropic-batch`. Local sub-agents pay full session token budget.
5052
- **Live ingest gaps.** The poll-fetch workflow already generates content at ingest time when the org is opted in, and a manual `source fetch` that inserts rows follows up with a fill pass over the source's remaining nulls (up to 100). If new rows are landing with nulls, the bug is in the worker, not in this skill.
5153

5254
## Step 1: Pick candidates
@@ -88,20 +90,20 @@ For experiments, write the candidate set to disk so the same input is reused acr
8890

8991
Three modes, pick by candidate count:
9092

91-
| Count | Mode | Model | Why |
92-
| ------ | ---------------------------------- | ---------- | -------------------------------------------------------------------------------------------------------- |
93-
| 1–10 | **Inline** (parent calls SDK) | Haiku 4.5 | Mirrors production ingest exactly (`summarizeRelease()`). Best for parity experiments. |
94-
| 10–200 | **Parallel sub-agents** | **Sonnet** | The agentic loop benefits from stronger reasoning around the long instruction-tuned prompt. |
95-
| 200+ | **Delegate to the Batches script** | Haiku 4.5 | Same model as production; 50% Batches API discount; ~24h latency. `scripts/generate-release-content.ts`. |
93+
| Count | Mode | Model | Why |
94+
| ------ | ----------------------------------- | ------------------------- | ---------------------------------------------------------------------------------------------------------------------------- |
95+
| 1–10 | **Inline** (parent calls SDK) | OpenRouter DeepSeek Flash | Mirrors production ingest (`summarizeRelease()` via OpenRouter when `openrouter-enabled`). Best for parity. |
96+
| 10–200 | **Parallel sub-agents** | **Sonnet** | The agentic loop benefits from stronger reasoning around the long instruction-tuned prompt. |
97+
| 200+ | **Delegate to the backfill script** | OpenRouter DeepSeek Flash | Same family as production summarize. Default: `scripts/generate-release-content.ts --apply`. Add `--anthropic-batch` for Haiku Batches (-50%, ~24h). |
9698

97-
**Why Sonnet for sub-agents but Haiku for the other two modes:** the production ingest path is a single-shot Anthropic API call (`summarizeRelease()` in `packages/ai/src/release-content.ts`) — Haiku 4.5 is great at that. A Claude Code sub-agent is _not_ a single-shot call; it's an agent loop that loads the SYSTEM_PROMPT from a file, narrates, and applies the rules across multiple turns. In a 4 vs 5 row Sonnet/Haiku A/B run on production candidates, Sonnet sub-agents were 4/4 clean; Haiku sub-agents were 3/5 — the two failures were a forbidden `"Neon blog:"` title prefix and a wrongly-null composition. The model difference vanishes once you drop the agent loop and call the SDK directly (Mode A / Mode C), which is why those modes use Haiku.
99+
**Why Sonnet for sub-agents but DeepSeek/Haiku for the other two modes:** the production ingest path is a single-shot `summarizeRelease()` call (OpenRouter DeepSeek Flash when `openrouter-enabled` is on; Anthropic Haiku as fail-open).
98100

99101
### Mode A: Inline (1 release at a time)
100102

101103
The parent — you — is the inference. No `Agent` dispatch. For each candidate row:
102104

103105
1. Build the input with `buildReleaseBlock(row)` from `@releases/ai-internal/release-content`.
104-
2. Call `summarizeRelease(client, input)` from the same module — this _is_ the production codepath. Pass an Anthropic client constructed against `ANTHROPIC_API_KEY` (the parent has env access; sub-agents do not).
106+
2. Call `summarizeRelease(model, input)` from the same module — this _is_ the production codepath. Prefer an OpenRouter TextModel via `buildLaneOpenRouterModel` + `OPENROUTER_API_KEY` (same as the backfill script default / prod summarize). Anthropic Haiku is the fail-open / `--anthropic` path.
105107
3. Write via the CLI (see Step 3).
106108

107109
This is the parity-experiment mode. Output matches what the ingest worker would write byte-for-byte, because it's running the same module with the same model.
@@ -203,15 +205,22 @@ After each agent returns, lint the payload before writing:
203205
- Convert any empty-string field to null before writing.
204206
- Re-prompt or fall back to inline mode for rejects.
205207

206-
### Mode C: Batches script (200+ releases)
208+
### Mode C: Backfill script (200+ releases)
207209

208210
Don't reinvent. Run:
209211

210212
```bash
213+
# Default: OpenRouter DeepSeek Flash realtime (matches prod summarize)
211214
bun scripts/generate-release-content.ts --orgs=<slugs> --since=<days> --apply
215+
216+
# Explicit Anthropic Message Batches (Haiku, 50% discount, up to ~24h)
217+
bun scripts/generate-release-content.ts --orgs=<slugs> --since=<days> --apply --anthropic-batch
218+
219+
# Explicit Anthropic Haiku realtime
220+
bun scripts/generate-release-content.ts --orgs=<slugs> --since=<days> --apply --anthropic
212221
```
213222

214-
The script handles candidate selection, empty-body short-circuit, batch submission, polling, budget guard (`--max-cost`, default $10), and per-row D1 writes. Use the local sub-agent path only when you specifically need it (prompt iteration, model bake-off, sub-200 patch-up).
223+
Requires `OPENROUTER_API_KEY` on the default path (override model with `SUMMARIZE_MODEL` or `RELEASE_CONTENT_MODEL`). Anthropic paths need `ANTHROPIC_API_KEY`. The script handles candidate selection, empty-body short-circuit, budget guard (`--max-cost`, default $10), and per-row D1 writes. `--no-batch` is a deprecated no-op (realtime is already the default). Use the local sub-agent path only when you specifically need it (prompt iteration, model bake-off, sub-200 patch-up).
215224

216225
## Step 3: Write the rows
217226

@@ -253,11 +262,11 @@ Real incidents have come from sub-agents quietly working around upstream errors:
253262

254263
- **Sub-agent's CLI write denied** (`Permission denied` on `releases admin release update`) → the agent should NOT retry with raw `curl` or invent a different write path. Return the structured payload to the parent; the parent writes.
255264
- **`wrangler d1 execute` errors** → STOP and surface. Do not generate from a partial candidate set silently.
256-
- **Provider API thoughts** inside a sub-agent ("let me just call Anthropic directly with `ANTHROPIC_API_KEY`") no. The skill assumes the parent's inference is the inference. Reading `.env` is forbidden across this corpus. The only legitimate provider call is the parent's own Anthropic call (Mode A) or the Batches script (Mode C).
265+
- **Provider API thoughts** inside a sub-agent ("let me just call OpenRouter/Anthropic directly") no. The skill assumes the parent's inference is the inference. Reading `.env` is forbidden across this corpus. The only legitimate provider call is the parent's own SDK call (Mode A) or the backfill script (Mode C).
257266
- **Out-of-skill data sources** ("let me also check the project blog for context") → no. The only input is the release row. If `content` is empty, the answer is NULLs, not enrichment.
258267

259268
## Composing With Other Skills
260269

261270
- **`parsing-changelogs`** is the upstream pipeline. If `content` looks structurally wrong (HTML chrome embedded, anchor IDs not stripped), the issue is in the source's parse step, not in this skill. Fix the source, refetch, then regenerate.
262271
- **`maintaining-orgs`** dispatches per-org regen agents that run `regenerating-overviews`. Release content is a finer grain — it can be invoked inside a `maintaining-orgs` run as a follow-up step on the same org window, or independently. The two skills don't share state.
263-
- **`scripts/generate-release-content.ts`** is the production backfill. This skill is its local-experiment sibling; the prompt and parse contract are shared via `packages/ai/src/release-content.ts`.
272+
- **`scripts/generate-release-content.ts`** is the production backfill (OpenRouter/DeepSeek by default; `--anthropic-batch` for Anthropic Batches). This skill is its local-experiment sibling; the prompt and parse contract are shared via `packages/ai/src/release-content.ts`.

.gitattributes

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
# Normalize line endings to LF in the repo.
2+
# Working trees on Windows still check out LF when core.autocrlf is false/input
3+
# (recommended for this repo).
4+
* text=auto eol=lf
5+
6+
# Explicit source / config
7+
*.ts text eol=lf
8+
*.tsx text eol=lf
9+
*.js text eol=lf
10+
*.jsx text eol=lf
11+
*.mjs text eol=lf
12+
*.cjs text eol=lf
13+
*.json text eol=lf
14+
*.jsonc text eol=lf
15+
*.md text eol=lf
16+
*.yml text eol=lf
17+
*.yaml text eol=lf
18+
*.toml text eol=lf
19+
*.css text eol=lf
20+
*.html text eol=lf
21+
*.svg text eol=lf
22+
*.sql text eol=lf
23+
*.sh text eol=lf
24+
*.env text eol=lf
25+
*.env.* text eol=lf
26+
.gitignore text eol=lf
27+
.gitattributes text eol=lf
28+
.editorconfig text eol=lf
29+
30+
# Windows-only scripts keep CRLF
31+
*.bat text eol=crlf
32+
*.cmd text eol=crlf
33+
*.ps1 text eol=crlf
34+
35+
# Binary
36+
*.png binary
37+
*.jpg binary
38+
*.jpeg binary
39+
*.gif binary
40+
*.webp binary
41+
*.ico binary
42+
*.woff binary
43+
*.woff2 binary
44+
*.ttf binary
45+
*.eot binary
46+
*.pdf binary
47+
*.zip binary
48+
*.gz binary
49+
*.tgz binary
50+
*.7z binary
51+
*.wasm binary

0 commit comments

Comments
 (0)