fix(api): forward the outbound-ping bindings the workflow env was dropping (#2200)

Follow-up to #2199. Two bugs, one root cause.

1. IndexNow has been a no-op on the workflow ingest path since it shipped.

`buildFetchOneEnv` is an exhaustive projection of the workflow env down to
`FetchOneEnv`, and it never forwarded `INDEXNOW_KEY`. Every workflow-driven
fetch therefore logged `indexnow / skipped / no_key_binding` and no search
engine was ever pinged. Confirmed in prod logs.

Every forwarded field is optional, so the omission type-checked. The
`GuardedFetchOneEnv` compile guard exists precisely to stop this -- it was
added after the same class of drop hit the Anthropic key, then again after
it hit the OpenRouter lane vars -- but its key list was named
`AiCriticalFetchKeys` and scoped to AI bindings, so a non-AI credential
fell straight through.

Renamed to `CriticalFetchKeys` and widened: the failure mode is not
"an AI pass is disabled", it is "an optional binding is dropped from an
exhaustive projection, the omission type-checks, and the feature degrades
to its disabled state in silence". Any binding whose absence is
indistinguishable from being deliberately switched off now belongs there.
`INDEXNOW_KEY`, `INDEXING_DISABLED`, `WEB_SERVICE_KEY` and `WEB_BASE_URL`
are now forwarded and guarded.

2. The #2199 revalidate ping missed the dominant ingest path.

`cron/poll-fetch.ts` runs its own post-insert effects block and never
calls `runBatchIngestEffects`, where the ping was wired -- so on-demand ISR
revalidation only fired for the manual fetch route and the scrape
persister, not for the workflow that drives most ingest. Wired into the
inline block beside the IndexNow call. It would also have skipped at
`no_secret_binding` regardless, for the reason above.

Both pings fail open to "skipped", which is why neither showed up as an
error -- the skip is indistinguishable from the feature being correctly
disabled. The extended `*-resolve-env` regression tests now pin all four.

Author: zachdunn

docs/architecture/web.md

@@ -55,7 +55,11 @@ Dynamic routes carry `revalidate = 86400` so first-render cost amortizes across
 
 ## ISR revalidation
 
-Org, source, product and release pages are statically rendered. **Freshness comes from an ingest-time ping, not from the clock.** When `runBatchIngestEffects` inserts releases it calls `notifyWebRevalidate` (`workers/api/src/lib/web-revalidate.ts`) — a sibling of the IndexNow ping, same trigger and same fire-and-forget semantics — which POSTs the affected `{ orgSlug, sourceSlug?, productSlug? }` to web's `POST /api/revalidate`. The route's whole contract (bearer auth, body validation, path derivation) lives in `web/src/lib/revalidate-request.ts` so it is testable without `next/cache`; the route module only injects `revalidatePath` and the secret.
+Org, source, product and release pages are statically rendered. **Freshness comes from an ingest-time ping, not from the clock.** `notifyWebRevalidate` (`workers/api/src/lib/web-revalidate.ts`) — a sibling of the IndexNow ping, same trigger and same fire-and-forget semantics — POSTs the affected `{ orgSlug, sourceSlug?, productSlug? }` to web's `POST /api/revalidate`.
+
+It is wired at **both** post-insert effect sites, which are separate code paths: `runBatchIngestEffects` (batch route + scrape persister) and the inline effects block in `cron/poll-fetch.ts` (the workflow path, which does not call `runBatchIngestEffects`). Wiring only the first leaves the ping firing for manual fetches alone — the workflow is the dominant ingest driver.
+
+The credentials reach `fetchOne` through `buildFetchOneEnv` (`workers/api/src/workflows/_fetch-env.ts`), an exhaustive projection guarded by `CriticalFetchKeys`. **Any binding whose absence is indistinguishable from the feature being switched off must be listed there** — `INDEXNOW_KEY` was not, and the IndexNow ping was a silent no-op on the workflow path from the day it shipped. The route's whole contract (bearer auth, body validation, path derivation) lives in `web/src/lib/revalidate-request.ts` so it is testable without `next/cache`; the route module only injects `revalidatePath` and the secret.
 
 `export const revalidate = 86400` on those pages is the **backstop for a dropped ping**, not the mechanism. Time-based ISR fits this content badly: the long tail of org pages is low-traffic enough that a short window regenerates pages nothing changed on (writes ≈ requests), while a long one would leave fresh releases invisible. Windows and the shared default live in `web/src/lib/isr.ts`.
 

workers/api/src/cron/poll-fetch.ts

@@ -86,6 +86,7 @@ import { invalidateLatestCache } from "../lib/latest-cache.js";
 import type { InvalidationEnv } from "../lib/latest-cache.js";
 import type { InsertedReleaseRow } from "../events/build-event.js";
 import { notifyIndexNowForSource, type IndexNowEnv } from "../lib/indexnow.js";
+import { notifyWebRevalidate, type WebRevalidateEnv } from "../lib/web-revalidate.js";
 import { clusterAndPersistCascades } from "../lib/cluster-cascades.js";
 import { resolveOrgSlug, resolveProductSlug } from "../lib/slug-lookups.js";
 import { logEvent } from "@releases/lib/log-event";
@@ -705,7 +706,8 @@ export const DEFAULT_FETCH_MAX_ENTRIES = 200;
 // source lock), STATUS_HUB, DETERMINISTIC_UPDATE_WORKFLOW, and the MA_* cap
 // vars. Summary-only crawl-enabled feeds delegate through it — see
 // {@link shouldDelegateToCrawl} / {@link delegateScrapeToUpdateWorkflow}.
-export interface FetchOneEnv extends IndexNowEnv, TextModelEnv, UpdateDispatchEnv {
+export interface FetchOneEnv
+  extends IndexNowEnv, WebRevalidateEnv, TextModelEnv, UpdateDispatchEnv {
   GITHUB_TOKEN?: string;
   /**
    * Optional Vectorize bindings for semantic-search side effects. Typed as
@@ -1600,18 +1602,38 @@ export async function ingestRawReleases(
   // INDEXNOW_KEY binding is absent (dev). Per-release URLs are intentionally
   // out of scope — see https://github.com/buildinternet/releases/issues/649.
   if (visiblePublishRows.length > 0) {
+    const slugResolvers = {
+      resolveOrgSlug: (id: string) => resolveOrgSlug(db, id),
+      resolveProductSlug: (id: string) => resolveProductSlug(db, id),
+    };
+
     await notifyIndexNowForSource(
       env,
+      slugResolvers,
       {
-        resolveOrgSlug: (id) => resolveOrgSlug(db, id),
-        resolveProductSlug: (id) => resolveProductSlug(db, id),
+        slug: source.slug,
+        orgId: source.orgId,
+        productId: source.productId,
+        isHidden: source.isHidden,
+        discovery: source.discovery,
       },
+      visiblePublishRows.length,
+    );
+
+    // Bust web's ISR entries for the pages this insert just changed. This path
+    // runs its own effects rather than going through `runBatchIngestEffects`,
+    // so the ping has to be wired here too — it is the workflow (and therefore
+    // the dominant) ingest path, and wiring only the batch path left it firing
+    // for manual fetches alone. Same fire-and-forget semantics as IndexNow;
+    // gating differs (see web-revalidate.ts on `discovery === "on_demand"`).
+    await notifyWebRevalidate(
+      env,
+      slugResolvers,
       {
         slug: source.slug,
         orgId: source.orgId,
         productId: source.productId,
         isHidden: source.isHidden,
-        discovery: source.discovery,
       },
       visiblePublishRows.length,
     );

workers/api/src/workflows/_fetch-env.ts

@@ -16,22 +16,32 @@
  * call for three weeks. That fail-open is deliberately SILENT (an empty model var
  * is the per-lane off switch), so the only symptom was an Anthropic bill — until
  * the account hit its spend cap on 2026-07-23 and both lanes started erroring.
- * The `AiCriticalFetchKeys` guard below now covers the OpenRouter set too.
+ * The guard below now covers the OpenRouter set too.
+ *
+ * And a third time: `INDEXNOW_KEY` was never forwarded at all, so the IndexNow
+ * ping logged `skipped / no_key_binding` on every workflow-driven fetch from the
+ * day this path shipped — no search engine was ever notified. The guard was
+ * named `AiCriticalFetchKeys` and scoped to AI bindings only, so a non-AI
+ * credential fell straight through it. It is now `CriticalFetchKeys` and covers
+ * any binding whose absence is indistinguishable from the feature being off.
  */
 import { getSecret } from "@releases/lib/secrets";
 import type { MediaTransformBinding } from "../lib/media-ingest.js";
 import type { FetchOneEnv } from "../cron/poll-fetch.js";
 import type { AnthropicEnv } from "../lib/anthropic.js";
 import type { TextModelEnv } from "../lib/text-model.js";
 import type { InvalidationEnv } from "../lib/latest-cache.js";
+import type { IndexNowEnv } from "../lib/indexnow.js";
+import type { WebRevalidateEnv } from "../lib/web-revalidate.js";
 
 /**
  * The workflow-env fields forwarded into a `FetchOneEnv`. `FLAGS` rides on
  * `InvalidationEnv`; the Anthropic key + gateway opts ride on `AnthropicEnv`.
  * Every field is optional, so any concrete workflow env (PollAndFetchWorkflowEnv,
  * OnboardSourceWorkflowEnv, …) is structurally assignable.
  */
-export interface WorkflowFetchEnv extends InvalidationEnv, AnthropicEnv, TextModelEnv {
+export interface WorkflowFetchEnv
+  extends InvalidationEnv, AnthropicEnv, TextModelEnv, IndexNowEnv, WebRevalidateEnv {
   GITHUB_TOKEN?: { get(): Promise<string> };
   RELEASES_INDEX?: unknown;
   CHANGELOG_CHUNKS_INDEX?: unknown;
@@ -64,13 +74,20 @@ export interface WorkflowFetchEnv extends InvalidationEnv, AnthropicEnv, TextMod
 }
 
 /**
- * The forwarded fields whose silent omission disables an ingest-time AI pass:
- * the Anthropic client inputs (enrichment + marketing classifier), the
- * feed-enrich tuning vars, and the Browser-Rendering creds enrichment escalates
- * with. This is the exact set the original drop no-opped, and the set the
- * `*-resolve-env` regression tests pin.
+ * The forwarded fields whose silent omission disables an ingest-time side
+ * effect: the Anthropic client inputs (enrichment + marketing classifier), the
+ * feed-enrich tuning vars, the Browser-Rendering creds enrichment escalates
+ * with, and the outbound-ping credentials (IndexNow, web ISR revalidation).
+ *
+ * Scope note: this list was `AiCriticalFetchKeys` and covered only the AI
+ * bindings, which is how `INDEXNOW_KEY` fell through and left the IndexNow ping
+ * a permanent no-op on the workflow path from the day it shipped. The failure
+ * mode is not specific to AI passes — it is "an optional binding is dropped
+ * from an exhaustive projection, the omission type-checks, and the feature
+ * degrades to its disabled state in silence." Any binding whose absence is
+ * indistinguishable from being deliberately switched off belongs here.
  */
-type AiCriticalFetchKeys =
+type CriticalFetchKeys =
   | "ANTHROPIC_API_KEY"
   | "ANTHROPIC_BASE_URL"
   | "AI_GATEWAY_TOKEN"
@@ -85,27 +102,33 @@ type AiCriticalFetchKeys =
   | "FEED_ENRICH_MAX_PER_FIRE"
   | "FEED_THIN_CHARS"
   | "CLOUDFLARE_ACCOUNT_ID"
-  | "CLOUDFLARE_API_TOKEN";
+  | "CLOUDFLARE_API_TOKEN"
+  // Outbound pings. Both fail open to "skipped", so a drop looks exactly like
+  // the feature being off — see the scope note above.
+  | "INDEXNOW_KEY"
+  | "INDEXING_DISABLED"
+  | "WEB_SERVICE_KEY"
+  | "WEB_BASE_URL";
 
 /**
- * `FetchOneEnv` with the AI-critical keys promoted from optional to required —
+ * `FetchOneEnv` with the critical keys promoted from optional to required —
  * `-?` forces each KEY to appear in the builder's return literal (dropping a
  * line is a compile error), while `| undefined` preserves fail-open: the binding
  * itself may still resolve to undefined at runtime. Note this is deliberately
  * NOT `Required<Pick<…>>`, which would strip `undefined` from the VALUE and
  * reject the genuinely-optional source bindings the builder forwards.
  */
 type GuardedFetchOneEnv = FetchOneEnv & {
-  [K in AiCriticalFetchKeys]-?: FetchOneEnv[K] | undefined;
+  [K in CriticalFetchKeys]-?: FetchOneEnv[K] | undefined;
 };
 
 /**
  * Project a workflow env down to the `FetchOneEnv` slice. The only async work is
  * resolving the GitHub token secret; everything else is a binding hand-off. Keep
  * the field list exhaustive — a dropped binding silently no-ops the corresponding
- * ingest-time AI pass (see the module header). The {@link GuardedFetchOneEnv}
- * return type turns dropping one of the AI-critical bindings into a compile
- * error rather than a silent prod regression.
+ * ingest-time pass or outbound ping (see the module header). The
+ * {@link GuardedFetchOneEnv} return type turns dropping one of the critical
+ * bindings into a compile error rather than a silent prod regression.
  */
 export async function buildFetchOneEnv(env: WorkflowFetchEnv): Promise<GuardedFetchOneEnv> {
   const githubToken = (await getSecret(env.GITHUB_TOKEN).catch(() => null)) ?? undefined;
@@ -153,5 +176,12 @@ export async function buildFetchOneEnv(env: WorkflowFetchEnv): Promise<GuardedFe
     FEED_THIN_CHARS: env.FEED_THIN_CHARS,
     CLOUDFLARE_ACCOUNT_ID: env.CLOUDFLARE_ACCOUNT_ID,
     CLOUDFLARE_API_TOKEN: env.CLOUDFLARE_API_TOKEN,
+    // Outbound pings fired from fetchOne's post-insert effects. Missing since
+    // the workflow path shipped, which is why every workflow-driven fetch
+    // logged `indexnow / skipped / no_key_binding`.
+    INDEXNOW_KEY: env.INDEXNOW_KEY,
+    INDEXING_DISABLED: env.INDEXING_DISABLED,
+    WEB_SERVICE_KEY: env.WEB_SERVICE_KEY,
+    WEB_BASE_URL: env.WEB_BASE_URL,
   };
 }

workers/api/src/workflows/poll-and-fetch.ts

@@ -54,6 +54,8 @@ export {
 };
 import { type AnthropicEnv } from "../lib/anthropic.js";
 import { type TextModelEnv } from "../lib/text-model.js";
+import { type IndexNowEnv } from "../lib/indexnow.js";
+import { type WebRevalidateEnv } from "../lib/web-revalidate.js";
 import { makeBotFetch } from "../lib/web-bot-auth-fetch.js";
 
 /**
@@ -67,7 +69,12 @@ export type PollAndFetchWorkflowEnv = InvalidationEnv &
   // `buildFetchOneEnv` can forward it. Omitting it is what silently pinned the
   // marketing-classifier + feed-enrich lanes to Anthropic Haiku — see the
   // history note in `_fetch-env.ts`.
-  TextModelEnv & {
+  TextModelEnv &
+  // Outbound-ping credentials, same reason: `buildFetchOneEnv` can only forward
+  // what this type admits. `INDEXNOW_KEY` was absent here, so the IndexNow ping
+  // skipped with `no_key_binding` on every workflow-driven fetch.
+  IndexNowEnv &
+  WebRevalidateEnv & {
     DB: D1Database;
     CRON_ENABLED?: string;
     GITHUB_TOKEN?: { get(): Promise<string> };

workers/api/test/poll-fetch-resolve-env.test.ts

@@ -35,6 +35,10 @@ function buildEnv(): PollAndFetchWorkflowEnv {
     FEED_ENRICH_MODEL: "deepseek/deepseek-v4-flash",
     SUMMARIZE_MODEL: "deepseek/deepseek-v4-flash",
     EXTRACT_MODEL: "deepseek/deepseek-v4-pro",
+    INDEXNOW_KEY: secret("indexnow-key"),
+    INDEXING_DISABLED: "false",
+    WEB_SERVICE_KEY: secret("web-service-key"),
+    WEB_BASE_URL: "https://releases.sh",
   } as unknown as PollAndFetchWorkflowEnv;
 }
 
@@ -82,4 +86,30 @@ describe("resolveFetchEnv (poll-and-fetch workflow)", () => {
     expect(fetchEnv.SUMMARIZE_MODEL).toBe("deepseek/deepseek-v4-flash");
     expect(fetchEnv.EXTRACT_MODEL).toBe("deepseek/deepseek-v4-pro");
   });
+
+  // The same drop, third occurrence — this time on the non-AI side effects, which
+  // the AI-only guard never covered. `INDEXNOW_KEY` was missing since the workflow
+  // path was introduced, so every workflow-driven fetch logged
+  // `indexnow / skipped / no_key_binding` and no search engine was ever pinged.
+  // Nothing errors; the skip is indistinguishable from "correctly disabled".
+  it("forwards the IndexNow key + kill switch so the ping isn't a permanent no-op", async () => {
+    const env = buildEnv();
+    const fetchEnv = await resolveFetchEnv(env);
+    expect(fetchEnv.INDEXNOW_KEY).toBe(env.INDEXNOW_KEY);
+    expect(fetchEnv.INDEXING_DISABLED).toBe("false");
+  });
+
+  it("forwards the web service key so on-demand ISR revalidation fires on ingest", async () => {
+    const env = buildEnv();
+    const fetchEnv = await resolveFetchEnv(env);
+    expect(fetchEnv.WEB_SERVICE_KEY).toBe(env.WEB_SERVICE_KEY);
+  });
+
+  // Shared by both pings for building target URLs. Both fall back to
+  // `https://releases.sh`, so a drop is invisible in prod and wrong everywhere
+  // else — staging/dev would silently aim at the production host.
+  it("forwards WEB_BASE_URL so the pings don't fall back to the prod host", async () => {
+    const fetchEnv = await resolveFetchEnv(buildEnv());
+    expect(fetchEnv.WEB_BASE_URL).toBe("https://releases.sh");
+  });
 });