Skip to content

Commit 2f1cf66

Browse files
authored
Merge pull request #290 from Blazity/feat/aiw-239-mcp-block-catalog
feat(worker): expose MCP block catalog and run stats for agent authors (AIW-239)
2 parents 03b3ba2 + 971457a commit 2f1cf66

17 files changed

Lines changed: 824 additions & 4 deletions

SETUP.md

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -621,6 +621,52 @@ The key is **required for the feature but optional at boot**: the worker starts
621621

622622
> **Security warning.** The shipped `webhook-ticket-triage` template feeds external input (a support ticket body) straight through to an automatically opened pull request with **no human gate**. HMAC authenticates the **channel**, not the **content**: anyone who can file a ticket into a connected Zendesk or Sentry controls the agent's prompt, and therefore the PR it opens. Before pointing a real sender at this template, add a human-approval gate before the `open_pr` block, or treat the workflow as triage-and-notify only.
623623
624+
### Remote MCP — connect your agent
625+
626+
A deployment can expose an [MCP](https://modelcontextprotocol.io) endpoint so an external agent (Claude, or any MCP-capable client) can inspect this deployment's workflows, block catalog and run history, and — if you grant it the scope — author and dispatch workflows itself. It is off by default and entirely additive: nothing here is required for the bot to run.
627+
628+
| Variable | Value |
629+
| --- | --- |
630+
| `MCP_ENABLED` | `true` to turn the endpoint on. Defaults to `false`. |
631+
| `MCP_ALLOW_PUBLIC_DCR` | `true` to let a connecting client register itself via OAuth Dynamic Client Registration, with no client pre-provisioned. Defaults to `false`. This is the path a customer engineer wiring up their own agent normally wants. |
632+
633+
```bash
634+
vercel env add MCP_ENABLED production
635+
vercel env add MCP_ALLOW_PUBLIC_DCR production
636+
```
637+
638+
Redeploy after setting either variable.
639+
640+
**Connecting.** Point your agent's MCP client at:
641+
642+
```
643+
https://<your-worker>.vercel.app/mcp
644+
```
645+
646+
using the Streamable HTTP transport at protocol version `2025-11-25` (`2025-06-18` is also accepted). Authentication is standard MCP OAuth: an MCP-compliant client discovers the authorization requirements from the endpoint itself (`WWW-Authenticate`, then `.well-known/oauth-protected-resource` and `.well-known/oauth-authorization-server`), registers itself if `MCP_ALLOW_PUBLIC_DCR` is on, and opens a browser tab for the connecting person to log in and approve a consent screen — nothing is pasted or configured by hand. Approve only the scopes the agent actually needs:
647+
648+
| Scope | Grants |
649+
| --- | --- |
650+
| `mcp:read` | Read tickets, runs, workflows, prompts and the block catalog. Enough to inspect a deployment without changing anything. |
651+
| `runs:dispatch` | Start a manual run, answer a run's clarification, cancel a run. |
652+
| `workflows:write` | Author workflows: create a definition, save a draft graph, publish it live. |
653+
| `prompts:write` | Edit the prompt library. |
654+
| `tickets:write` | Comment on, transition, or create a ticket in the connected tracker. |
655+
656+
**Verifying the connection.** Ask the connected agent to call `system.capabilities` (confirms the handshake and reports the enabled tool domains) and then `blocks.list` (confirms it can read this deployment's block catalog — every block type the editor offers, with its input and output contract, so an authoring agent can compose a valid graph without guessing a field name and finding out from a `VALIDATION_FAILED`).
657+
658+
**A worked example: a loop and a branch together.** [`docs/example-workflows/loop-branch-workflow.json`](./docs/example-workflows/loop-branch-workflow.json) is a complete, valid workflow graph an agent can read for a concrete pattern rather than reasoning from the block catalog alone. Shape:
659+
660+
```
661+
trigger_ticket_ai -> planning_agent -> branch(gate)
662+
gate --true--> send_slack_message -> terminate (needs a human, stop and say so)
663+
gate --false--> implementation_agent -> run_pre_pr_checks -> branch(verdict)
664+
verdict --true--> finalize_workspace -> open_pr (checks passed, ship it)
665+
verdict --false--> loop(retry) --continue--> review_agent(fix) -> back to run_pre_pr_checks
666+
```
667+
668+
`branch` reads a `condition` param and fires its `true` or `false` port; `loop` re-enters its `continue` port up to `maxAttempts` times before taking `onExhaust`. To try it against a real deployment, hand the file to `workflows.create` + `workflows.save_draft` (the same graph the dashboard editor would save), then `workflows.publish` when ready to go live — publishing arms whatever triggers the graph contains, so read each tool's description before calling it against anything but a scratch definition.
669+
624670
---
625671

626672
## 13. Troubleshooting

apps/worker/src/mcp/contracts.test.ts

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,9 @@ describe("MCP public contracts", () => {
5353
"tickets.comment",
5454
"tickets.transition",
5555
"tickets.create",
56+
"blocks.list",
57+
"blocks.get",
58+
"runs.stats",
5659
]);
5760
expect(new Set(FIRST_SLICE_TOOLS).size).toBe(FIRST_SLICE_TOOLS.length);
5861
});

apps/worker/src/mcp/contracts.ts

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,20 @@ export const FIRST_SLICE_TOOLS = [
7272
"tickets.comment",
7373
"tickets.transition",
7474
"tickets.create",
75+
// A new domain, appended rather than interleaved with the read tools above for
76+
// the same reason every earlier addition was: the published order of what
77+
// already shipped stays byte-identical. blocks.list/blocks.get close the gap
78+
// an authoring agent hit first -- workflows.save_draft takes a graph, but
79+
// nothing published what a node's params, inputs or output actually look like,
80+
// so a caller could only learn a block's contract by trial and VALIDATION_FAILED.
81+
"blocks.list",
82+
"blocks.get",
83+
// Named under the "runs" domain it belongs to, even though (per the rule above)
84+
// it registers last rather than beside runs.get/runs.trace/runs.result/
85+
// runs.diagnose: an agent asking "how has this deployment been doing" had
86+
// per-run detail and nothing that rolled runs up, the same gap prompts.list
87+
// once closed for prompts.get.
88+
"runs.stats",
7589
] as const;
7690
export type McpToolName = (typeof FIRST_SLICE_TOOLS)[number];
7791

apps/worker/src/mcp/contracts/mcp-contract.json

Lines changed: 71 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
{
2-
"contractHash": "881de2fae17a183a44645d8d6c6c8c8089e604fcc2197d98d4f27acb66ec2f7b",
2+
"contractHash": "9d49bdaa24bf72887ec0477a779ab5f81f88e8805afd4806685b0ddb6b3561fe",
33
"errorCodes": [
44
"UNAUTHENTICATED",
55
"INSUFFICIENT_SCOPE",
@@ -845,6 +845,76 @@
845845
"idempotentHint": true,
846846
"openWorldHint": true
847847
}
848+
},
849+
{
850+
"name": "blocks.list",
851+
"description": "List every block type this deployment's workflow editor offers, with its presentation (label, group, description), its input contract, its output contract and the status variants it can report. `availability.available` is false for a block this deployment cannot run today (no provider configured), naming why in `unavailableReason`; the block still lists, because a graph authored now may become runnable once the provider is.",
852+
"inputSchema": {
853+
"type": "object",
854+
"properties": {}
855+
},
856+
"annotations": {
857+
"readOnlyHint": true,
858+
"destructiveHint": false,
859+
"idempotentHint": true,
860+
"openWorldHint": false
861+
}
862+
},
863+
{
864+
"name": "blocks.get",
865+
"description": "Read one block type's contract by name: the same object blocks.list returns for it. An unrecognized `type` is refused with NOT_FOUND rather than VALIDATION_FAILED, since block types are versioned by the deployment, not by this catalog.",
866+
"inputSchema": {
867+
"type": "object",
868+
"properties": {
869+
"type": {
870+
"type": "string",
871+
"minLength": 1,
872+
"maxLength": 64
873+
}
874+
},
875+
"required": [
876+
"type"
877+
],
878+
"additionalProperties": false,
879+
"$schema": "http://json-schema.org/draft-07/schema#"
880+
},
881+
"annotations": {
882+
"readOnlyHint": true,
883+
"destructiveHint": false,
884+
"idempotentHint": true,
885+
"openWorldHint": false
886+
}
887+
},
888+
{
889+
"name": "runs.stats",
890+
"description": "Roll up recent run outcomes and aggregate spend for a time window (default 24h, matching the dashboard's own default). `runs` is the newest page of outcomes in the window (`runsTruncated` says whether more exist); `cost` is the same windowed total, per-workflow breakdown and daily series the dashboard's cost view reads, computed from persisted per-run cost rather than an external provider.",
891+
"inputSchema": {
892+
"type": "object",
893+
"properties": {
894+
"window": {
895+
"type": "string",
896+
"enum": [
897+
"24h",
898+
"7d",
899+
"30d",
900+
"all"
901+
]
902+
},
903+
"limit": {
904+
"type": "integer",
905+
"minimum": 1,
906+
"maximum": 100
907+
}
908+
},
909+
"additionalProperties": false,
910+
"$schema": "http://json-schema.org/draft-07/schema#"
911+
},
912+
"annotations": {
913+
"readOnlyHint": true,
914+
"destructiveHint": false,
915+
"idempotentHint": true,
916+
"openWorldHint": false
917+
}
848918
}
849919
]
850920
}

apps/worker/src/mcp/policy.ts

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -259,6 +259,15 @@ const TOOL_POLICY = {
259259
"tickets.comment": TICKET_WRITE_POLICY,
260260
"tickets.transition": TICKET_TRANSITION_POLICY,
261261
"tickets.create": TICKET_WRITE_POLICY,
262+
// Plain reads, same reasoning as workflows.list above: the block catalog is
263+
// this deployment's own static configuration, not a customer's data, and an
264+
// agent needs it BEFORE it knows whether it may author or dispatch anything.
265+
"blocks.list": READ_POLICY,
266+
"blocks.get": READ_POLICY,
267+
// A rollup over runs.get's own data, gated the same way: seeing how the fleet
268+
// has been doing costs nothing beyond what runs.get already exposes one run
269+
// at a time.
270+
"runs.stats": READ_POLICY,
262271
} satisfies Record<McpToolName, McpToolPolicy>;
263272

264273
export function policyFor(tool: McpToolName): McpToolPolicy {

apps/worker/src/mcp/server.test.ts

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,9 @@ const PUBLISHED: McpToolName[] = [
5252
"tickets.comment",
5353
"tickets.transition",
5454
"tickets.create",
55+
"blocks.list",
56+
"blocks.get",
57+
"runs.stats",
5558
];
5659

5760
const cleanups: Array<() => Promise<void>> = [];
@@ -158,7 +161,7 @@ describe("createMcpServer", () => {
158161
data: {
159162
protocolVersions: ["2025-11-25", "2025-06-18"],
160163
serverVersion: "0.1.0",
161-
enabledDomains: ["system", "tickets", "runs", "workflows", "prompts"],
164+
enabledDomains: ["system", "tickets", "runs", "workflows", "prompts", "blocks"],
162165
// These deps carry no messaging adapter, which is the same answer a
163166
// deployment with no chat credentials gives: the authoring announcements
164167
// those tools send would reach nobody, and a client is told so rather than

apps/worker/src/mcp/server.ts

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,11 @@ import { executeMcpRead } from "./execute-tool.js";
66
import { MCP_CONTRACT_HASH } from "./sanitize-result.js";
77
import { MCP_ENABLED_DOMAINS, registerCatalogTool } from "./tool-catalog.js";
88
import { authoringAnnouncementDelivery } from "./tools/authoring-support.js";
9+
import { registerBlockTools } from "./tools/blocks.js";
910
import { registerDiscoveryTools } from "./tools/discovery.js";
1011
import { registerPromptAuthoringTools } from "./tools/prompt-authoring.js";
1112
import { registerRunControlTools } from "./tools/run-control.js";
13+
import { registerRunStatsTools } from "./tools/run-stats.js";
1214
import { registerRunTools } from "./tools/runs.js";
1315
import { registerTicketWriteTools } from "./tools/ticket-write.js";
1416
import { registerTicketTools } from "./tools/tickets.js";
@@ -63,6 +65,8 @@ export function createMcpServer(deps: McpToolDependencies): McpServer {
6365
registerWorkflowAuthoringTools(server, deps);
6466
registerRunControlTools(server, deps);
6567
registerTicketWriteTools(server, deps);
68+
registerBlockTools(server, deps);
69+
registerRunStatsTools(server, deps);
6670

6771
return server;
6872
}

apps/worker/src/mcp/surface-e2e.test.ts

Lines changed: 27 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,12 @@ const state = vi.hoisted(() => ({
4141
// just the bearer/GitHub regexes.
4242
JIRA_API_TOKEN: "jira-e2e-4d9f1b7c3a8e2510-secret",
4343
MAX_CONCURRENT_AGENTS: 4,
44+
// Read by blocks.list/blocks.get through workflowBlockRegistryContextFromEnv,
45+
// whose defaultAgent.model feeds resolveLlmProvider (lib/llm-provider.ts),
46+
// which crashes on an undefined model rather than defaulting.
47+
AGENT_KIND: "claude",
48+
CLAUDE_MODEL: "claude-opus-4-8",
49+
CODEX_MODEL: "gpt-5.4",
4450
},
4551
requireMcpActor: vi.fn(),
4652
createAdapters: vi.fn<() => Record<string, unknown>>(() => ({})),
@@ -120,6 +126,9 @@ const PUBLISHED = [
120126
"tickets.comment",
121127
"tickets.transition",
122128
"tickets.create",
129+
"blocks.list",
130+
"blocks.get",
131+
"runs.stats",
123132
];
124133

125134
const READ_ANNOTATIONS = {
@@ -216,9 +225,15 @@ const EXPECTED_ANNOTATIONS: Record<string, Record<string, boolean>> = {
216225
"tickets.comment": TICKET_WRITE_ANNOTATIONS,
217226
"tickets.transition": TICKET_TRANSITION_ANNOTATIONS,
218227
"tickets.create": TICKET_WRITE_ANNOTATIONS,
228+
// The block catalog is this deployment's own static configuration and the
229+
// run rollup is read-only over data runs.get already exposes one row at a
230+
// time, so both keep the plain read annotations.
231+
"blocks.list": READ_ANNOTATIONS,
232+
"blocks.get": READ_ANNOTATIONS,
233+
"runs.stats": READ_ANNOTATIONS,
219234
};
220235

221-
const DOMAINS = ["system", "tickets", "runs", "workflows", "prompts"];
236+
const DOMAINS = ["system", "tickets", "runs", "workflows", "prompts", "blocks"];
222237

223238
// The committed artifact, read as a file. This is the independent source for the
224239
// contract hash: MCP_CONTRACT_HASH is computed at runtime from the same catalog
@@ -833,6 +848,17 @@ const READ_TOOL_CASES = [
833848
args: { slug: PROMPT_SLUG },
834849
data: { slug: PROMPT_SLUG, version: 1, body: PROMPT_BODY, archived: false },
835850
},
851+
// Not blocks.list: the full block catalog runs past this file's deliberately
852+
// tight MCP_MAX_RESULT_BYTES (64 KiB, sized for runs.trace's pagination
853+
// tests), so it exercises the envelope's own oversized-payload fallback
854+
// rather than a normal read -- real coverage of the tool's own shape lives in
855+
// tools/blocks.test.ts, against the production-sized budget every other tool
856+
// test file uses.
857+
{
858+
tool: "blocks.get",
859+
args: { type: "loop" },
860+
data: { type: "loop" },
861+
},
836862
] as const;
837863

838864
describe("B. every read tool, over HTTP, on seeded data", () => {

apps/worker/src/mcp/tool-catalog.test.ts

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,9 @@ const CATALOGUED = [
6060
"tickets.comment",
6161
"tickets.transition",
6262
"tickets.create",
63+
"blocks.list",
64+
"blocks.get",
65+
"runs.stats",
6366
] as const;
6467

6568
// Captured off the real McpServer, through the real createMcpServer, because the

apps/worker/src/mcp/tool-catalog.ts

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -109,6 +109,17 @@ const TICKET_LABELS_MAX = 20;
109109
const TRACE_CURSOR_MAX_LENGTH = 512;
110110
const PR_URL_MAX_LENGTH = 2_048;
111111
const TRIGGER_NODE_ID_MAX_LENGTH = 200;
112+
// The longest WorkflowBlockType literal today ("trigger_pr_checks_failed") is 25
113+
// characters; this is generous headroom, not a fitted bound. Kept a plain string
114+
// rather than a zod enum of every block type on purpose: the block registry is
115+
// worker-internal and the module doc above forbids importing it here, so the one
116+
// authority on whether a type is real is blocks.get's own lookup, which answers
117+
// an unknown one with NOT_FOUND instead of a catalog-level VALIDATION_FAILED.
118+
const BLOCK_TYPE_MAX_LENGTH = 64;
119+
// Mirrors WINDOWS in db/queries/runs-read.ts (a literal, not imported, for the
120+
// reason the module doc above gives): tool-catalog.test.ts asserts the two stay
121+
// equal.
122+
const RUN_STATS_WINDOWS = ["24h", "7d", "30d", "all"] as const;
112123

113124
const runIdInputSchema = z.object({ runId: z.string().trim().min(1).max(RUN_ID_MAX_LENGTH) });
114125

@@ -419,6 +430,31 @@ export const MCP_TOOL_CATALOG = {
419430
.strict(),
420431
annotations: policyFor("tickets.create").annotations,
421432
},
433+
"blocks.list": {
434+
description:
435+
"List every block type this deployment's workflow editor offers, with its presentation (label, group, description), its input contract, its output contract and the status variants it can report. `availability.available` is false for a block this deployment cannot run today (no provider configured), naming why in `unavailableReason`; the block still lists, because a graph authored now may become runnable once the provider is.",
436+
inputSchema: z.object({}).strict().default({}),
437+
annotations: policyFor("blocks.list").annotations,
438+
},
439+
"blocks.get": {
440+
description:
441+
"Read one block type's contract by name: the same object blocks.list returns for it. An unrecognized `type` is refused with NOT_FOUND rather than VALIDATION_FAILED, since block types are versioned by the deployment, not by this catalog.",
442+
inputSchema: z
443+
.object({ type: z.string().trim().min(1).max(BLOCK_TYPE_MAX_LENGTH) })
444+
.strict(),
445+
annotations: policyFor("blocks.get").annotations,
446+
},
447+
"runs.stats": {
448+
description:
449+
"Roll up recent run outcomes and aggregate spend for a time window (default 24h, matching the dashboard's own default). `runs` is the newest page of outcomes in the window (`runsTruncated` says whether more exist); `cost` is the same windowed total, per-workflow breakdown and daily series the dashboard's cost view reads, computed from persisted per-run cost rather than an external provider.",
450+
inputSchema: z
451+
.object({
452+
window: z.enum(RUN_STATS_WINDOWS).optional(),
453+
limit: z.number().int().min(1).max(MAX_RUNS_LIMIT).optional(),
454+
})
455+
.strict(),
456+
annotations: policyFor("runs.stats").annotations,
457+
},
422458
} satisfies Record<McpToolName, McpToolDefinition>;
423459

424460
export const MCP_ENABLED_DOMAINS = [
@@ -427,6 +463,7 @@ export const MCP_ENABLED_DOMAINS = [
427463
"runs",
428464
"workflows",
429465
"prompts",
466+
"blocks",
430467
] as const;
431468

432469
const CATALOG: Record<McpToolName, McpToolDefinition> = MCP_TOOL_CATALOG;

0 commit comments

Comments
 (0)