Skip to content

Commit 7648f06

Browse files
kirill-markinclaude
andcommitted
fix(chat): give every chat tool failure the shared error shape
Every chat tool failure now carries an HttpError's code and details the same way. The list_workspaces and get_guide runner previously dropped both, while its own remediation text told the model to read details. All three chat runners now build details with createPublicHttpErrorDetails, the sanitizer MCP and REST already use, so every surface carries the same value and only the field path differs. This also stops a server-generated id collision from ever handing the model another workspace's id. It also brings the documentation up to date with the review-tools change: the chat now serves all seven registry tools, the reviewId decision and the new REVIEW_ID_CARD_MISMATCH code are documented in docs/conversational-reviews.md, and docs/agent-tool-surfaces.md, docs/fsrs-scheduling-logic.md and README.md no longer say the chat has no review tools. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 68a1239 commit 7648f06

5 files changed

Lines changed: 111 additions & 69 deletions

File tree

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ Card scheduling uses FSRS-based spaced repetition. Detailed scheduling rules liv
2222

2323
The hosted MCP server is available at `https://mcp.flashcards-open-source-app.com/mcp` and is listed in MCP registries as `com.flashcards-open-source-app/flashcards`. It exposes seven tools, most of them workspace-scoped: `list_workspaces`, `sql_query`, `sql_execute`, `get_guide`, `next_review_card`, `reveal_answer`, and `submit_review`.
2424

25-
The dedicated review tools support one-question-at-a-time conversations and idempotent FSRS scheduling. See [conversational reviews](docs/conversational-reviews.md) for the MCP/Agent API contract and voice-session examples. ChatGPT Voice currently does not invoke apps/MCP; these tools do not remove that external limitation.
25+
The dedicated review tools support one-question-at-a-time conversations and idempotent FSRS scheduling. See [conversational reviews](docs/conversational-reviews.md) for the contract shared by MCP, the in-app chat, and the Agent API, and for voice-session examples. ChatGPT Voice currently does not invoke apps/MCP; these tools do not remove that external limitation.
2626

2727
Interactive clients authenticate with OAuth 2.1 authorization code + PKCE and Dynamic Client Registration. Headless clients can use an `fca_` Bearer token.
2828

apps/backend/src/chat/openai/tools/tools.ts

Lines changed: 25 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ import {
99
} from "../../../database/transient";
1010
import { GeneratedMediaPromotionStorageTransientError } from "../../../mediaAssets/storage";
1111
import { resolveAccessibleChatWorkspaceId } from "../../../server/requestContext";
12-
import { HttpError } from "../../../shared/errors";
12+
import { createPublicHttpErrorDetails, HttpError } from "../../../shared/errors";
1313
import {
1414
ensureAIChatSyncReplica,
1515
ensureAIChatSyncReplicaWithDeadline,
@@ -452,7 +452,7 @@ function toComparableKeyList(keys: ReadonlyArray<string>): string {
452452
* optional `workspaceId` - would otherwise reach the chat model as something else: an added
453453
* optional argument would be invisible to it, and an added required one would fail every call
454454
* inside the spec's own parse against a schema the model was never shown. Property types stay
455-
* unguarded; the one enum that could drift is spread from `GUIDE_TOPICS`.
455+
* unguarded; the two enums that could drift are spread from `GUIDE_TOPICS` and `REVIEW_RATINGS`.
456456
*/
457457
function requireChatFunctionTool(spec: AgentToolSpec): OpenAI.Responses.FunctionTool {
458458
const functionTool = CHAT_FUNCTION_TOOLS[spec.name];
@@ -927,7 +927,7 @@ async function executeSqlChatToolCall(
927927
error: serializeToolError(error),
928928
instructions,
929929
code: error.code ?? undefined,
930-
details: error.details ?? undefined,
930+
details: createPublicHttpErrorDetails(error.details) ?? undefined,
931931
}
932932
: {
933933
sql,
@@ -959,8 +959,9 @@ async function executeSqlChatToolCall(
959959

960960
/**
961961
* A failure, including arguments the schema rejects, comes back as the same `{ ok: false }`
962-
* envelope a failed SQL call returns rather than as a throw, because a thrown tool call ends the
963-
* run: the model repairs its call and continues on its own remediation instructions instead.
962+
* envelope a failed SQL call returns, carrying an `HttpError`'s `code` and `details`, rather than
963+
* as a throw, because a thrown tool call ends the run: the model repairs its call and continues on
964+
* its own remediation instructions instead.
964965
*/
965966
async function executeReadOnlyChatToolCall<Data>(
966967
spec: AgentToolSpec<Data>,
@@ -988,15 +989,25 @@ async function executeReadOnlyChatToolCall<Data>(
988989
toolErrorClass: null,
989990
};
990991
} catch (error) {
991-
return {
992-
output: createToolErrorResult(spec.name, {
992+
const instructions = createAgentRemediationInstructions(
993+
error instanceof HttpError ? error.code : null,
994+
getChatToolFailureStatusCode(error),
995+
{ surface: "chat", toolName: spec.name },
996+
);
997+
const payload: ToolErrorPayload = error instanceof HttpError
998+
? {
993999
error: serializeToolError(error),
994-
instructions: createAgentRemediationInstructions(
995-
error instanceof HttpError ? error.code : null,
996-
getChatToolFailureStatusCode(error),
997-
{ surface: "chat", toolName: spec.name },
998-
),
999-
}),
1000+
instructions,
1001+
code: error.code ?? undefined,
1002+
details: createPublicHttpErrorDetails(error.details) ?? undefined,
1003+
}
1004+
: {
1005+
error: serializeToolError(error),
1006+
instructions,
1007+
};
1008+
1009+
return {
1010+
output: createToolErrorResult(spec.name, payload),
10001011
isMutating: false,
10011012
succeeded: false,
10021013
shouldInvalidateMainContent: false,
@@ -1070,7 +1081,7 @@ async function executeReviewChatToolCall(
10701081
error: serializeToolError(error),
10711082
instructions,
10721083
code: error.code ?? undefined,
1073-
details: error.details ?? undefined,
1084+
details: createPublicHttpErrorDetails(error.details) ?? undefined,
10741085
}
10751086
: {
10761087
error: serializeToolError(error),

docs/agent-tool-surfaces.md

Lines changed: 30 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -20,9 +20,9 @@ registers the specs that list its own surface
2020
| `sql_execute` | MCP, chat |
2121
| `list_workspaces` | MCP, chat |
2222
| `get_guide` | MCP, chat |
23-
| `next_review_card` | MCP |
24-
| `reveal_answer` | MCP |
25-
| `submit_review` | MCP |
23+
| `next_review_card` | MCP, chat |
24+
| `reveal_answer` | MCP, chat |
25+
| `submit_review` | MCP, chat |
2626
| `add_generated_image_to_card` | chat only, not a registry spec |
2727

2828
Every workspace-scoped registry spec takes the same optional `workspaceId`
@@ -46,13 +46,16 @@ the MCP gateway's 29-second integration timeout
4646
(`infra/aws/lib/gateways/mcp-gateway.ts`). That is one dated sample rather than a
4747
standing contract; re-measure before deciding.
4848

49-
The chat deliberately has no review tools. A review needs one durable `reviewId`
50-
per learner review, scoped to the authenticated connection, and the chat's
51-
connection id is the constant `"chat-v2"`
52-
(`apps/backend/src/chat/openai/tools/tools.ts`), which cannot scope one the way
53-
an API key connection does. Until that identity is decided, the chat calls the
54-
SQL, workspace, and guide tools only. The review contract itself is
55-
[conversational reviews](conversational-reviews.md).
49+
The chat serves all seven registry tools, review included. The model supplies
50+
the `reviewId` there exactly as it does on MCP, and a `reviewId` reused on a
51+
different card is refused on every surface with its own code instead of passing
52+
as a retry (`submitAgentReview` in `apps/backend/src/agent/reviews.ts`). What
53+
the surfaces do not share is the sync replica that forms half of that dedup key:
54+
the chat authenticates as no agent connection, so a review is stored against the
55+
workspace's shared AI-chat replica (`buildChatAgentToolContext` in
56+
`apps/backend/src/chat/openai/tools/tools.ts`), which makes one `reviewId`
57+
namespace cover every member and every session of that workspace. The review
58+
contract itself is [conversational reviews](conversational-reviews.md).
5659

5760
## REST is not a registry surface
5861

@@ -82,7 +85,16 @@ result envelope, and output budget, and each of those stays in its own adapter:
8285
failing it (`apps/backend/src/chat/openai/tools/toolResults.ts`). Its envelope is
8386
`{ ok, tool, data, instructions }` with no `docs` block, and a failure comes
8487
back as `{ ok: false }` rather than as a throw, because a thrown tool call
85-
ends the run.
88+
ends the run. That failure carries an `HttpError`'s `code` and `details` at
89+
the envelope's top level, beside an `error` of just `name` and `message`
90+
(`createToolErrorResult`, same module), where MCP and REST nest both under the
91+
agent envelope's `error` (`createAgentErrorEnvelope` in
92+
`apps/backend/src/agent/envelope.ts`). Only `code` and `details` move with
93+
that, and `details` moves unchanged, because all three surfaces build it with
94+
`createPublicHttpErrorDetails` (`apps/backend/src/shared/errors.ts`); `ok`,
95+
`instructions`, and `error.message` are read at the same path on either shape,
96+
which is why a remediation meaning can be reworded for the chat over the path
97+
alone.
8698
- REST: the same agent envelope as MCP, built per route against the request URL
8799
(`apps/backend/src/routes/agent.ts`).
88100

@@ -94,7 +106,8 @@ Two invariants are load-bearing and invisible unless you look for them:
94106
workspace.
95107
- Every action a surface's tools do not reach is bound to
96108
`unboundAgentToolAction` (`apps/backend/src/aiTools/toolRegistry/actions.ts`).
97-
Each adapter names all of them, so omitting one is a type error and a tool
109+
Both adapters bind every action for real today, so nothing reaches it, but each
110+
adapter still names all of them, so omitting one is a type error and a tool
98111
added to a surface later cannot quietly reach production code around that
99112
surface's dependencies and its tests' fakes.
100113

@@ -134,11 +147,11 @@ The per-code map and the full remediation text live in the remediation module,
134147
which is keyed by HTTP error code. Four facts are surface-specific enough to
135148
record here:
136149

137-
- `WORKSPACE_NOT_FOUND` (404) is reachable from the chat's SQL tools when the
138-
model passes a `workspaceId` the account cannot access: the chat's workspace
139-
resolution reaches `assertUserHasWorkspaceAccess`
140-
(`apps/backend/src/workspaces/selection.ts`) before any SQL runs. It is the
141-
only meaning the chat alone words.
150+
- `WORKSPACE_NOT_FOUND` (404) is reachable from every workspace-scoped chat tool
151+
when the model passes a `workspaceId` the account cannot access: the chat's
152+
workspace resolution reaches `assertUserHasWorkspaceAccess`
153+
(`apps/backend/src/workspaces/selection.ts`) before the tool's own work runs.
154+
It is the only meaning the chat alone words.
142155
- `WORKSPACE_SELECTION_REQUIRED` (409) is reachable on REST and MCP but never on
143156
the chat, whose tool context always passes the session workspace as the
144157
selected default, so its resolver never sees none (`buildChatAgentToolContext`

docs/conversational-reviews.md

Lines changed: 49 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,11 @@
1-
# Conversational reviews over MCP and the Agent API
1+
# Conversational reviews over MCP, the in-app chat, and the Agent API
22

3-
An MCP-capable voice client can review one question at a time, assess the learner's
4-
answer, explain any gaps, choose a rating, and persist the next FSRS schedule
5-
without asking the learner to rate every card. The calling agent performs the
6-
assessment; the backend accepts its rating and schedules the review. No SQL writes
7-
to review history or hidden scheduler columns are allowed.
3+
An MCP-capable voice client, the in-app chat, or an Agent API caller can review
4+
one question at a time, assess the learner's answer, explain any gaps, choose a
5+
rating, and persist the next FSRS schedule without asking the learner to rate
6+
every card. The calling agent performs the assessment; the backend accepts its
7+
rating and schedules the review. No SQL writes to review history or hidden
8+
scheduler columns are allowed.
89

910
**External limitation:** ChatGPT Voice currently does not invoke apps/MCP. These
1011
server tools do not remove that OpenAI product limitation. They support clients
@@ -14,12 +15,11 @@ microphone, speech recognition, speech synthesis, or a ChatGPT Voice integration
1415

1516
## Tools and HTTP actions
1617

17-
The three review tools are the only ones MCP serves alone; `list_workspaces`,
18-
`sql_query`, `sql_execute`, and `get_guide` are shared with the in-app chat,
19-
which has no review tools ([agent tool surfaces](agent-tool-surfaces.md)).
20-
`get_guide` topic `review_flow` returns this review contract.
18+
MCP and the in-app chat serve the same registry tools, the three review tools
19+
included ([agent tool surfaces](agent-tool-surfaces.md)). `get_guide` topic
20+
`review_flow` returns this review contract.
2121

22-
| MCP tool | Agent API action | Result in `data` | Effect |
22+
| Tool | Agent API action | Result in `data` | Effect |
2323
| --- | --- | --- | --- |
2424
| `next_review_card` | `POST /v1/agent/reviews/next` | `workspaceId`, `card: {cardId, frontText}` or `card: null` | Read only |
2525
| `reveal_answer` | `POST /v1/agent/reviews/reveal` | `workspaceId`, `cardId`, `backText` | Read only |
@@ -29,10 +29,13 @@ The optional `workspaceId` argument and the rejection of unknown arguments are
2929
shared agent-tool rules, described once in
3030
[agent tool surfaces](agent-tool-surfaces.md). An HTTP request with no body at
3131
all is a valid call. `reveal_answer` requires `cardId`. HTTP actions use
32-
`Authorization: ApiKey <fca_...>` and the same JSON arguments as the MCP tools.
33-
MCP continues to accept OAuth authorization or an API key as a Bearer token.
34-
Authentication and current workspace membership are checked on each request. IDs
35-
are UUIDs.
32+
`Authorization: ApiKey <fca_...>` and the same JSON arguments the tools take.
33+
MCP continues to accept OAuth authorization or an API key as a Bearer token. The
34+
chat has no API key of its own: it runs under the chat session's transport,
35+
including a native guest session (`assertSupportedTransport` in
36+
`apps/backend/src/chat/http/dependencies.ts`), and nothing on the review path
37+
gates on being signed in. Authentication and current workspace membership are
38+
checked on each request. IDs are UUIDs.
3639

3740
## Choosing the next card
3841

@@ -122,8 +125,10 @@ values remain 0–3. What each rating means, how to judge an equivalent or
122125
ambiguous answer, when to honor a learner's explicit rating, and that “perfectly
123126
remembered” is only a spoken alias for `Easy`, are all one constant:
124127
`REVIEW_FLOW_INSTRUCTIONS` in `apps/backend/src/agent/reviewContract.ts`, which
125-
every review tool returns in full with each result and `get_guide` topic
126-
`review_flow` serves on demand.
128+
MCP and the HTTP actions return in full with each result and `get_guide` topic
129+
`review_flow` serves on demand. A chat review result points the model at that
130+
topic instead of carrying the constant (`CHAT_REVIEW_RESULT_INSTRUCTIONS` in
131+
`apps/backend/src/chat/openai/tools/tools.ts`).
127132

128133
## Result, retries, and offline behavior
129134

@@ -136,18 +141,27 @@ day interval and can be zero for a minutes-long learning step. `state` is
136141
writable through this contract.
137142

138143
- Generate and durably retain one `reviewId` UUID per learner review, scoped to
139-
the authenticated connection and workspace. Keep the same connection when
140-
recovering an uncertain request. Reconnecting with a different connection or
141-
changing the ID is a new review, not a retry.
144+
the workspace and to the calling surface's own sync replica
145+
([agent tool surfaces](agent-tool-surfaces.md)): on MCP and the HTTP actions
146+
that replica is the agent connection, so keep the same connection when
147+
recovering an uncertain request and treat a different one as a new review; on
148+
the chat it is shared, so a `reviewId` must be unique across the whole
149+
workspace. Changing the ID is a new review, not a retry.
142150
- A repeated submission is deduplicated by the
143151
`UNIQUE (workspace_id, replica_id, client_event_id)` constraint on
144152
`content.review_events`, so it can never record a second review or advance the
145-
schedule again. A retry whose review already landed answers
146-
`409 REVIEW_EVENT_CONFLICT` with the card's current schedule in
147-
`error.details.reviewSchedule` (`cardId`, `dueAt`, `intervalSeconds`,
148-
`scheduledDays`, `state`, `reps`, `lapses`). The same code answers an
149-
unrelated pre-existing event identity, which likewise cannot advance
150-
scheduling. What the calling agent should do about it is in
153+
schedule again. Retrying the same card answers `409 REVIEW_EVENT_CONFLICT`
154+
with that card's current schedule in the failure's `reviewSchedule` details
155+
(`cardId`, `dueAt`, `intervalSeconds`, `scheduledDays`, `state`, `reps`,
156+
`lapses`), wherever that surface's envelope carries a failure's `details`
157+
([agent tool surfaces](agent-tool-surfaces.md)); on the chat, whose replica is
158+
shared, two members or two sessions of one workspace that chose the same
159+
`reviewId` for that card reach the same code without either retrying. Retrying
160+
that card is still the only supported reuse of a `reviewId`.
161+
- `409 REVIEW_ID_CARD_MISMATCH` means the `reviewId` already identifies a
162+
recorded review of a different card, so nothing was stored for the card just
163+
submitted and no schedule advanced. Generate a new `reviewId` for it and
164+
submit again. What the calling agent should do about either code is in
151165
`REVIEW_FLOW_INSTRUCTIONS`.
152166
- `409 REVIEW_STALE` means the card's stored `fsrs_last_reviewed_at` is at or after
153167
the current server time, so the scheduler cannot move forward from it. Only server
@@ -162,9 +176,10 @@ writable through this contract.
162176

163177
## Implementation and verification
164178

165-
`apps/backend/src/agent/reviewContract.ts` shares strict schemas across MCP and
166-
HTTP. `apps/backend/src/agent/reviews.ts` uses the existing agent replica identity
167-
and delegates to `cards/review/reviews.ts::submitReviewInExecutor`. The scheduler
179+
`apps/backend/src/agent/reviewContract.ts` shares strict schemas across MCP, the
180+
chat, and HTTP. `apps/backend/src/agent/reviews.ts` takes the sync replica from
181+
its caller rather than choosing one, and delegates to
182+
`cards/review/reviews.ts::submitReviewInExecutor`. The scheduler
168183
algorithm is unchanged. One workspace-locked transaction inserts the review,
169184
updates FSRS state, and records progress/activity facts and hot sync metadata;
170185
review history continues through its append-only sequence, and post-commit
@@ -173,8 +188,11 @@ permission was added, and this work adds no table of its own.
173188

174189
Run backend `npm test`, `npm run lint`, `npm run test:mcp`, and
175190
`npm run test:postgres-integration` with an isolated PostgreSQL 18 administrative
176-
URL in `POSTGRES_INTEGRATION_ADMIN_URL`. The deployment smoke script also checks
177-
the tool inventory.
191+
URL in `POSTGRES_INTEGRATION_ADMIN_URL`. The deployment smoke script
192+
`scripts/checks/check-mcp-smoke.sh` checks the tool inventory and then runs one
193+
real review over MCP with no model in the loop: next, reveal, submit, and a
194+
repeat of that submission, which must answer `REVIEW_EVENT_CONFLICT` with the
195+
stored schedule.
178196

179197
For a manual voice smoke check after deployment, follow the session above with
180198
one disposable card for each rating. Verify that the agent explains gaps,

docs/fsrs-scheduling-logic.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -405,9 +405,9 @@ Any scheduler change must update:
405405
## Dedicated agent review adapter
406406

407407
The [conversational review contract](conversational-reviews.md) exposes the existing
408-
scheduler through MCP `submit_review` and HTTP `POST /v1/agent/reviews/submit`, and
409-
selects its next card with the queue order above. It maps exact
410-
`Again`/`Hard`/`Good`/`Easy` strings to 0–3, stamps the review instant on the server
411-
because the surface is online only, and returns the due time, interval, state, reps,
412-
and lapses. It does not change the scheduler algorithm or released first-party sync
413-
contracts.
408+
scheduler through the MCP and in-app chat `submit_review` tools and HTTP
409+
`POST /v1/agent/reviews/submit`, and selects its next card with the queue order above.
410+
It maps exact `Again`/`Hard`/`Good`/`Easy` strings to 0–3, stamps the review instant
411+
on the server because the surface is online only, and returns the due time, interval,
412+
state, reps, and lapses. It does not change the scheduler algorithm or released
413+
first-party sync contracts.

0 commit comments

Comments
 (0)