Skip to content

Commit 2ac1952

Browse files
committed
docs(20.2): record the catch-all rule and the JSONDecodeError trap
Both are non-obvious enough to reintroduce. Catching only HTTPStatusError looks like error handling but covers a 500 and nothing else; JSONDecodeError subclassing ValueError is easy to miss and silently blames the caller. Adds the phase plan documenting why coverage had to land before the decode guard, and why the first offender count of 136 was wrong.
1 parent d4167a6 commit 2ac1952

2 files changed

Lines changed: 76 additions & 0 deletions

File tree

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
---
2+
phase: "20.2"
3+
plan: "01"
4+
title: Catch-all coverage for every tool + decode guard in api_get
5+
status: complete
6+
---
7+
8+
# Plan 20.2-01: Catch-all coverage for every tool + decode guard in api_get
9+
10+
<objective>
11+
Make every module classify an upstream failure as an upstream failure. Two
12+
independent defects produce the same symptom — an outage that reaches the agent
13+
as caller error, or as no envelope at all.
14+
</objective>
15+
16+
## Findings that shaped the plan
17+
18+
Measured with an AST sweep over all 271 `@tool` functions, not by grep:
19+
20+
| Handler shape | Tools | Consequence |
21+
|---|---|---|
22+
| `except httpx.HTTPStatusError` only | 68 | timeout / connect / decode escape raw |
23+
| no exception handling at all | 41 | everything escapes raw |
24+
| `except WfsError` only (british_columbia) | 15 | everything else escapes raw |
25+
| `ValueError` + `HTTPStatusError` | 12 | mislabels **and** leaks |
26+
| **total genuinely uncovered** | **108** | across 13 modules |
27+
28+
A first pass counted 136. That was wrong: `york_region`'s 28 tools delegate to a
29+
`_call_client` helper that already has `except Exception -> UPSTREAM_ERROR`, so
30+
they were false positives. The detector now resolves module-level helper
31+
delegation before reporting.
32+
33+
The second defect: `json.JSONDecodeError` subclasses `ValueError`. An upstream
34+
answering HTTP 200 with an HTML error page raises it out of
35+
`shared/http.py:api_get`, straight into the `except ValueError -> INVALID_INPUT`
36+
arms of statcan, ircc, manitoba, saskatchewan, nova_scotia, british_columbia and
37+
datastore.
38+
39+
**Why the obvious one-line fix was rejected.** Raising `httpx.DecodingError`
40+
from `api_get` alone would have made things *worse*: it is an `httpx.HTTPError`
41+
but **not** an `httpx.HTTPStatusError`, and 5 modules caught only the latter — so
42+
a mislabelled error would have become an unhandled one, the exact failure Phase
43+
20.1 removed. Coverage had to land first. Order is load-bearing.
44+
45+
## Tasks
46+
47+
- [x] **RED**`tests/test_tool_error_handling.py`: AST guard asserting every
48+
`@tool` is covered by `@upstream_guard`, a broad `except`, or a safe helper.
49+
Failed with 108 offenders. Includes a self-test proving the detector rejects
50+
an `HTTPStatusError`-only shape, so it cannot pass vacuously.
51+
- [x] **GREEN** — applied `@upstream_guard(<api_name>)` beneath `@tool` on all 108.
52+
`api_name` derived per-tool from each tool's own `make_response`/`make_error`
53+
call (107 of 108); `wx_get_lightning` makes no network call and takes the
54+
module constant so the rule has zero exceptions.
55+
- [x] **RED**`tests/test_http.py`: `api_get` must raise `httpx.DecodingError`,
56+
and explicitly *not* a `ValueError`, on a malformed body.
57+
- [x] **GREEN** — decode guard in `api_get`.
58+
- [x] **Behavioural cover**`tests/test_upstream_error_classification.py` pins
59+
the user-visible contract: timeout / connect-error / malformed-JSON each
60+
return a transient envelope, malformed JSON is never `INVALID_INPUT`,
61+
genuine bad input still is, and `lang` survives onto the envelope.
62+
- [x] **Non-vacuity check** — reverted `ircc/tools.py` + `shared/http.py` and
63+
confirmed 7 of the 8 new behavioural tests fail. The one that stays green is
64+
`genuine_bad_input_is_still_invalid_input`, which is behaviour to preserve.
65+
66+
## Deliberately not done
67+
68+
- **`upstream_guard` is additive, not a rewrite.** It wraps the function, so each
69+
tool's existing handlers still run first and the guard only catches what
70+
escapes. No working error path was removed.
71+
- **The 41 unhandled tools were not given bespoke handlers.** Uniform decoration
72+
is verifiable by the AST guard; 41 hand-written blocks would not be.

CLAUDE.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -96,6 +96,10 @@ Every module in `src/mcp_canada/modules/{name}/`:
9696

9797
**Every `@tool` must:** use standalone `@tool` from `fastmcp.tools`, include `lang: Literal["en", "fr"]`, return `make_response()`/`make_error()`, have `Use for:` + `Keywords:` in docstring, use module prefix (`boc_`, `parl_`, etc.).
9898

99+
**Every `@tool` must have catch-all error coverage** — enforced by `tests/test_tool_error_handling.py` in the default unit suite. Satisfy it with `@upstream_guard(<api_name>)` beneath `@tool` (preferred — it is additive, so any handlers inside the function still run first), a broad `except Exception`/`httpx.HTTPError`, or delegation to a module helper that has one. **Catching only `httpx.HTTPStatusError` is not enough:** it covers a 500 but not a timeout, a connect error or a malformed body, each of which escapes as a raw `ToolError`. Phase 20.2 found 108 of 271 tools in that state.
100+
101+
**Never let `json.JSONDecodeError` reach an `except ValueError` arm.** It subclasses `ValueError`, so an upstream answering HTTP 200 with an HTML error page gets reported as `INVALID_INPUT` — blaming the caller for someone else's outage, and failing live tests with a misleading code (`assert_live_or_transient` tolerates only `UPSTREAM_ERROR`/`RATE_LIMITED`/`UPSTREAM_UNAVAILABLE`). `shared/http.py:api_get` converts it to `httpx.DecodingError`, which is an `HTTPError` but not a `ValueError`. Any client calling `.json()` directly must do the same.
102+
99103
**Every client function must:** return `(data, was_cached)`, use `cached_fetch()` + `get_limiter()`, flatten responses aggressively.
100104

101105
**Don't:** add dependencies, modify `server.py` for new modules, put module tests in top-level `tests/`, skip rate limiting, mix refactoring with feature work.

0 commit comments

Comments
 (0)