test: pre-release verification harness (make verify)#60
Merged
Conversation
Standing release gate that turns 'did we miss anything?' into a number. Four layers: 1. wire coverage — fail if any endpoint lacks a wire test (static) 2. live read sweep — auto-enumerate + hit every no-arg read endpoint (live) 3. field-drop audit — fields the API returns but the SDK doesn't type (live) 4. helper coverage — every hand-written helper has a unit test (static) Live layers need SMALLEST_API_KEY; hard gates (1,2,4) fail the build, field-drop is a warning (tracked as spec completeness). Run: make verify. Also adds tests/custom/test_helpers_rest.py (KB/Audience/Campaign/Page — the helpers the harness flagged as untested). First run found: calls/agent-schema field-drops (escalated) + 4 untested helpers (now covered). mypy green (703 files).
Makes the verification harness self-enforcing: a new endpoint without a wire test, or a hand-written helper without a unit test, now fails CI automatically (no need to remember 'make verify'). Live layers skip without SMALLEST_API_KEY.
hamees-sayed
approved these changes
Jun 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
make verify— the standing release gate (wire coverage + live read sweep + field-drop audit + helper coverage), the systematic answer to test-coverage assurance.What it does
One command, four layers, one pass/fail report:
Live layers need
SMALLEST_API_KEY. Hard gates (1,2,4) fail; field-drop is a warning (tracked as spec completeness).First-run findings (already acted on)
tests/custom/test_helpers_rest.pycalls+ agent schema → escalated to docs for 5.1.x spec completenessVerified: mypy==1.13.0 green (703 files). Does not change the package version → no re-publish.
🤖 Generated with Claude Code