Skip to content

Commit a1a04e0

Browse files
ajcarberryclaude
andcommitted
fix(write-tests): boundary/wire mocking over interface fakes
The skill taught patterns that a real Go CLI testing effort found to be anti-patterns. Bring it in line. - Replace the three-layer strategy (which framed interface-based fakes as a legitimate boundary approach) with boundary-first testing: fake the executable on PATH and assert the real argv, or httptest and assert the method/path/body. Drop the CommandRunner interface pattern — asserting "Run was called" proves the code called the fake, not that it built the right command, and it couples tests to a test-only interface. - Stop recommending t.Skip() when the binary is absent — that is a false-green (the coverage gate runs `go test` without building first). Build the binary in TestMain; a miss is a hard failure. Note that subprocess tests don't count toward a package's coverage, so command logic needs in-process cobra tests (with an os.Stdout capture helper, since commands print via fmt.Printf, not cmd.OutOrStdout). - Switch examples from testify to stdlib `testing` (t.Fatalf preconditions, t.Errorf checks, reflect.DeepEqual, errors.Is/As) to match stdlib codebases; keep a note to match testify only where a project already uses it. - Add the "prove the negative" pattern for destructive ops (fake fails the test on any mutating call) and a Determinism section (map-iteration and unseeded-RNG bugs). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AfCuqUGhxYxbvo4FgkBtFm
1 parent 419f3ac commit a1a04e0

2 files changed

Lines changed: 330 additions & 307 deletions

File tree

plugins/launchpad/skills/write-tests/SKILL.md

Lines changed: 43 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
name: write-tests
3-
description: Use when writing tests, adding test coverage, choosing test types, testing a specific function, or when asked "should I mock this", "how should I test this", "write tests for", "add tests", "test this function", "write integration tests", or "rewrite test suite". Provides three-layer testing strategy, specification-grade testing workflow, boundary mocking, and Go-specific patterns including Cobra command, Bubble Tea model, and runner testing.
3+
description: Use when writing tests, adding test coverage, choosing test types, testing a specific function, or when asked "should I mock this", "how should I test this", "write tests for", "add tests", "test this function", "write integration tests", or "rewrite test suite". Provides a boundary-first testing strategy (wire-level fakes, httptest, in-process commands), specification-grade workflow, and Go-specific patterns including Cobra command, Bubble Tea model, and runner testing.
44
---
55

66
# Writing Specification-Grade Tests
@@ -58,42 +58,45 @@ tools/cluster/
5858
- **`test/`** — E2E tests that execute the built binary
5959
- **`testdata/`** — static fixtures (ignored by Go tooling)
6060

61-
## Three-Layer Testing Strategy
61+
## Testing Strategy: Test at the Boundary
6262

6363
> "The more your tests resemble the way your software is used, the more
6464
> confidence they can give you." — Kent C. Dodds
6565
66-
**Every testable behavior gets a Layer 3 (E2E) test.** Layers 1 and 2 supplement
67-
Layer 3 — they keep CI green when real tools aren't available, but they never
68-
replace E2E coverage.
69-
70-
| Layer | Technique | Purpose |
71-
|-------|------------------------|-----------------------------------------------|
72-
| 3 | Real execution (E2E) | **Required.** Proves the real thing works |
73-
| 1 | Filesystem isolation | CI fallback — logic with real files via `t.TempDir()` |
74-
| 2 | Interface-based fakes | CI fallback — verifies command wiring only |
75-
76-
**Layer 3 is the standard.** Build the binary, run real commands, assert on real
77-
output. Guard E2E tests with `t.Skip()` when the environment isn't available
78-
(missing binary, no cluster) — but the tests must exist.
79-
80-
**Layers 1 and 2 are CI safety nets.** They catch regressions fast when E2E can't
81-
run. Layer 2 fakes can only verify your code builds the *intended* command — they
82-
cannot prove the command actually works. Treat them as regression guards, not proof
83-
of correctness.
84-
85-
**Choosing the primary layer by function type:**
86-
- Reads/writes files → **Layer 1** (integration with `t.TempDir()`)
87-
- Shells out to external tools → **Layer 2** (boundary fake) + **Layer 3** (E2E with `t.Skip()`)
88-
- Pure computation → **Unit test**
89-
- State machine (TUI) → State transition tests (send messages, assert model)
90-
91-
**Mock at the boundary, not inside:**
92-
- External commands (nomad, terraform, ansible) — define a narrow interface, inject a fake
93-
- Filesystem operations — use `t.TempDir()` with real reads and writes, not mocks
94-
- Internal modules and your own code — always use the real thing
95-
96-
See [references/go.md](references/go.md) for the CommandRunner pattern.
66+
Test the observable contract, and mock only at the process/network boundary —
67+
never inside your own code.
68+
69+
| What the code does | How to test it |
70+
|--------------------|----------------|
71+
| Pure computation | Call it with constructed inputs; assert the return and its edge cases |
72+
| Reads/writes files | Real files in `t.TempDir()` — no filesystem mocks |
73+
| Shells out to a tool (nomad/terraform/uv) | Fake the executable on `PATH`, record its argv, assert the exact command built |
74+
| Talks HTTP (an API, Prometheus, Loki) | `httptest.Server` — assert the request sent, parse a real response |
75+
| Command (Cobra) | Execute it in-process; assert stdout/stderr and error/exit behavior |
76+
| State machine (TUI) | Send messages to `Update`, assert the returned model; assert `View` output |
77+
78+
**Mock at the wire, never internals.** For a runner that does
79+
`exec.Command("nomad", "job", "run", path)`, install a fake `nomad` on `PATH`
80+
that records its argv, then assert the argv. Do **not** define a `CommandRunner`
81+
interface and assert "Run was called" — that proves the code called your fake,
82+
not that it built the right command, and it breaks on harmless refactors. Same
83+
for HTTP: point the client at an `httptest.Server`; don't stub the client type.
84+
Your own internal modules are always used for real.
85+
86+
**Build the binary; never skip on its absence.** End-to-end tests that run the
87+
built CLI belong in `test/`, and `TestMain` builds the binary once before they
88+
run — a missing binary is a hard failure, not a `t.Skip()`. (Skipping here is a
89+
false-green: if the coverage gate runs `go test` without building first, every
90+
skipped test silently "passes.") A subprocess test does not count toward the
91+
tested package's coverage, so cover command logic **in-process** as well.
92+
93+
**Prove the negative for destructive ops.** For a dry-run, a declined
94+
confirmation, or an empty-input guard, wire the fake boundary to **fail the
95+
test** if a mutating call arrives — making "did not mutate" a structural
96+
guarantee, not an assertion a later edit could quietly drop.
97+
98+
See [references/go.md](references/go.md) for the wire-fake, `httptest`, and
99+
in-process command patterns.
97100

98101
## Table-Driven Tests
99102

@@ -144,6 +147,10 @@ See [references/go.md](references/go.md) for the full assertion reference.
144147
| Bad Pattern | Good Pattern |
145148
|----------------------------------|--------------------------------------------|
146149
| Testing mock behavior | Test actual outcome with real dependencies |
150+
| Interface fake asserting "method X was called" | Fake the executable on `PATH` / `httptest`; assert the real argv or request |
151+
| Asserting a struct literal you just built | Assert the effect through the boundary (the flag the fake received) |
152+
| `t.Skip()` on a missing-but-buildable binary/tool | Build it in `TestMain`; skip only a genuinely external live service |
153+
| Regression test that passes without the fix | Ensure it fails against the pre-fix code |
147154
| One assertion per function | Group related assertions in one test |
148155
| Copy-pasting setup across tests | Extract to `t.Helper()` function |
149156
| Percentage-based coverage goals | Cover behavior and edge cases |
@@ -158,7 +165,9 @@ See [references/go.md](references/go.md) for the full assertion reference.
158165
- [ ] Happy path covered
159166
- [ ] Error conditions and edge cases handled
160167
- [ ] Error messages asserted (not just `wantErr: true`)
161-
- [ ] Real dependencies used (fakes only at external boundaries)
168+
- [ ] Boundaries mocked at the wire (fake exec on `PATH` / `httptest`), not via internal interfaces
169+
- [ ] Destructive-op guards prove the negative (the fake fails the test on any mutating call)
170+
- [ ] A bug fix ships a regression test that fails before the fix
162171
- [ ] Tests survive refactoring
163172
- [ ] Test names read as specifications
164173
- [ ] Table-driven where multiple scenarios exist

0 commit comments

Comments
 (0)