You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: aviator/commands/spec-submit.md
+12-29Lines changed: 12 additions & 29 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -51,11 +51,12 @@ If a candidate AC fails either reader, it doesn't belong on the list.
51
51
52
52
##### The north-star test — governs every other rule below
53
53
54
-
Before writing or keeping any AC, ask: *"If this AC were violated, what specifically would get worse, and for whom?"*
54
+
Before writing or keeping any AC, both must hold:
55
55
56
-
If you can name a specific impact and a specific party affected — a user sees a bug, the build fails, the next maintainer is confused by an inconsistent pattern, the prod operator can't debug a silent failure — the AC is load-bearing, keep it.
56
+
1.**This change can break it.** Trace the AC to the diff. If nothing in this change could flip it from pass to fail — it names a code path the diff doesn't touch, it guards behavior outside this change's scope — it is out of scope. Drop it, however important the behavior is in the abstract.
57
+
2.**A violation has a named victim.** Ask "if this were violated, what gets worse, and for whom?" A specific impact on a specific party — a user hits a bug, the next maintainer trips on an inconsistent pattern, the prod operator can't debug a silent failure — means keep it. A vague answer ("things would just be less good", "code wouldn't be as clean") means filler, delete it.
57
58
58
-
If the honest answer is vague ("things would just be less good", "code wouldn't be as clean"), the AC is filler, delete it.
59
+
Test 2 alone is a trap: "if the auth cron broke, logins would fail" is severe and real, yet if this change never touches the auth cron, test 1 already ruled it out. Severity is not a license to gate what this change can't affect.
59
60
60
61
**Both functional issues *and* codebase-health issues** (broken conventions, duplicated logic, missing observability) count — anyone downstream of this change (user, reviewer, maintainer, operator) is a valid party. The build pipeline is not on this list: a green type-check/lint/format/CI run is never the impact an AC defends (see the anti-pattern below).
61
62
@@ -64,8 +65,6 @@ If the honest answer is vague ("things would just be less good", "code wouldn't
64
65
-**Functional correctness (the core of every list):** the change does the right thing — golden path, edge cases, failure modes, invariants. *"`divide(1, 0)` returns `Err(DivByZero)`"*.
65
66
-**Codebase fit (only when it's a genuine gate):** the change sits well with existing code — reuses existing helpers instead of duplicating logic, matches established patterns, doesn't quietly change a public API, doesn't pull in a new dependency. *"reuses `internal/retry.Backoff` instead of a new loop"*, *"no new entries in `package.json``dependencies`"*. Skip this axis when the change has no such gate — don't manufacture one to fill a quota.
66
67
67
-
The build pipeline is never an axis. Passing type-check, lint, formatting, compilation, or CI is not an acceptance criterion (see the anti-pattern below) — it says nothing about whether the feature works.
68
-
69
68
70
69
##### Sources to draw AC from — prioritize code over plan
71
70
@@ -88,7 +87,7 @@ If the code would fail one of your AC, that's a signal: either the AC is wrong,
88
87
89
88
Cover the meaningful behavior changes the runbook delivers. Each criterion is a gate that genuinely affects whether the work is done. Few and sharp beats many and shallow. A long list of overlapping restatements is worse than a short well-chosen set.
90
89
91
-
When the runbook's deliverable is preserved behavior — refactors, restyles, migrations, dependency upgrades, performance work — "the existing X still works" is a meaningful gate, not a cop-out. Examples: "All existing article actions remain functional and resolve to their previous routes." "Existing tests continue to pass after the change." Do not drop these from preserve-behavior runbooks just because they sound generic.
90
+
When the runbook's deliverable is preserved behavior — refactors, restyles, migrations, dependency upgrades, performance work — "the existing X still works" is a meaningful gate, not a cop-out. Examples: "All existing article actions remain functional and resolve to their previous routes." "Existing tests continue to pass after the change." Do not drop these from preserve-behavior runbooks just because they sound generic. But this only applies to behavior the change actually touches or could regress. Code the diff never goes near is unrelated code, not preserved behavior; "X still works" is not a gate when the change couldn't have broken X.
92
91
93
92
-**Verifiable, not necessarily runnable.** Runnable checks (a command exits 0, an endpoint returns 401) are the gold standard, but behavioral, qualitative, and UX criteria are equally valid as long as the criterion is sharp enough that different reviewers would reach the same verdict. The disqualifying test is ambiguity, not un-runnable-ness — rewrite or delete any AC whose pass/fail depends on interpretation.
94
93
@@ -105,16 +104,18 @@ When the runbook's deliverable is preserved behavior — refactors, restyles, mi
105
104
- Bad: an AC bullet followed by a fenced ```json``` block showing the expected response.
106
105
- Good: "The articles list response includes a published timestamp in ISO-8601 format for every article."
107
106
108
-
**Subjective taste words.** Words like "readable," "comfortable," "airy," "clean," "modern," "elegant" describe taste. The verifier has no deterministic check for them. If the spec used these words, translate the underlying intent into a checkable structural property — an observable layout assertion, not a measurement — or drop the criterion.
107
+
**Subjective taste words.** Words like "readable," "comfortable," "airy," "clean," "modern," "intuitive," "polished," "elegant" name a feeling, not a gate — two reviewers can disagree and both be right, so the verifier has no deterministic check. If the spec used these words, translate the underlying intent into a checkable structural property — an observable layout assertion, not a measurement — or drop the criterion.
109
108
- Bad: "The article body has comfortable line height and airy paragraph spacing."
110
109
- Good (if the intent is layout constraint): "The article body is constrained to a centered column rather than spanning the full viewport width."
111
110
- Or: drop the criterion entirely if other gates already capture the intent. Taste is not a gate.
112
111
113
-
**Exact file paths, module paths, function/class names, and line numbers — strictly prohibited.** These describe implementation, not outcome, and are noise to a human reader. This covers absolute paths (`src/<area>/<file>.py`), dotted module/function paths (`package.module.helper_function`), GraphQL resolver or schema-type names, and any function/class/type/module name used as the subject of the AC. If you find yourself naming an internal identifier to make the AC sound concrete, rewrite to describe the user-visible or externally observable behavior instead.
114
-
- Bad: "A new test module exists at `tests/api/users_test.py`."
115
-
- Good: "Requests with a malformed JWT return 401 from any protected endpoint."
112
+
**Internal identifiers — exact paths, function/class/type names, line numbers, infra component names — are noise, and never the subject of an AC.** Absolute paths (`src/<area>/<file>.py`), dotted paths (`package.module.helper`), GraphQL resolver/schema-type names, function/handler/class/component/hook/prop names, internal route paths, queue/task names, table/column names, and infrastructure names (Redis, Postgres, Celery, Kafka) describe implementation, not outcome. When one becomes the *subject* of a criterion, the AC reads like a code annotation rather than a behavioral gate. Reframe so the subject is the user, the customer-visible surface, or an externally observable outcome — what the caller *sees*, not which internal step produced it.
116
113
- Bad: "Function `validateJwt` exists in `src/auth/jwt.py`."
117
114
- Good: "JWT tokens are validated before any request reaches a protected route."
115
+
- Bad (mechanism-led): "Inbound SMS for unpaid customers is dropped at `/api/sms/inbound` before `process_sms` is enqueued."
116
+
- Good (outcome-led): "Inbound SMS for unpaid customers is accepted with HTTP 200 but produces no auto-replies, inbox entries, or downstream automation."
117
+
118
+
If a value or identifier IS the externally observable contract — an HTTP status code, a public API field name, a customer-facing CLI flag, a documented config key — keep it. The rule targets internal mechanism leaking into AC, not all technical specifics.
118
119
119
120
**Internal data shapes or private structures.** A reader cannot see private state without the code; the verifier checking it conflates implementation with outcome.
120
121
- Bad: "The `User` class has a `roles` list attribute."
@@ -128,29 +129,11 @@ When the runbook's deliverable is preserved behavior — refactors, restyles, mi
128
129
- When the value IS the contract, keep it verbatim. Spec said "API returns 429 when rate-limited." Keep "API returns 429 when rate-limited."
129
130
- When the value is incidental, name its role. Spec gave a specific color hex for badges. Better: "Badges use the brand accent color."
130
131
131
-
**Build / lint / type-check / format / CI gates — never an AC.** "Typecheck passes", "lint is clean", "`prettier` reports no changes", "the code compiles", "CI is green", "all tests pass" are never acceptance criteria — they are the pipeline's job and tell a reviewer nothing about whether the feature works. Do not add them, not even as a secondary item and never as a catch-all at the end of the list. The one adjacent case that IS valid is a deliberate regression guard on *preserved* behavior ("existing X still works after the refactor"), and even then phrase it as the behavior, not as "the test suite passes".
132
+
**Build / lint / type-check / format / CI gates — never an AC.** "Typecheck passes", "lint is clean", "`prettier` reports no changes", "the code compiles", "CI is green", "all tests pass" are never acceptance criteria — they are the pipeline's job and tell a reviewer nothing about whether the feature works. Do not add them, not even as a secondary item and never as a catch-all at the end of the list. The one adjacent case that IS valid is a deliberate regression guard on behavior *this change could plausibly break* ("existing X still works after the refactor"), and even then phrase it as the behavior, not as "the test suite passes".
132
133
- Don't cite the verification mechanism as the acceptance criterion. Test commands, runner invocations, CI job names, and test file paths describe *how* the behavior is checked, not *what* the behavior is — name the outcome the check defends instead. Bad: "All test cases pass." Good: "Requests to protected routes without a valid token return 401."
133
134
134
135
**Implementation choices and tradeoffs we made — not AC.** Decisions reached while building (an icon instead of a text label, a fallback format kept for older targets, building markup via the DOM rather than string concatenation) are *how* the feature was built, not gates on *what* it does. An AC states the user-visible outcome, not the option picked to reach it. If a candidate criterion would only make sense to someone who watched the session — "uses an icon, not text", "keeps a markdown fallback" — either reframe it as the behavior it produces ("the stack pastes as clickable links in chat") or drop it. The current observable behavior is the contract, not the menu of choices behind it.
135
136
136
-
**Internal code identifiers as the subject of behavioral AC.** Function names, handler names, celery/queue task names, internal route paths, middleware steps (signature validation, auth check ordering), class/component/hook/prop/attribute names, internal table/column names, and infrastructure component names (Redis, Postgres, Celery, Kafka) — when any of these become the *subject* of the criterion, the AC reads like a code annotation rather than a behavioral gate. Reframe so the subject is the user, the customer-visible product/surface, or an externally observable outcome. Describe what the user or caller *sees*, not which internal step produced it.
137
-
138
-
UI example:
139
-
- Bad: "On the queue, logs, and release pages, a polling network error shows the dismissible banner above existing data."
140
-
- Good: "When background polling fails, a dismissible error banner appears above the existing data, and the previously loaded data remains visible."
141
-
142
-
Backend example — also note how splitting one mechanism-laden bullet yields two cleaner outcome bullets:
143
-
- Bad (one bullet, mechanism-led): "Inbound SMS for unpaid customers is dropped at `/api/sms/inbound` before `process_sms` is enqueued; signature validation is skipped. The `_reactivate_account` path invalidates the Redis entry on reactivation."
144
-
- Good (split into two outcome-led bullets):
145
-
- "Inbound SMS for unpaid customers is accepted with HTTP 200 but produces no auto-replies, agent inbox entries, or downstream automation runs."
146
-
- "When an unpaid account settles its balance — via card retry success or admin override — inbound SMS resumes flowing immediately, without the customer waiting out a cache TTL."
147
-
148
-
If a value or identifier IS the externally observable contract — an HTTP status code, a public API field name, a customer-facing CLI flag, a documented config key — keep it. The rule targets internal mechanism leaking into AC, not all technical specifics.
149
-
150
-
-**Subjective taste vocabulary.** UI/UX criteria like *readable, clean, modern, intuitive, comfortable, polished, easy to use* name a feeling, not a gate — two reviewers can disagree and both be right. Reframe as a structural property the eye can verify, or delete the criterion.
151
-
- Bad: "The dashboard layout is clean and easy to scan."
152
-
- Good: "The article body is constrained to a centered column rather than spanning the full viewport width."
153
-
154
137
### When the value IS the deliverable
155
138
156
139
When the runbook's purpose is to change a specific value — a version bump, a color token swap, a config introduction — that value belongs in the AC because it IS the gate.
0 commit comments