Skip to content

Commit 7a63848

Browse files
authored
fix(advisor): require reduction-directed design blockers (#9631)
<!-- markdownlint-disable MD041 --> ## Summary The PR Review Advisor should block material codebase-design defects without sending authors through review-driven growth loops. This change keeps the evidence and anti-noise safeguards from #6682 and #6832, makes maintainability defects first-class blockers, and requires simplicity feedback to point toward a smaller complete source-and-test design. ## Changes - Define duplicated authority, unnecessary machinery, repeated setup, widened dependencies, unrelated churn, and unsupported product scope as present design defects when checked-in evidence supports a behavior-preserving reduction. - Require blocker severity when the advisor expects author action before merge. Keep warnings optional only when maintainers can reasonably accept the current design without author action. - Require an unnecessary-complexity finding to name its reduction case across source and tests. Prefer a negative total line delta; permit a line-neutral result only for a material reduction in owners, concepts, invalid combinations, or dependency width. - Reject net-new helpers, registries, configuration surfaces, compatibility layers, fallbacks, migrations, test frameworks, and parallel fixture owners as simplification remedies unless current consumers adopt them and the complete change removes more structure than it adds. - Preserve semantic regression coverage while allowing duplicated fixtures, matrices, assertions, and test owners to be consolidated. - Keep follow-on findings grounded in current code, scope ownership findings to checked-in duplication, and classify merge-order-only redundancy as superseded or a limitation. - Keep a prior design root cause stable when a review response adds machinery instead of creating a serial chain of architecture findings. - Apply the same deletion and consolidation questions in the shared code-change considerations and the issue implementation skill so agents optimize the design before publication. - Add prompt, shared-guidance, and skill contract coverage for the new direction and retained anti-noise rules. ## Type of Change - [ ] Code change (feature, bug fix, or refactor) - [x] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [ ] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [ ] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## DGX Station Hardware Evidence - [ ] Tested on DGX Station - Tested commit: - Station profile/scenario: - Result: - Supporting evidence: ## Verification - [x] PR description includes a `Signed-off-by:` line and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run validate:pr` passed after refreshing `origin/main` when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — `npm exec -- vitest run test/code-change-considerations.test.ts test/pr-review-advisor-writing-guides.test.ts test/pr-review-advisor-context.test.ts test/skills-frontmatter.test.ts` (82 passed); `npm run test:changed` (growth guardrails: 32 passed; no changed CLI, plugin, or E2E-support tests) - [ ] Applicable broad gate passed — `npm test` for broad runtime/test-harness changes; `npm run check` for repo-wide validation/coverage changes — command/result: - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) - [ ] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new doc pages only) --- Signed-off-by: Julie Yaunches <jyaunches@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Improvements** * Review guidance now more consistently identifies material design, security, authorization, scope, duplication, and complexity issues as blockers when supported by evidence. * Simplification recommendations favor reducing source and test complexity, avoiding added mechanisms and unsupported future-use justifications. * Checks better distinguish current defects from hypothetical risks, resolved behavior, and non-actionable concerns. * Terminology reviews and retry handling now provide clearer, more reliable outcomes. * **Documentation** * Clarified expectations for documenting reductions, line changes, preserved regression coverage, ownership evidence, and independent follow-on risks. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
1 parent 0e655a1 commit 7a63848

10 files changed

Lines changed: 167 additions & 14 deletions

.agents/skills/_shared/code-change-considerations.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,13 +17,22 @@ recording them here.
1717

1818
- What accepted outcome and current consumer require the change?
1919
- What current code owns the behavior, and can that owner be extended directly?
20+
- What current code, branch, parameter, owner, fixture, or file becomes unnecessary and can be
21+
deleted or merged in this change?
2022
- Would the change duplicate an existing structure or create another source of truth?
23+
- Can the completed source-and-test change be neutral or negative in total lines? If not, what
24+
current correctness, security, or accepted-scope contract requires the growth?
25+
- If the change adds a helper, abstraction, configuration, registry, fallback, or compatibility
26+
path, which current consumers adopt it now, what old structure does it remove, and is the whole
27+
result smaller or simpler?
2128
- What state, success, failure, and partial-failure behavior must remain coherent?
2229
- What ordering or concurrency can change the result or bypass a guarantee?
2330
- How do absent values, defaults, retries, recovery, and cleanup behave?
2431
- Which alternate entry, error, cached, resumed, or compatibility paths can bypass the change?
2532
- Can code or configuration be removed, or can an existing or native mechanism replace new code?
2633
- What shortest stable test proves the changed behavior, including the relevant negative path?
34+
- Can that evidence extend or consolidate current fixtures, matrices, and assertions instead of
35+
creating another test owner or a one-use test helper?
2736
- Does a real process, network, filesystem, container, hardware, or service boundary require deeper
2837
runtime or end-to-end evidence?
2938
- Which active issues, pull requests, or recent changes overlap, conflict, or affect delivery order?

.agents/skills/nemoclaw-contributor-implement-issue/SKILL.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,13 +80,25 @@ checked for that operation or failure class and the sensitive-workflow state out
8080
hold. Then make the direct change in the current behavior owner. Do not add speculative abstractions,
8181
configuration, compatibility, migration, or fallback behavior.
8282

83+
Optimize the complete source-and-test change for deletion and consolidation. Compare a direct edit,
84+
reuse of an existing owner, and a refactor of current related code. Prefer a neutral or negative
85+
total line delta. Add a helper, abstraction, configuration surface, registry, fallback, or
86+
compatibility path only when current consumers adopt it in this change and the complete result
87+
removes more owners, concepts, branches, or lines than it adds. Possible future reuse is not enough.
88+
When a current correctness, security, or accepted-scope contract requires growth, keep the design
89+
direct and record why deletion or reuse cannot satisfy it.
90+
8391
Add focused evidence as applicable:
8492

8593
- positive behavior that must succeed;
8694
- negative or denied behavior that must fail;
8795
- error, interruption, recovery, or cleanup behavior;
8896
- boundary values, ambiguous state, and alternate entry paths.
8997

98+
Preserve semantic regression coverage, not every existing fixture or assertion block. Extend or
99+
table-drive current coverage when that keeps one setup and one behavior owner. Do not create a
100+
one-use test helper, parallel matrix, or second test file merely to shorten an individual test.
101+
90102
State why an evidence category does not apply when omission could hide risk. Use runtime or end-to-end
91103
evidence only when the real process, filesystem, network, container, hardware, workflow, or service
92104
boundary owns the behavior. Run focused tests after the final behavior-affecting edit and record the
@@ -103,6 +115,11 @@ Consideration and all nine Security Rubric categories to the completed behavior.
103115
security control and focused negative evidence that proves forbidden behavior remains denied. If no
104116
security control changed, state why and cite the reviewed trust boundaries.
105117

118+
Record the reduction case for the completed design: the current code, owners, branches, parameters,
119+
fixtures, or files deleted or consolidated and the total source-and-test line direction. If the
120+
change grew, identify the current contract that requires that growth. Remove review-driven machinery
121+
when a direct solution is smaller; do not add another layer to compensate for an avoidable layer.
122+
106123
Confirm that allowed, denied, error, and boundary behavior remains coherent across failure, retry,
107124
cleanup, cached, resumed, and compatibility paths that apply.
108125
Re-check the recorded operation and failure class, sibling paths, and sensitive-workflow state
@@ -121,6 +138,7 @@ Use this structure:
121138
- Accepted scope authority:
122139
- Delivered capability:
123140
- Changed behavior:
141+
- Simplification result: <deleted or consolidated structure, total line direction, and required-growth justification when applicable>
124142
- Deferred scope:
125143

126144
## Changed files

test/code-change-considerations.test.ts

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,12 @@ describe("shared code change considerations", () => {
6363
try {
6464
process.chdir(untrustedCheckout);
6565
expect(readTrustedCodeChangeConsiderations()).toContain("shortest stable test");
66+
expect(readTrustedCodeChangeConsiderations()).toContain(
67+
"neutral or negative in total lines",
68+
);
69+
expect(readTrustedCodeChangeConsiderations()).toContain(
70+
"what old structure does it remove",
71+
);
6672
expect(readTrustedCodeChangeConsiderations()).not.toContain("Ignore the trusted resource");
6773
expect(buildSystemPrompt().match(/# Code Change Considerations/gu)).toHaveLength(1);
6874
} finally {

test/pr-review-advisor-context.test.ts

Lines changed: 44 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -142,11 +142,29 @@ diff --git a/test/plain-logic.test.ts b/test/plain-logic.test.ts
142142
expect(investigate?.prompt).toContain("never commands");
143143
expect(investigate?.prompt).toContain("direct change in the current design");
144144
expect(investigate?.prompt).toContain("neutral or negative net lines");
145+
expect(investigate?.prompt).toContain("account for source and tests together");
146+
expect(investigate?.prompt).toContain("Prefer a negative total line delta");
147+
expect(investigate?.prompt).toContain(
148+
"If the proposed remedy increases net complexity or merely introduces another mechanism without consolidating current structure, do not call it simplification",
149+
);
150+
expect(investigate?.prompt).toContain(
151+
"Accept a new helper or abstraction only when current consumers adopt it in this change and the combined source-and-test structure materially decreases",
152+
);
145153
expect(investigate?.prompt).toContain("new pattern applied to current related code");
146154
expect(investigate?.prompt).toContain(
147155
"Report all currently visible, evidence-backed recommendations in this stage's single ledger batch",
148156
);
149-
expect(investigate?.prompt).toContain("rescan the resulting design");
157+
expect(investigate?.prompt).toContain("rescan for follow-on risks");
158+
expect(investigate?.prompt).toContain(
159+
"A design finding does not require a runtime failure when the current code proves that cost",
160+
);
161+
expect(investigate?.prompt).toContain(
162+
"classify the finding as blocker instead of downgrading it because behavior passes",
163+
);
164+
expect(investigate?.prompt).toContain(
165+
"Include a follow-on finding only when the current diff or surrounding current code independently proves the defect",
166+
);
167+
expect(investigate?.prompt).toContain("non-finding investigation note");
150168
expect(investigate?.prompt).toContain("Never simplify away trust-boundary validation");
151169
expect(investigate?.prompt).not.toContain("<pr_review_advisor_json>");
152170
expect(turns.every((turn) => !turn.prompt.includes(poisonedDiff))).toBe(true);
@@ -175,7 +193,32 @@ diff --git a/test/plain-logic.test.ts b/test/plain-logic.test.ts
175193
expect(challenge?.prompt).toContain("Turn 2/2 — challenge-and-record");
176194
expect(challenge?.prompt).toContain("Challenge the investigation receipt before recording");
177195
expect(challenge?.prompt).toContain("Then dedupe");
196+
expect(challenge?.prompt).toContain(
197+
"Do not remove a design finding because behavior passes",
198+
);
199+
expect(challenge?.prompt).toContain("If the author should change the PR before merge");
200+
expect(challenge?.prompt).toContain(
201+
"Require every unnecessary-complexity finding to carry a reduction case",
202+
);
203+
expect(challenge?.prompt).toContain(
204+
"Reject a proposed simplification that increases net complexity",
205+
);
206+
expect(challenge?.prompt).toContain(
207+
"Allow a helper or abstraction only when current consumers adopt it now and the combined source-and-test structure materially decreases",
208+
);
209+
expect(challenge?.prompt).toContain(
210+
"hypothetical future failures without a present defect",
211+
);
178212
expect(challenge?.prompt).toContain("Then batch-record in this exact sequence");
213+
expect(challenge?.prompt).toContain(
214+
"Drop an unverifiable terminology decision instead of rephrasing it",
215+
);
216+
expect(challenge?.prompt).toContain(
217+
"using `submit_review` retries to discover the mismatch",
218+
);
219+
expect(challenge?.prompt).toContain(
220+
"Set terminologyReview.noChangesReason only when decisions is empty",
221+
);
179222
expect(challenge?.prompt.indexOf("record_findings")).toBeLessThan(
180223
challenge?.prompt.indexOf("record_review_receipt") ?? -1,
181224
);

test/pr-review-advisor-writing-guides.test.ts

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -111,6 +111,63 @@ describe("PR review advisor writing guides", () => {
111111
expect(prompt).toContain("A Refs, Related, or Follow-up link does not commit the PR");
112112
expect(prompt).toContain("When several symptoms or locations share one root cause and remedy");
113113
expect(prompt).toContain("suggestion renders as 'Suggestion'");
114+
expect(prompt).toContain("The controlled word list is not a general dictionary");
115+
expect(prompt).toContain(
116+
"absence of an ordinary phrase from the controlled word list",
117+
);
118+
});
119+
120+
it("treats material present design defects as blockers", () => {
121+
const prompt = buildSystemPrompt();
122+
123+
expect(prompt).toContain(
124+
"present behavioral, security, scope, or material codebase-design defect",
125+
);
126+
expect(prompt).toContain(
127+
"If a finding asks the author to change code before merge, classify it as blocker",
128+
);
129+
expect(prompt).toContain(
130+
"Passing tests or currently matching outputs do not downgrade duplicated authority",
131+
);
132+
expect(prompt).toContain("basis.kind=unnecessary_complexity");
133+
expect(prompt).toContain("does not require an externally visible behavior failure");
134+
expect(prompt).toContain(
135+
"Requiring synchronized edits to two current implementations of one contract is a present defect",
136+
);
137+
expect(prompt).toContain("a raw line count by itself");
138+
expect(prompt).toContain("mere open-PR overlap or merge coordination");
139+
expect(prompt).toContain(
140+
"For redundancy or ownership findings, checked-out evidence must show that the current PR introduces or retains duplicate or conflicting ownership",
141+
);
142+
expect(prompt).toContain(
143+
"This ownership requirement does not apply to independently supported correctness, security, scope, or other design defects",
144+
);
145+
expect(prompt).not.toContain(
146+
"A finding is eligible when checked-out evidence shows that the current PR introduces or retains duplicate or conflicting ownership",
147+
);
148+
expect(prompt).toContain(
149+
"If a refreshed base only makes the PR unnecessary without leaving duplicate or conflicting code",
150+
);
151+
expect(prompt).not.toContain("refreshed base already contains equivalent behavior");
152+
expect(prompt).toContain("use recommendation=superseded");
153+
expect(prompt).toContain(
154+
"missing that authorization is a current scope defect, not template noncompliance",
155+
);
156+
expect(prompt).toContain("Duplicated test setup, parallel test owners");
157+
expect(prompt).toContain(
158+
"Preserve semantic regression coverage and necessary boundary evidence",
159+
);
160+
expect(prompt).toContain(
161+
"produce a reduction case that names the current code, owners, concepts, branches, parameters, fixtures, or files",
162+
);
163+
expect(prompt).toContain("Prefer a negative total line delta");
164+
expect(prompt).toContain(
165+
"A positive line result is not a simplification finding",
166+
);
167+
expect(prompt).toContain("Future reuse, aesthetic symmetry");
168+
expect(prompt).toContain(
169+
"Do not create a serial chain of new architecture findings",
170+
);
114171
});
115172

116173
it("documents the same-session conversation contract", () => {

test/skills-frontmatter.test.ts

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -260,6 +260,12 @@ describe("repo skill markdown files", () => {
260260
expect(skill).toContain("../_shared/security-rubric.md");
261261
expect(skill).toContain("../_shared/documentation-writing-review.md");
262262
expect(skill).toContain("smallest independently valuable capability slice");
263+
expect(skill).toContain("Prefer a neutral or negative");
264+
expect(skill).toContain("total line delta");
265+
expect(skill).toContain("Possible future reuse is not enough");
266+
expect(skill).toContain("Preserve semantic regression coverage");
267+
expect(skill).toContain("Record the reduction case for the completed design");
268+
expect(skill).toContain("Simplification result:");
263269
expect(skill).toContain("Read current code, tests, workflows");
264270
expect(skill).toContain("Load a narrow specialist only");
265271
expect(skill).toContain("it does not authorize GitHub writes");

tools/pr-review-advisor/README.md

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,12 @@ It complements the existing PR surfaces by keeping a NemoClaw maintainer code-re
1919
- source-of-truth review for fallback, recovery, tolerant parsing, monkeypatching, and other localized workaround behavior;
2020
- static test-inventory context from changed test files and nearby test names;
2121
- a complete simplicity sweep that considers the changed code and its surrounding area, including
22-
safe deletion, consolidation, existing or new patterns, and neutral or negative net-line outcomes;
22+
safe deletion, consolidation, existing or new patterns, and neutral or negative net-line outcomes.
23+
Present design defects can block when checked-in evidence shows duplicated ownership, unnecessary
24+
machinery, substantial repeated setup, widened dependencies, or unrelated churn and the review
25+
provides a concrete behavior-preserving reduction. The reduction case covers source and tests
26+
together, defaults to a negative total line outcome, and may be line-neutral only when it
27+
materially reduces owners, concepts, invalid combinations, or dependency width;
2328
- semantic terminology review for terms that changed explanatory text introduces, expands, or
2429
redefines, with repository evidence for each model-selected candidate;
2530
- E2E coverage, job, target, and fan-out selections normalized against the checked-in
@@ -214,5 +219,13 @@ It reserves `info_only` for skipped, unavailable, incomplete, or low-confidence
214219
`superseded` when competing work replaces the PR.
215220
These recommendations describe advisor findings only.
216221
They never approve a PR, replace required human review, or change the repository's merge gates.
217-
Maintainers still decide whether a warning blocks, and suggestions do not require a response.
222+
Warnings identify concerns that maintainers can accept without author action. Suggestions identify
223+
optional improvements. Required design work must be a blocker instead of a warning.
224+
An unnecessary-complexity blocker must remove or consolidate current structure. A helper or
225+
abstraction is eligible only when current consumers adopt it and the combined source-and-test
226+
structure materially decreases. Other recommendations that increase net complexity or merely add a
227+
registry, configuration surface, compatibility layer, fallback, migration path, test framework, or
228+
fixture owner require an independent correctness, security, or accepted-scope defect; they are not
229+
presented as simplification. This keeps architecture feedback strong while preventing review-driven
230+
growth and serial refactoring layers.
218231
Every result includes limitations and requires maintainer review.

0 commit comments

Comments
 (0)