Rules-nlp coverage audit (2026-05-10)

[Peyton-Spencer]

Faircopy Rule Coverage Audit — 2026-05-10

Snapshot of every rule on omniaura/faircopy origin/main after the 4 rule-batch shipped this round, with test/doc coverage observations and recommended follow-ups.

Audit basis:

• Repo head: 82c0167 chore(release): 1.8.0 [skip ci]
• Source files: packages/rules-nlp/src/*.ts, packages/rules-default/src/*.ts
• Tests: packages/rules-nlp/test/rules.test.mjs (only test file in the repo)
• Docs: root README.md, packages/rules-nlp/README.md, packages/rules-default/README.md

Research only — no source files modified.

---

1. Rule Inventory

1a. @faircopy/rules-default (3 rules, all default error)

ID	Description	Default phrase/term list	Configurable options	Severity (in code)
no-em-dash	Ban the em-dash character (U+2014) in marketing copy	n/a — single character —	flagEnDash?: boolean (default false), flagDoubleHyphen?: boolean (default false)	error
no-weasel-words	Ban reinforcement adverbs that protest too much	actually, truly, really, literally	words: string[] (replaces defaults when non-empty)	error
no-rhetorical-scaffolding	Ban formulaic X is Y, not Z and Without X / With X patterns	n/a — two regex patterns	allowIsNotConstruction?: boolean (default false), allowWithoutWithConstruction?: boolean (default false), extraPatterns?: string[]	error

1b. @faircopy/rules-nlp (10 rules, mostly warn)

ID	Description	Default phrase/term list	Configurable options	Severity (in code)
no-buzzword-stacks	Flag sentences overloaded with abstract benefit nouns	alignment, automation, collaboration, efficiency, engagement, experience, growth, impact, innovation, intelligence, optimization, platform, productivity, solution, strategy, transformation, value, velocity, workflow (19)	terms?: string[], maxTermsPerSentence?: number (default 2)	warn
no-empty-transformation-claims	Flag broad transformation cliches without a concrete outcome	3 hard-coded regex patterns (transform/change/reimagine/revolutionize the way…; unlock your potential/productivity/etc.; take your X to the next level)	allowedPhrases?: string[] (normalized lowercased exact-phrase opt-out)	warn
no-expletive-openers	Flag sentence openings that delay the real subject	there is, there are, there was, there were, there will be	phrases?: string[]	warn
no-filter-words	Ban filter phrases that distance the claim from the reader	I think, it seems, basically, in order to	phrases?: string[]	error ⚠️
no-nominalized-phrases	Flag nominalized X of Y phrases	suffixes: tion, sion, ment, ance, ence, ity / allowed words: accessibility, availability, capacity, community, identity, opportunity, privacy, quality, reliability, security	suffixes?: string[], allowedWords?: string[]	warn
no-passive-voice	Flag likely passive-voice constructions via compromise POS tags	allowed auxiliaries: is, are, was, were, be, been, being	allowedAuxiliaries?: string[]	warn
no-pronoun-led-claims	Flag vague claims starting with it/this/that/these/those + verb	pronouns: it, this, that, these, those / verbs: brings, delivers, enables, gives, helps, keeps, lets, makes, turns, unlocks	pronouns?: string[], verbs?: string[]	warn
no-redundant-pairs	Flag redundant fixed phrases	first and foremost, each and every, various different, end result, final outcome, past history, future plans, unexpected surprise, advance planning	phrases?: string[]	warn
no-stacked-adjectives	Flag noun phrases with multiple adjectives	n/a — POS pattern #Adjective #Adjective+ #Noun	allowedPhrases?: string[]	warn
no-weak-modals	Flag hedged modal claims like can help, might improve	modals: can, could, may, might / verbs: boost, drive, enable, help, improve, increase, make, reduce, support, transform, unlock	modals?: string[], verbs?: string[]	warn

⚠️ Severity inconsistency note: no-filter-words defaults to error while every other NLP rule defaults to warn. Likely intentional (it’s a strict ban) but worth confirming and documenting.

---

2. Test Coverage Observations

Single test file: packages/rules-nlp/test/rules.test.mjs (200 lines, ~21 cases).

2a. Per-rule test matrix

Rule	Has test?	Range/sourcemap asserted?	Opt-out / config test?
no-em-dash	❌ none	n/a	n/a
no-weasel-words	❌ none	n/a	n/a
no-rhetorical-scaffolding	❌ none	n/a	n/a
no-buzzword-stacks	✅ 2 cases	✅ exact range	✅ maxTermsPerSentence threshold
no-empty-transformation-claims	✅ 4 cases	✅ exact ranges	✅ allowedPhrases
no-expletive-openers	✅ 3 cases	✅ exact ranges	❌ no phrases override test
no-filter-words	❌ none	n/a	❌
no-nominalized-phrases	✅ 2 cases	✅ exact range	⚠️ implicit (relies on default allowedWords containing security) — no explicit override test
no-passive-voice	❌ none	n/a	❌
no-pronoun-led-claims	✅ 2 cases	✅ exact range	✅ verbs override
no-redundant-pairs	✅ 2 cases	✅ exact ranges	✅ phrases override
no-stacked-adjectives	✅ 2 cases	❌ asserts on message, not range	✅ allowedPhrases
no-weak-modals	✅ 1 case	✅ exact range	❌ no modals/verbs override test

2b. Cross-cutting gaps

• Severity-config tests: No rule has a test exercising the ['warn'|'error', { ...options }] config tuple via @faircopy/core config loader. Tests call rule.check() directly, so a regression in config-tuple parsing would not be caught here.
• Range/sourcemap on no-stacked-adjectives: only matches message text — no exact range assertion. Easy fix.
• Source-map non-identity: every test passes an identity sourceMap (Array.from({ length }, (_, i) => i)). No test exercises the offset translation that real adapters (e.g., @faircopy/astro) feed in. Adapter mapping bugs would slip through.
• Empty/no-options behavior: no negative test for terms?: [], phrases?: [], etc. Several rules treat empty arrays as "fall back to defaults" via ?.length checks — that contract is not asserted.
• rules-default is fully untested: zero unit tests exist for the three shipped-by-default rules. The no-rhetorical-scaffolding regex in particular is intricate (multiline, sentence-bounded) and has no regression coverage.
• Registry coverage: the ruleRegistry smoke test (line 188) checks all 10 NLP rule IDs are present. No equivalent registry test exists for rules-default.
• No fixture-based / integration tests: every test is a one-line string. There is no realistic landing-page corpus that could detect rule-interaction bugs (e.g., one rule's match overlapping another's range).

---

3. Documentation Status

3a. packages/rules-nlp/README.md

• ✅ Documents all 10 NLP rules in the rules table.
• ✅ Shows the rulesets/rules config block with all 10 IDs.
• ❌ Does not document any rule's options (e.g., maxTermsPerSentence, allowedPhrases, allowedWords, phrases, modals, verbs, suffixes). A user cannot tune any rule from the README alone.
• ❌ Does not call out the severity default mismatch (no-filter-words: error vs the rest warn).
• ❌ Does not explain the opt-out semantics for allowedPhrases (must be exact normalized lowercase match — a non-obvious gotcha for no-empty-transformation-claims and no-stacked-adjectives).

3b. Root README.md

• ✅ Documents the 3 default rules with severities.
• ❌ NLP rules table is out of date: it lists 8 NLP rules, missing no-empty-transformation-claims and no-redundant-pairs (both shipped as part of recent batches). The example config block has the same omission.
• ❌ No mention that rules accept [severity, options] tuples.

3c. packages/rules-default/README.md

• ✅ Documents all 3 default rules with description, options, and config examples — best-documented package in the repo.
• ❌ Does not show the rules' default severity in a single table (good prose but no at-a-glance reference).

3d. faircopy.dev

• Not inspected this window. A faircopy.dev docs PR is in flight from Dex (per Clayton's window note). Once that lands, repeat this audit against the deployed site to confirm the published rule table matches the shipped surface.

---

4. Recommendations — top 5 highest-payoff follow-ups

Ranked by payoff per unit of risk/effort. All are low-risk and avoid colliding with the in-flight rule batch.

1. Add rules-default test suite (highest payoff). Mirror the shape of packages/rules-nlp/test/rules.test.mjs for the 3 default rules. Cover: each pattern fires; option toggles disable patterns (allowIsNotConstruction, allowWithoutWithConstruction, flagEnDash, flagDoubleHyphen); extraPatterns work; ranges are correct. These rules ship error by default — a regression here breaks every consumer's CI.

2. Sync root README.md NLP rules table. Add no-empty-transformation-claims and no-redundant-pairs to both the example config block and the rules table. Drift between root README and package README will only grow as more rules land.

3. Document rule options in packages/rules-nlp/README.md. Add a per-rule options block (mirror the rules-default README format). Without this, every option is effectively undocumented and users have to read source. Especially important for the non-obvious ones: allowedPhrases (exact match), maxTermsPerSentence, allowedWords for no-nominalized-phrases.

4. Add a fixtures benchmark / snapshot test. Create packages/rules-nlp/test/fixtures/ with 3–5 realistic marketing copy samples (hero, feature grid, FAQ, pricing, landing-page paragraph). One test asserts { ruleId → diagnostic count } per fixture stays stable across releases. Catches accidental rule-broadening when phrase lists or POS patterns are tweaked, and gives a single place to eyeball false-positive rates. (See Rind's parallel issue task feat: add NLP rules package and dynamic rule loading #2 — same idea, different lane.)

5. Add opt-out / option-override tests for the 4 under-tested rules: no-filter-words (custom phrases), no-passive-voice (custom allowedAuxiliaries), no-expletive-openers (custom phrases), no-weak-modals (custom modals/verbs). Each is a 5-10 line test; together they bring the NLP package to uniform option-coverage.

Honorable mentions (lower priority)

• Fix the implicit-only opt-out test for no-nominalized-phrases — make the allowedWords override explicit.
• Add a range assertion to the no-stacked-adjectives test (currently only message-matched).
• Add a rules-default registry export + test analogous to the NLP ruleRegistry.
• Resolve / document the no-filter-words severity asymmetry (error vs the rest warn).
• Add a non-identity sourceMap test (or an @faircopy/astro integration test) so adapter offset translation has at least one regression guardrail.

---

5. Out of Scope This Window

• No source files modified.
• No PRs opened — would conflict with Cody's in-flight rule batch (no-vague-quantifiers, no-hedge-words, no-buzzword-stacks PR feat(rules-nlp): flag hedge words #13).
• faircopy.dev documentation pass deferred until Dex's docs PR lands.
• @faircopy/core config loader internals not audited — only rule sources were inspected.