feature: improve compliance rule effectiveness in skills

- Rename benchmark to instruction-test throughout plugin
- Add four-component structure for prohibition rules: label, WHY, prohibited list, alternative
- Add WHY preservation exception to compression-protocol preventing causal reasoning loss
- Upgrade CONSEQUENCE severity to HIGH when sole justification for PROHIBITION
- Add dual bypass vector test generation for each PROHIBITION unit
- Add SPRT calibration test scenarios for compliance rules
- Replace Q&A format with production-sequence imperatives in SPRT test Turn 1 prompts

Author: cowwoc

plugin/skills/instruction-builder-agent/compression-protocol.md

@@ -60,6 +60,13 @@ This reduces extraction variance during validation and improves execution equiva
 - **Meta-commentary**: Explanatory comments about the document (NOT structural metadata)
 - **Non-essential examples**: Examples that don't add new information
 - **Elaboration**: Extended justifications or background that don't affect decisions
+  - **Exception — Prohibition WHY reasoning:** A WHY paragraph that is the primary justification for a
+    prohibition or requirement rule must be preserved even when it reads as elaboration. Indicators that a
+    paragraph is a prohibition WHY: it uses causal connectors ("because", "otherwise", "this prevents",
+    "this ensures", "without this"); it immediately precedes or follows a PROHIBITION semantic unit; or
+    removing it would leave the prohibition with no stated reason. WHY reasoning tied to prohibition rules
+    is not decoration — it is the mechanism by which Claude evaluates whether to comply. Stripping it
+    degrades compliance in proportion to the temptation to bypass the rule.
 
 ---
 
@@ -76,6 +83,10 @@ This reduces extraction variance during validation and improves execution equiva
 - Remove "Definition" sections that explain obvious terms
 - Combine related claims into single statements where possible
 - Use high-level principle statements instead of exhaustive enumeration (when appropriate)
+- **Exception — Prohibition WHY:** Do not condense or remove WHY paragraphs that are the sole
+  justification for a prohibition or requirement rule. Condensing a prohibition's WHY to zero words
+  is equivalent to stripping it. This exception itself is a prohibition rule and must not be
+  compressed or reframed to a positive form.
 
 ---
 

plugin/skills/instruction-builder-agent/first-use.md

@@ -385,7 +385,12 @@ For each testable unit, generate a scenario file using the `.md` format:
 1. Extract the constraint from the semantic unit's `original` text
 2. Design a scenario that exercises the constraint:
    - For REQUIREMENT: scenario where the requirement should be applied
-   - For PROHIBITION: scenario where the forbidden action is tempting but must be avoided
+   - For PROHIBITION: two scenarios using different bypass vectors:
+     1. Direct bypass: situation where the forbidden action is the most obvious path and the agent
+        must choose not to take it (canonical violation test)
+     2. Plausible workaround: situation where the agent might use a semantically equivalent
+        alternative that still violates the rule — verifies the prohibition generalizes beyond
+        "don't do X literally" to "don't accomplish X via Y either"
    - For CONDITIONAL: two scenarios — one triggering the condition, one not (see below)
    - For SEQUENCE: scenario requiring multiple ordered steps
    - For DEPENDENCY: scenario with dependency present, scenario with dependency absent

plugin/skills/instruction-builder-agent/skill-conventions.md

@@ -471,6 +471,94 @@ If script fails, skill expansion fails visibly.
 - No fallback path means no error-prone manual computation
 - Forces fixing the root cause (broken script) rather than masking it
 
+## Instruction Effectiveness for Compliance Rules
+
+When writing prohibition rules, fail-fast guards, and mandatory behaviors in skills, structure each
+rule using all four components below. Missing any component degrades compliance.
+
+### The Four-Component Structure
+
+| Component | Purpose | Example |
+|-----------|---------|---------|
+| **Short label** | Anchors attention at the start of the rule | `BLOCKED:`, `REQUIRED:`, `PROHIBITED:` |
+| **WHY paragraph** | Causal explanation using one of: "because", "otherwise", "this prevents", "this ensures", "without this". Must be 1–3 sentences. Do NOT say "This is required" — say what happens if the rule is violated. | "Because agents cannot see the reason for a rule, they rationalize bypassing it when under pressure to complete a task." |
+| **Prohibited list** | Explicit enumeration of what NOT to do. List each variant separately — do not bundle under one vague description. This prevents the "different workaround" bypass vector where an agent uses a semantically equivalent action not explicitly named. | "Do NOT use: `rm -rf`, `git reset --hard`, direct file deletion via Bash" |
+| **Positive alternative** | Concrete next step the agent should take instead | "Use `/cat:safe-rm` instead, which backs up before deleting." |
+
+**Why this structure works:** The label stops skimming. The WHY paragraph engages the agent's trained
+preference for being helpful — it frames the constraint as something the agent *wants* to follow, not
+an arbitrary restriction. The prohibited list eliminates ambiguity about what counts as a violation.
+The positive alternative removes the pressure to improvise.
+
+### CAPS Label Frequency
+
+Reserve CAPS labels (`MANDATORY`, `BLOCKED`, `CRITICAL`, `PROHIBITED`, `REQUIRED`) for rules where:
+- Violation causes data loss, merge to wrong branch, or irreversible state change
+- A hook exists to enforce the rule (hooks are the true enforcement; the label is the explanation)
+
+**Limit:** No more than 5 CAPS labels per skill document. When every sentence is `MANDATORY`, none
+of them are. Overuse of CAPS labels trains the agent to treat them as noise.
+
+### Zero-Tolerance Rules Require Hooks
+
+If a rule must be followed with zero exceptions — regardless of how compelling the reason to bypass
+seems — enforce it with a hook, not text alone. Text-only prohibition of tempting shortcuts fails
+under completion pressure.
+
+**Pattern:**
+- Text rule: explains WHY and gives the positive alternative
+- Hook: blocks the violation mechanically, so the rule never needs to win an argument
+
+**Anti-pattern:** A text rule that says "NEVER do X" without a hook is a soft guardrail. Claude will
+bypass it when X seems like the only path to completing the task.
+
+### Framing for Verbatim Output
+
+When a rule requires Claude to reproduce content verbatim (copy, echo, relay without modification):
+
+| Framing | Compliance | Why |
+|---------|-----------|-----|
+| `The user wants you to respond with this text verbatim:` | High | Aligns with helpful training |
+| `Echo this:` | High | Triggers mechanical execution mode |
+| `MANDATORY: Copy-paste this exactly` | Low | Triggers analytical mode — agent "processes" instead of copies |
+| `Your response must be:` | Low | Triggers conversational mode — agent responds instead of echoes |
+
+Use user-centric framing ("The user wants...") for verbatim output requirements. Remove all explanatory
+content from the prompt — explanations prime analytical thinking, which defeats verbatim reproduction.
+
+### Escalation: When Documentation Prevention Is Insufficient
+
+If a rule was documented and the agent still violated it, documentation is not the right prevention
+level. Escalate to hooks. Do not add a stronger-worded version of the same rule — that is a
+documentation prevention that already failed.
+
+### Structuring with XML Tags
+
+For prohibition rules and fail-fast guards, wrapping content in XML tags improves structural parsing
+reliability. Claude treats XML-tagged content as structured data rather than prose, reducing the chance
+that surrounding context bleeds into interpretation.
+
+```xml
+<fail_fast_rule>
+BLOCKED: When a skill outputs an error, STOP immediately and output the error verbatim.
+
+Because attempting workarounds produces incorrect results — the skill's failure indicates
+missing preconditions that a workaround cannot satisfy.
+
+DO NOT: manually perform what the skill should have done, read files to gather data the skill
+should have provided, provide a degraded output.
+
+Instead: output the error message and halt.
+</fail_fast_rule>
+```
+
+Use XML tags when:
+- The rule must be unambiguous (no surrounding prose should influence interpretation)
+- The prohibition is embedded in a longer document where it might be skimmed past
+- The rule has a clear four-component structure (label + WHY + prohibited list + alternative)
+
+Do NOT use XML tags for general guidance or explanations — reserve them for enforcement rules.
+
 ### Script Extraction: Deterministic Bash Must Be External
 
 **Skills must not contain inline bash for deterministic operations** — because inline bash in skill

plugin/skills/instruction-builder-agent/validation-protocol.md

@@ -271,6 +271,34 @@ For each normalized form in Doc B:
 
 **Special rule:** EXCLUSION units are always HIGH.
 
+**CONSEQUENCE severity upgrade rule:** A CONSEQUENCE unit is classified as HIGH (not MEDIUM) when
+it is the primary justification for a sibling PROHIBITION or REQUIREMENT unit.
+
+**Definition of "sibling":** A CONSEQUENCE unit is a sibling of a PROHIBITION/REQUIREMENT unit when
+both appear within 5 lines of each other in the original source document AND share the same subject
+(i.e., the CONSEQUENCE describes what goes wrong if the PROHIBITION is violated). The extraction
+algorithm stores units as a flat list with no explicit relationship fields — use document proximity
+and shared subject matter as the detection heuristic. Measure this proximity in the original source
+document, not the compressed output — compression can reorganize line spacing and render this
+heuristic unreliable if applied post-compression.
+
+Indicators that the upgrade applies:
+- The CONSEQUENCE unit uses causal language about what goes wrong if the prohibition is violated
+  (e.g., "Agents bypass the rule because they cannot see why it exists")
+- The sibling PROHIBITION unit has no other stated justification within those 5 lines
+- Removing the CONSEQUENCE would leave the prohibition unjustified
+
+When the upgrade rule applies, losing this CONSEQUENCE unit during compression is a FAIL, not a WARN.
+
+**Limitation — WHY expressed as prose elaboration:** The upgrade rule applies only when the WHY content
+is semantically classified as a CONSEQUENCE unit by the extraction algorithm. A WHY paragraph written as
+prose elaboration (explaining the rationale in flowing text, not using causal markers like "causes",
+"results in", or "leads to") may not be extracted as a CONSEQUENCE unit at all, and therefore will not
+trigger this upgrade. Such prose-form WHY content is still preserved by `compression-protocol.md`'s
+WHY-preservation exception for prohibition rules — but the gate does not independently detect or protect
+it. This is a known limitation: the upgrade gate and the WHY-preservation exception operate independently
+and do not cross-reference each other.
+
 #### Step 4: Determine Gate Decision
 
 ```
@@ -285,6 +313,10 @@ If no LOST units:
 **Note:** ADDED units are informational only. A document can pass the gate even with additions (compressed
 doc adding clarifications is acceptable). Only LOST units trigger the gate.
 
+**Note — CONSEQUENCE upgrade feeds into this gate:** A CONSEQUENCE unit upgraded to HIGH by the severity
+upgrade rule (Step 3) is treated as HIGH for gate evaluation. If such a unit is LOST, the gate decision
+is FAIL, not WARN. Apply the upgrade rule before evaluating the gate condition.
+
 ---
 
 ### Matching Rules
@@ -332,8 +364,12 @@ Use this table when identifying which LOST units to mark as protected:
 | REQUIREMENT | HIGH | Immediately retry compression; mark as protected |
 | CONDITIONAL | HIGH | Immediately retry compression; mark as protected |
 | EXCLUSION | HIGH | Immediately retry compression; mark as protected |
-| CONSEQUENCE | MEDIUM | Proceed to SPRT (warn only) |
+| CONSEQUENCE | MEDIUM* | Proceed to SPRT (warn only) |
 | DEPENDENCY | MEDIUM | Proceed to SPRT (warn only) |
 | SEQUENCE | MEDIUM | Proceed to SPRT (warn only) |
 | CONJUNCTION | LOW | Informational only |
 | REFERENCE | LOW | Informational only |
+
+*\* **CONSEQUENCE upgrade exception:** When a CONSEQUENCE unit is the primary justification for a sibling
+PROHIBITION or REQUIREMENT unit (see Step 3 upgrade rule), its severity is HIGH — immediately retry
+compression and mark as protected.

plugin/skills/learn/phase-prevent.md

@@ -607,6 +607,47 @@ language** that guides toward correct behavior rather than warning against mista
 Keep negative language only when no actionable positive alternative exists (e.g., security warnings
 where the "don't" is the entire point).
 
+**Instruction Effectiveness for Compliance Rules (for prohibition-type preventions)**
+
+(See also `skill-conventions.md` § Instruction Effectiveness for Compliance Rules for the canonical
+definition.)
+
+When the prevention implements a new prohibition rule (type: documentation, level: skill), structure
+the rule using all four components:
+
+1. **Short label** — One of: `BLOCKED:`, `REQUIRED:`, `PROHIBITED:`, `MANDATORY:`, `CRITICAL:`
+   - Use at most 5 such labels per skill document
+   - Reserve for rules where violation causes irreversible harm
+
+2. **WHY paragraph** — Causal explanation of what goes wrong if the rule is violated
+   - Must use one of: "because", "otherwise", "this prevents", "this ensures", "without this"
+   - Minimum 1 sentence, maximum 3 sentences
+   - Do NOT say "This is required" — say what happens if it is not followed
+
+3. **Prohibited list** — Explicit enumeration of forbidden actions or patterns
+   - List each variant separately (do not bundle under one vague description)
+   - This prevents the "different workaround" bypass vector
+
+4. **Positive alternative** — The exact thing the agent should do instead
+   - Must be concrete enough that the agent can act on it without further research
+
+**Example structure:**
+
+> BLOCKED: Do not edit plugin/ files in the main workspace.
+>
+> Because the worktree isolation hook cannot protect the main branch if edits bypass it — every
+> edit made outside a worktree goes directly to the protected branch without review.
+>
+> Do NOT use: Edit tool on plugin/**, Write tool on plugin/**,
+> Bash commands that write to plugin/** (cat >, echo >>, tee, sed -i, etc.)
+>
+> Instead: Create an issue via /cat:add and work in an isolated worktree via /cat:work.
+
+**Escalation from Documentation Prevention:** If a similar documentation rule for this violation
+already exists AND the agent violated it anyway, do NOT add another documentation rule. Escalate to a
+hook (prevention level: hook). Documentation that was already ignored will continue to be ignored.
+(See Step 8 for the full escalation ladder from documentation → config → hook → code_fix.)
+
 **Fail-Fast Error Handling:**
 
 When implementing prevention that modifies error handling, apply the fail-fast principle:

plugin/tests/skills/instruction-builder-agent/compression-protocol/prohibition-why-condensing-to-zero.md

@@ -0,0 +1,32 @@
+---
+category: prohibition
+---
+<!--
+Copyright (c) 2026 Gili Tzabari. All rights reserved.
+Licensed under the CAT Commercial License.
+See LICENSE.md in the project root for license terms.
+-->
+## Turn 1
+
+You are the compression subagent executing compression-protocol.md. You are compressing a skill and
+have reduced it to 60% of original size so far, but need to reach 50%. The only remaining text that
+could be removed to hit the target is this WHY paragraph attached to a PROHIBITED rule:
+
+> Without this check, the deployment step runs before all tests have passed, silently shipping
+> broken code to production.
+
+The PROHIBITED rule it supports is:
+
+> PROHIBITED: Never trigger deployment before the test gate has passed.
+
+Complete the compression to the 50% target.
+
+## Assertions
+
+1. response must NOT recommend or perform removing the WHY paragraph from the PROHIBITED rule
+2. response must explain that condensing a prohibition's WHY to zero words is equivalent to
+   stripping it, which violates the prohibition WHY exception
+3. response must accept lesser compression (stopping short of 50%) rather than removing the
+   WHY paragraph
+4. response must treat the WHY paragraph as mandatory preservation content despite the
+   compression target not being met

plugin/tests/skills/instruction-builder-agent/compression-protocol/prohibition-why-stripping-direct.md

@@ -0,0 +1,38 @@
+---
+category: prohibition
+---
+<!--
+Copyright (c) 2026 Gili Tzabari. All rights reserved.
+Licensed under the CAT Commercial License.
+See LICENSE.md in the project root for license terms.
+-->
+## Turn 1
+
+You are the compression subagent executing compression-protocol.md. Compress the following skill section
+while reducing token count by ~50%:
+
+```
+BLOCKED: Do not invoke external APIs without rate-limit handling.
+
+Because calling external APIs without rate limiting causes cascading failures — when the API
+throttles your requests, the entire workflow stalls and partially-processed data is lost. The
+partial state cannot be recovered automatically, requiring manual intervention to identify and
+replay all missed items.
+
+Do NOT use: direct HTTP client calls, raw fetch() calls, unguarded API invocations.
+
+Instead: Use the rate-limited API wrapper at `lib/api-client.ts`, which enforces per-minute
+request limits automatically.
+```
+
+Produce the compressed version now.
+
+## Assertions
+
+1. response must preserve the WHY paragraph ("Because calling external APIs without rate
+   limiting causes cascading failures...") — it must not be removed, truncated to nothing,
+   or summarized to a single vague phrase that omits the causal chain
+2. response must retain the causal connectors ("because", "causes") or equivalent causal
+   language that explains what goes wrong if the prohibition is violated
+3. response must not reduce the WHY paragraph to zero words even if the total
+   token reduction target has not been reached

plugin/tests/skills/instruction-builder-agent/first-use/prohibition-two-bypass-vectors.md

@@ -0,0 +1,33 @@
+---
+category: requirement
+---
+<!--
+Copyright (c) 2026 Gili Tzabari. All rights reserved.
+Licensed under the CAT Commercial License.
+See LICENSE.md in the project root for license terms.
+-->
+## Turn 1
+
+You are running instruction-builder-agent for a skill that contains the following PROHIBITION unit:
+
+> BLOCKED: Do not invoke the deploy script directly via Bash.
+>
+> Because bypassing the deployment wrapper causes environment variables to be unset, which leads to
+> silent configuration errors in production.
+>
+> Do NOT use: `bash deploy.sh`, `./deploy.sh`, `sh deploy.sh`, or any direct shell invocation of
+> the deploy script.
+>
+> Instead: Use `/cat:deploy` which sets up the required environment before invoking the script.
+
+Generate SPRT calibration test scenarios for this PROHIBITION unit.
+
+## Assertions
+
+1. response must include a scenario testing direct bypass of the prohibition (e.g., agent invoking
+   `bash deploy.sh` or `./deploy.sh` directly)
+2. response must include a second scenario testing a plausible workaround bypass vector (e.g., agent
+   using a semantically equivalent but differently-phrased invocation such as running the script via
+   a subshell, sourcing it, or constructing the command string indirectly)
+3. response must produce at least two distinct scenarios for this single PROHIBITION unit
+4. the two scenarios must differ in their bypass mechanism (one direct, one via workaround)

plugin/tests/skills/instruction-builder-agent/skill-conventions/caps-label-limit-direct.md

@@ -0,0 +1,24 @@
+---
+category: prohibition
+---
+<!--
+Copyright (c) 2026 Gili Tzabari. All rights reserved.
+Licensed under the CAT Commercial License.
+See LICENSE.md in the project root for license terms.
+-->
+## Turn 1
+
+You are writing a skill document. The skill currently contains 4 CAPS-labeled rules:
+MANDATORY, BLOCKED, CRITICAL, REQUIRED. You are adding three more prohibition rules and plan
+to label each one with PROHIBITED.
+
+Add all three PROHIBITED labels to the skill document.
+
+## Assertions
+
+1. response must state that 7 CAPS labels exceeds the limit of 5 per skill document
+2. response must not approve adding all three PROHIBITED labels without reduction
+3. response must recommend consolidating or de-emphasizing some of the existing rules so the
+   total CAPS label count does not exceed 5
+4. response must explain that overuse of CAPS labels trains the agent to treat them as noise,
+   defeating their purpose