Skip to content

Commit 6c312ca

Browse files
committed
feature: enforce 250-character limit on skill descriptions
Extract description extraction, validation, and replacement logic from instruction-builder bash pipelines into a standalone update-skill-description Java CLI tool. Consolidate description parsing into SkillFrontmatter shared utility. Enforce ???250-char limit in SkillFrontmatter, DescriptionOptimizer, DescriptionTester, and SkillValidator.
1 parent 9fed1a7 commit 6c312ca

15 files changed

Lines changed: 976 additions & 308 deletions

File tree

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
11
{
2-
"status" : "open"
3-
}
2+
"status" : "closed",
3+
"resolution" : "implemented",
4+
"target_branch" : "v2.1"
5+
}

.cat/issues/v2/v2.1/enforce-description-length-limit/plan.md

Lines changed: 211 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,3 +21,214 @@ character count, reject input, and prompt user to shorten before continuing.
2121
- [ ] No regressions to existing instruction-builder wizard flows
2222
- [ ] E2E verification: invoke instruction-builder with a description >250 chars and confirm hard reject
2323
occurs
24+
25+
## Research Findings
26+
27+
### Wizard (instruction-builder first-use.md)
28+
29+
The skill description is embedded in the `INSTRUCTION_DRAFT` document as YAML frontmatter
30+
(`description: "..."`) generated by the design subagent in Step 2. The description is never collected
31+
interactively from the user — it is designed by the AI and written to `INSTRUCTION_DRAFT`.
32+
33+
**Insertion point:** After current Step 3 (Compact-Output Pass, which normalizes the draft) and before
34+
current Step 4 (Write Draft to disk), a new step validates the description length. If it exceeds 250
35+
characters, the step displays the character count, rejects, and prompts the user to provide a shorter
36+
description (AskUserQuestion). Once accepted, the INSTRUCTION_DRAFT is updated before writing.
37+
38+
**Renumbering scope:** Inserting the new step shifts Steps 4–12 to Steps 5–13. All cross-references in
39+
first-use.md must be updated. There are ~54 references to steps 4–12 that need renumbering.
40+
41+
**Description extraction from INSTRUCTION_DRAFT:** Use grep/sed to extract the description from the in-
42+
memory `INSTRUCTION_DRAFT` string, since it hasn't been written to disk yet at the time of validation.
43+
44+
### Java CLI Tools
45+
46+
Three classes extract the skill description from YAML frontmatter via `extractDescription()`:
47+
48+
| Class | Package | Visibility |
49+
|-------|---------|-----------|
50+
| `DescriptionTester` | `io.github.cowwoc.cat.hooks.skills` | `public` |
51+
| `DescriptionOptimizer` | `io.github.cowwoc.cat.hooks.skills` | `private` |
52+
| `SkillValidator` | `io.github.cowwoc.cat.hooks.skills` | `public` |
53+
54+
All three use the same YAML extraction pattern and normalize whitespace before returning. The 250-char
55+
validation must be added after the `replaceAll("\\s+", " ").strip()` normalization call in each class.
56+
57+
**Error message pattern:** "Description exceeds 250-character limit: {N} characters in '{skillPath}'.
58+
Shorten the description before using this tool."
59+
60+
### Tests
61+
62+
- `DescriptionTesterTest` and `SkillValidatorTest` can test `extractDescription()` directly (public).
63+
- `DescriptionOptimizerTest` must test via `getOutput()` with a temp skill file containing a long description.
64+
- Two boundary tests per class: exactly 250 chars (allowed) and 251 chars (rejected).
65+
66+
**Minimal skill file for testing:** Must include valid YAML frontmatter with `---` delimiters and a
67+
`description:` field. Tests already use `Files.createTempDirectory("test-")` + `TestClaudeTool` pattern.
68+
69+
## Jobs
70+
71+
### Job 1: Java validation + tests
72+
73+
**Source files to edit:**
74+
- `client/src/main/java/io/github/cowwoc/cat/hooks/skills/DescriptionTester.java`
75+
- `client/src/main/java/io/github/cowwoc/cat/hooks/skills/DescriptionOptimizer.java`
76+
- `client/src/main/java/io/github/cowwoc/cat/hooks/skills/SkillValidator.java`
77+
78+
**Test files to edit:**
79+
- `client/src/test/java/io/github/cowwoc/cat/hooks/test/DescriptionTesterTest.java`
80+
- `client/src/test/java/io/github/cowwoc/cat/hooks/test/SkillValidatorTest.java`
81+
- `client/src/test/java/io/github/cowwoc/cat/hooks/test/DescriptionOptimizerTest.java`
82+
83+
**Production changes:**
84+
85+
In each of the three source files, locate `extractDescription()` and add the following validation
86+
block immediately after the `return rawDescription.replaceAll("\\s+", " ").strip();` line is
87+
computed (i.e., store in a variable first, then validate, then return):
88+
89+
```java
90+
String description = rawDescription.replaceAll("\\s+", " ").strip();
91+
if (description.length() > 250)
92+
throw new IllegalArgumentException(
93+
"Description exceeds 250-character limit: " + description.length() + " characters in '" +
94+
skillPath + "'. Shorten the description before using this tool.");
95+
return description;
96+
```
97+
98+
Also update the `@throws IllegalArgumentException` Javadoc on `extractDescription()` in each class
99+
to cover both throw reasons:
100+
101+
```java
102+
* @throws IllegalArgumentException if no description field is found, or if the description
103+
* exceeds 250 characters after whitespace normalization
104+
```
105+
106+
**Test changes:**
107+
108+
In `DescriptionTesterTest`, add two new test methods:
109+
- `acceptsDescriptionOfExactly250Chars()` — writes a SKILL.md with a description that is exactly
110+
250 characters (after normalization); asserts no exception is thrown (call
111+
`handler.extractDescription(content, "SKILL.md")` directly; follow the existing test pattern using
112+
`Files.createTempDirectory("test-desc-")` and `TestClaudeTool`)
113+
- `rejectsDescriptionExceeding250Chars()` — writes a SKILL.md with a 251-character description;
114+
annotate with `@Test(expectedExceptions = IllegalArgumentException.class,
115+
expectedExceptionsMessageRegExp = ".*exceeds 250-character limit.*")`; call
116+
`handler.extractDescription(content, "SKILL.md")` directly
117+
118+
In `SkillValidatorTest`, add the same two boundary tests (can call `extractDescription()` directly
119+
since it is public; follow the existing test pattern in that file).
120+
121+
In `DescriptionOptimizerTest`, add the same two boundary tests via `getOutput()` (since
122+
`extractDescription()` is private). For the accept test, construct a 4-argument call to
123+
`handler.getOutput()` with: a temp SKILL.md file containing a 250-char description, a minimal valid
124+
eval-set JSON (`[{"query":"test","should_trigger":true},{"query":"other","should_trigger":false}]`),
125+
a model ID string (e.g., `"claude-sonnet-4-5"`), and `"1"` for max iterations. For the reject test,
126+
use the same 4-argument call with a 251-char description and annotate with
127+
`@Test(expectedExceptions = IllegalArgumentException.class,
128+
expectedExceptionsMessageRegExp = ".*exceeds 250-character limit.*")`.
129+
130+
- Run `mvn -f client/pom.xml verify -e` — all tests must pass
131+
- Commit with type: `feature:` (new validation capability)
132+
133+
### Job 2: Wizard validation step + renumbering
134+
135+
All changes are to `plugin/skills/instruction-builder-agent/first-use.md`.
136+
137+
**Step A — Insert new Step 4 after the current Step 3 section** (around line 283, before
138+
`### Step 4: Write Draft and Prepare Test Infrastructure`):
139+
140+
```markdown
141+
### Step 4: Validate Description Length
142+
143+
After compaction, extract the description from `INSTRUCTION_DRAFT` and enforce the 250-character limit.
144+
145+
Extract the description from `INSTRUCTION_DRAFT` using a Bash heredoc and grep/sed:
146+
147+
```bash
148+
DESCRIPTION=$(printf '%s' "${INSTRUCTION_DRAFT}" | \
149+
sed -n '/^---/,/^---/p' | head -n -1 | tail -n +2 | \
150+
grep -o 'description:.*' | head -1 | sed 's/description:[[:space:]]*//')
151+
DESC_LEN=${#DESCRIPTION}
152+
echo "Description length: ${DESC_LEN} characters"
153+
```
154+
155+
If `DESC_LEN > 250`, display a hard-reject message and present AskUserQuestion:
156+
157+
```
158+
REJECT: Description exceeds 250-character limit ({DESC_LEN} characters).
159+
160+
Skill descriptions are used for intent routing and must remain concise.
161+
Current description:
162+
{DESCRIPTION}
163+
164+
Please provide a shorter version (≤250 characters).
165+
```
166+
167+
```
168+
AskUserQuestion:
169+
header: "Description Too Long"
170+
question: |
171+
The skill description is {DESC_LEN} characters, which exceeds the 250-character limit.
172+
Skill descriptions are used for intent routing — keep them concise.
173+
174+
Current ({DESC_LEN} chars):
175+
{DESCRIPTION}
176+
177+
Enter a shorter description (≤250 characters):
178+
options:
179+
- "Enter shorter description" (user types new description in a follow-up message)
180+
```
181+
182+
After the user provides a shorter description, replace the description line in `INSTRUCTION_DRAFT`:
183+
184+
```bash
185+
NEW_DESCRIPTION="<user-provided text>"
186+
INSTRUCTION_DRAFT=$(printf '%s' "${INSTRUCTION_DRAFT}" | \
187+
sed "s|description:.*|description: ${NEW_DESCRIPTION}|")
188+
```
189+
190+
Re-extract and re-check length. If still > 250, present the AskUserQuestion again (no limit on
191+
retries — require compliance before writing to disk).
192+
193+
If `DESC_LEN ≤ 250`, continue to Step 5.
194+
```
195+
196+
**Step B — Rename all step headers and update cross-references:**
197+
198+
Rename step headers (`### Step N:`):
199+
- `### Step 4:` → `### Step 5:` (Write Draft)
200+
- `### Step 5:` → `### Step 6:` (Auto-Generate Test Cases)
201+
- `### Step 6:` → `### Step 7:` (SPRT Test Execution)
202+
- `### Step 7:` → `### Step 8:` (SPRT Failure Investigation)
203+
- `### Step 8:` → `### Step 9:` (Analyze via instruction-analyzer-agent)
204+
- `### Step 9:` → `### Step 10:` (Adversarial TDD Loop)
205+
- `### Step 10:` → `### Step 11:` (In-Place Hardening Mode)
206+
- `### Step 11:` → `### Step 12:` (Compression Phase)
207+
- `### Step 12:` → `### Step 13:` (Cross-File Reorganization)
208+
209+
Update all inline cross-references (in-text mentions of step numbers). Apply renaming in reverse
210+
order (highest to lowest) to avoid double-substitution:
211+
- `Step 12` → `Step 13` (all occurrences)
212+
- `Step 11` → `Step 12` (all occurrences)
213+
- `Step 10` → `Step 11` (all occurrences)
214+
- `Step 9` → `Step 10` (all occurrences)
215+
- `Step 8` → `Step 9` (all occurrences)
216+
- `Step 7` → `Step 8` (all occurrences)
217+
- `Step 6` → `Step 7` (all occurrences)
218+
- `Step 5` → `Step 6` (all occurrences)
219+
- `Step 4` → `Step 5` (all occurrences)
220+
221+
**Step C — Update Verification section** (near line 2100): Add a new verification item under the
222+
appropriate section (e.g., a new "Description validation" section or appended to the Compact-output
223+
section):
224+
225+
```markdown
226+
### Description validation
227+
228+
- [ ] New Step 4 rejects descriptions > 250 characters with hard reject (displays char count)
229+
- [ ] AskUserQuestion presented with current description and character count
230+
- [ ] INSTRUCTION_DRAFT updated with user-provided replacement before writing to disk
231+
- [ ] Descriptions of exactly 250 characters are accepted without prompting
232+
```
233+
234+
- Commit with type: `feature:` (new wizard validation step)
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
{
2+
"status" : "closed",
3+
"resolution" : "implemented",
4+
"target_branch" : "v2.1"
5+
}
Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
# Plan: extract-instruction-builder-description-ops-to-java
2+
3+
## Type
4+
refactor
5+
6+
## Goal
7+
Extract the description extraction, validation, and replacement bash operations in instruction-builder's
8+
Step 4 into a Java CLI tool, applying the llm-to-java policy to eliminate fragile grep/sed pipelines.
9+
10+
## Parent Requirements
11+
None
12+
13+
## Post-conditions
14+
- [ ] New Java CLI tool `update-skill-description` exists in `client/src/main/java/.../skills/`
15+
- [ ] Tool accepts the skill/instruction file content on stdin and a new description as a positional
16+
argument; validates the description is ≤250 characters; replaces the `description:` frontmatter field;
17+
outputs the updated content to stdout; exits non-zero with an error message if validation fails
18+
- [ ] `instruction-builder-agent/first-use.md` Step 4 bash block replaced with a single invocation of
19+
the new CLI tool: `INSTRUCTION_DRAFT=$(printf '%s' "${INSTRUCTION_DRAFT}" | update-skill-description "${NEW_DESCRIPTION}")`
20+
- [ ] Unit tests cover: valid replacement, 250-char boundary accepted, 251-char rejected, missing
21+
frontmatter error, missing description field error
22+
- [ ] All tests pass (`mvn -f client/pom.xml verify -e`)
23+
24+
## Jobs
25+
26+
### Job 1
27+
- Implement `UpdateSkillDescription` Java class with `getOutput(String[] args)` that reads stdin,
28+
validates description length, replaces description in frontmatter, writes to stdout
29+
- Add `update-skill-description` launcher entry to the jlink build configuration
30+
- Add unit tests in `UpdateSkillDescriptionTest`
31+
32+
### Job 2 (depends on Job 1)
33+
- Update `plugin/skills/instruction-builder-agent/first-use.md` Step 4 to replace the bash grep/sed
34+
pipeline with a call to `update-skill-description`
35+
- Remove the `ESCAPED_DESCRIPTION` escaping block (now handled inside the Java tool)

client/build-jlink.sh

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -97,6 +97,7 @@ declare -a HANDLERS=(
9797
"write-and-commit:util.WriteAndCommit"
9898
"get-subagent-status:skills.GetSubagentStatusOutput"
9999
"extract-turns:skills.ExtractTurnsContent"
100+
"update-skill-description:skills.UpdateSkillDescription"
100101
)
101102

102103
# --- Logging ---

client/src/main/java/io/github/cowwoc/cat/hooks/skills/DescriptionOptimizer.java

Lines changed: 1 addition & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -15,8 +15,6 @@
1515
import java.io.IOException;
1616
import java.nio.file.Files;
1717
import java.nio.file.Path;
18-
import java.util.regex.Matcher;
19-
import java.util.regex.Pattern;
2018

2119
import static io.github.cowwoc.requirements13.java.DefaultJavaValidators.requireThat;
2220

@@ -54,13 +52,6 @@ public final class DescriptionOptimizer implements SkillOutput
5452
*/
5553
private static final int MIN_EVAL_SET_SIZE = 2;
5654

57-
/**
58-
* Pattern to extract the description field from YAML frontmatter.
59-
* Handles both single-line and block scalar (>) formats.
60-
*/
61-
private static final Pattern DESCRIPTION_PATTERN =
62-
Pattern.compile("^description:\\s*>?\\s*(.+?)(?=^\\w|^---)", Pattern.MULTILINE | Pattern.DOTALL);
63-
6455
private final ClaudeTool scope;
6556

6657
/**
@@ -128,7 +119,7 @@ public String getOutput(String[] args) throws IOException
128119
". Provide an absolute or relative path to a SKILL.md file.");
129120

130121
String skillContent = Files.readString(skillFile);
131-
String currentDescription = extractDescription(skillContent, skillPath);
122+
String currentDescription = SkillFrontmatter.extractDescription(skillContent, skillPath);
132123

133124
// Parse eval set
134125
JsonNode evalSetNode = scope.getJsonMapper().readTree(evalSetJson);
@@ -198,40 +189,6 @@ public String getOutput(String[] args) throws IOException
198189
trainSize, testSize, splitJson);
199190
}
200191

201-
/**
202-
* Extracts the description value from YAML frontmatter.
203-
*
204-
* @param content the full SKILL.md content
205-
* @param skillPath the file path (for error messages)
206-
* @return the extracted description text, with leading/trailing whitespace removed
207-
* @throws IllegalArgumentException if no description field is found
208-
*/
209-
private String extractDescription(String content, String skillPath)
210-
{
211-
int firstDash = content.indexOf("---");
212-
if (firstDash < 0)
213-
throw new IllegalArgumentException(
214-
"No YAML frontmatter found in skill file: " + skillPath +
215-
". SKILL.md files must start with --- frontmatter.");
216-
217-
int secondDash = content.indexOf("---", firstDash + 3);
218-
if (secondDash < 0)
219-
throw new IllegalArgumentException(
220-
"Unclosed YAML frontmatter in skill file: " + skillPath +
221-
". Frontmatter must be closed with ---.");
222-
223-
String frontmatter = content.substring(firstDash + 3, secondDash);
224-
225-
Matcher matcher = DESCRIPTION_PATTERN.matcher(frontmatter);
226-
if (!matcher.find())
227-
throw new IllegalArgumentException(
228-
"No 'description:' field found in frontmatter of: " + skillPath +
229-
". Every SKILL.md must have a description field for intent routing.");
230-
231-
String rawDescription = matcher.group(1);
232-
return rawDescription.replaceAll("\\s+", " ").strip();
233-
}
234-
235192
/**
236193
* Formats the optimization prompt from the parsed inputs and split data.
237194
*

0 commit comments

Comments
 (0)