Skip to content

Commit fb4f9a6

Browse files
qazbnm456claude
andcommitted
docs: reposition as a general RLM harness (not security-specific); unify examples
rlm-kit is domain-agnostic — anything dspy.RLM can do (deep research, an RSS-digest-to-webhook agent, structured extraction, detection authoring). The "security tasks" framing was an artifact of the author's first use, not the kit's scope. Generalize it everywhere; keep the real security PROPERTIES (sandbox-is-the-boundary, SSRF guard, MCP host-side, SECURITY.md) since those hold for any task that runs model-written code over untrusted input. - pyproject: description -> "harness for building tasks on DSPy RLMs"; drop the `security` keyword (add `harness`) and the `Topic :: Security` classifier (add `Application Frameworks`). - README/CLAUDE/CONTRIBUTING/__init__/task/fetch/sandbox: intros say "tasks", not "security tasks"; add a domain-agnostic positioning line; neutral quickstart. - examples: SecurityTriage/RateSeverity -> neutral Research/Summarize tasks. - Unify the two examples dirs: move rlm_kit/examples/harness_run.py to the top-level examples/ and delete the in-package rlm_kit/examples/. Examples no longer ship inside the importable wheel (conventional layout); the `examples/harness_run.py` references in task.py/README are now accurate. uv build: wheel ships no examples, Summary updated. pytest 156 passed; ruff clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 1a41045 commit fb4f9a6

11 files changed

Lines changed: 63 additions & 57 deletions

File tree

CLAUDE.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# rlm-kit — agent guide
22

33
`rlm-kit` is a small, reusable scaffold over `dspy.RLM` (Recursive Language Models)
4-
for building security tasks. A task is a *declaration* — a `RLMTask` subclass with
4+
for building tasks (of any kind). A task is a *declaration* — a `RLMTask` subclass with
55
a `signature`, `output_field`, optional `output_model`, `instructions`, and
66
`tools`; retry/validation, sandbox selection, budget caps, and observability are
77
inherited. See `README.md` for the full layout and usage.
@@ -124,7 +124,7 @@ One companion rule ships under `.claude/rules/`:
124124

125125
## Consumer-driven hardening
126126

127-
- This kit is driven by a real downstream consumer (a security task that builds on the
127+
- This kit is driven by a real downstream consumer (a task that builds on the
128128
scaffold via an editable path dep). That dogfooding is the design loop: when the consumer
129129
forces a workaround, log the **reusable** gap and fix it in the kit — do not special-case
130130
the consumer. Generic mechanics get promoted here via the base/wrap split (a generic base +

CONTRIBUTING.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# Contributing to rlm-kit
22

33
Thanks for helping improve `rlm-kit` — a small, reusable scaffold over `dspy.RLM`
4-
(Recursive Language Models) for building security tasks. This guide is the short
4+
(Recursive Language Models) for building tasks of any kind. This guide is the short
55
version; the deep design rules live in [`CLAUDE.md`](./CLAUDE.md) and the
66
extension contract in the README's [**Building a consumer**](./README.md#building-a-consumer).
77

README.md

Lines changed: 16 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# rlm-kit
22

3-
A clean, reusable scaffold for building **security tasks** on top of
3+
A clean, reusable harness for building **any task** on top of
44
[DSPy](https://dspy.ai)'s Recursive Language Model module (`dspy.RLM`).
55

66
RLMs ([Zhang & Khattab, MIT, arXiv:2512.24601](https://arxiv.org/abs/2512.24601))
@@ -10,6 +10,11 @@ first-party implementation (Khattab co-authored both DSPy and the RLM paper) —
1010
works with existing Signatures and is optimizer-compatible (GEPA/MIPRO). This kit
1111
distills the boilerplate around it into one small, opinionated layer.
1212

13+
**rlm-kit is domain-agnostic** — anything `dspy.RLM` can do fits: multi-hop "deep
14+
research", an RSS-digest agent that posts to a webhook, structured extraction,
15+
detection authoring, you name it. Security happens to be the author's own first
16+
use of it, but it isn't the kit's scope.
17+
1318
## Why this exists
1419

1520
Using `dspy.RLM` directly leaves you re-writing the same plumbing for every task:
@@ -21,19 +26,19 @@ from rlm_kit import RLMConfig, RLMTask, configure
2126
from rlm_kit.tools import make_schema_validator
2227
from pydantic import BaseModel
2328

24-
class Finding(BaseModel):
29+
class Article(BaseModel):
2530
title: str
26-
severity: str
31+
summary: str
2732

28-
class TriageDiff(RLMTask):
29-
signature = "diff: str -> finding: Finding"
30-
output_field = "finding"
31-
output_model = Finding
32-
instructions = "You are a secure-code reviewer."
33-
tools = [make_schema_validator(Finding)]
33+
class Summarize(RLMTask):
34+
signature = "document: str -> article: Article"
35+
output_field = "article"
36+
output_model = Article
37+
instructions = "Read the document and produce a title and a one-paragraph summary."
38+
tools = [make_schema_validator(Article)]
3439

3540
configure(RLMConfig.from_env())
36-
finding = TriageDiff().run(diff=big_patch_text) # validated Finding
41+
article = Summarize().run(document=long_text) # validated Article
3742
```
3843

3944
The retry loop, pydantic validation, sandbox selection, and budget caps are all
@@ -290,7 +295,7 @@ and `max_iterations` there so an offline reader reads the real per-run values, n
290295
## Building a consumer
291296

292297
`rlm-kit` is the ROLLOUT floor; a consumer is a thin declaration on top of it. `examples/harness_run.py`
293-
is a minimal worked example — a security-triage task that wires the sub-LM hook, skills, tracing, and
298+
is a minimal worked example — a task that wires the sub-LM hook, skills, tracing, and
294299
RL export together. Five steps:
295300

296301
1. **Declare the task.** Subclass `RLMTask`: a `signature`, `output_field`, an `output_model`
Lines changed: 13 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -32,22 +32,22 @@
3232
)
3333

3434

35-
class Finding(BaseModel):
36-
title: str = Field(..., description="Short finding title.")
37-
severity: str = Field(..., description="low|medium|high|critical")
35+
class Note(BaseModel):
36+
title: str = Field(..., description="Short note title.")
37+
takeaway: str = Field(..., description="the key takeaway, one line")
3838

3939

4040
def _non_empty(text: str):
4141
return None if text.strip() else "empty response"
4242

4343

44-
class SecurityTriage(RLMTask):
45-
signature = "evidence: str -> finding: Finding"
46-
output_field = "finding"
47-
output_model = Finding
44+
class Research(RLMTask):
45+
signature = "topic: str -> note: Note"
46+
output_field = "note"
47+
output_model = Note
4848
instructions = (
49-
"You are a security analyst. Use list_skills/read_skill to consult "
50-
"playbooks, then emit a Finding JSON."
49+
"You are a research assistant. Use list_skills/read_skill to consult "
50+
"reference notes, then emit a Note JSON."
5151
)
5252

5353
def __init__(self, skills_dir: str, **kw):
@@ -65,15 +65,15 @@ async def main() -> None:
6565
base_sub, validators=[_non_empty], postprocessors=[str.strip], name="local-sub"
6666
)
6767

68-
task = SecurityTriage(
68+
task = Research(
6969
skills_dir=os.getenv("SKILLS_DIR", "./skills"),
7070
sub_lm=intercepted_sub,
7171
)
7272

7373
trace_path = "./traces/run.jsonl"
74-
with TraceRecorder(trace_path, run_id="triage-001", meta={"task": "triage"}):
75-
finding = await task.arun(evidence="...redacted evidence blob...")
76-
print(finding.model_dump_json(indent=2))
74+
with TraceRecorder(trace_path, run_id="research-001", meta={"task": "research"}):
75+
note = await task.arun(topic="...the topic to research...")
76+
print(note.model_dump_json(indent=2))
7777

7878
# The same trace doubles as an Agentic-RL dataset source.
7979
runs = group_by_run(load_events(trace_path))

examples/mini_run.py

Lines changed: 16 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -27,18 +27,18 @@
2727
)
2828

2929

30-
class Severity(BaseModel):
31-
label: str = Field(..., description="one of: low, medium, high, critical")
32-
reason: str = Field(..., description="one short sentence")
30+
class Summary(BaseModel):
31+
title: str = Field(..., description="a short title")
32+
gist: str = Field(..., description="one short sentence")
3333

3434

35-
class RateSeverity(RLMTask):
36-
signature = "advisory: str -> severity: Severity"
37-
output_field = "severity"
38-
output_model = Severity
35+
class Summarize(RLMTask):
36+
signature = "document: str -> summary: Summary"
37+
output_field = "summary"
38+
output_model = Summary
3939
instructions = (
40-
"You are a vulnerability triage assistant. Read the advisory text and "
41-
"return a Severity JSON. Keep it short."
40+
"Read the document and return a Summary JSON (a title and a one-sentence gist). "
41+
"Keep it short."
4242
)
4343

4444

@@ -47,15 +47,16 @@ async def main() -> None:
4747
print(f"main={cfg.main_model} sub={cfg.sub_model} interpreter={cfg.interpreter}")
4848

4949
# Keep the loop tiny/cheap.
50-
task = RateSeverity(max_retries=2)
50+
task = Summarize(max_retries=2)
5151

5252
trace_path = "./traces/mini_run.jsonl"
53-
advisory = (
54-
"A remote unauthenticated attacker can send a crafted HTTP request to the "
55-
"admin endpoint and execute arbitrary OS commands as root."
53+
document = (
54+
"Recursive Language Models treat unbounded context as a variable in a sandboxed "
55+
"REPL and recursively call sub-models over it, instead of stuffing everything into "
56+
"one prompt."
5657
)
57-
with TraceRecorder(trace_path, run_id="mini-001", meta={"task": "rate_severity"}):
58-
result = await task.arun(advisory=advisory)
58+
with TraceRecorder(trace_path, run_id="mini-001", meta={"task": "summarize"}):
59+
result = await task.arun(document=document)
5960

6061
print("\n=== RESULT ===")
6162
print(result.model_dump_json(indent=2))

pyproject.toml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
[project]
22
name = "rlm-kit"
33
version = "0.2.0"
4-
description = "A clean, reusable scaffold for building security tasks on DSPy Recursive Language Models (RLM)."
4+
description = "A clean, reusable harness for building tasks on DSPy Recursive Language Models (RLMs)."
55
readme = "README.md"
66
requires-python = ">=3.11"
77
license = "MIT"
@@ -14,7 +14,7 @@ keywords = [
1414
"llm",
1515
"agent",
1616
"reinforcement-learning",
17-
"security",
17+
"harness",
1818
]
1919
classifiers = [
2020
"Development Status :: 4 - Beta",
@@ -24,8 +24,8 @@ classifiers = [
2424
"Programming Language :: Python :: 3.12",
2525
"Programming Language :: Python :: 3.13",
2626
"Operating System :: OS Independent",
27-
"Topic :: Security",
2827
"Topic :: Scientific/Engineering :: Artificial Intelligence",
28+
"Topic :: Software Development :: Libraries :: Application Frameworks",
2929
"Typing :: Typed",
3030
]
3131
dependencies = [

rlm_kit/__init__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
"""rlm-kit — a clean, reusable scaffold for security tasks on DSPy RLMs.
1+
"""rlm-kit — a clean, reusable harness for building tasks on DSPy RLMs.
22
33
Public surface::
44

rlm_kit/examples/__init__.py

Whitespace-only changes.

rlm_kit/sandbox.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
"""Sandbox / code-interpreter selection — the security boundary of the scaffold.
22
33
RLM works by letting the model write and execute Python in a REPL. When that
4-
REPL is fed half-trusted scraped content (exactly the security use case this
5-
scaffold targets), the interpreter choice *is* the attack surface.
4+
REPL is fed half-trusted scraped content (a common case the moment a task pulls
5+
from the web or any untrusted source), the interpreter choice *is* the attack surface.
66
77
Policy:
88

rlm_kit/task.py

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,13 @@
11
"""The ``RLMTask`` base class — the one abstraction this scaffold exists for.
22
3-
A security task is declared by subclassing ``RLMTask`` and filling four fields:
4-
5-
class TriageCommit(RLMTask):
6-
signature = "diff: str -> finding: CommitFinding"
7-
output_field = "finding"
8-
output_model = CommitFinding # a pydantic BaseModel
9-
instructions = "You are a secure-code reviewer..."
10-
tools = [make_schema_validator(CommitFinding)]
3+
A task is declared by subclassing ``RLMTask`` and filling four fields:
4+
5+
class Summarize(RLMTask):
6+
signature = "document: str -> article: Article"
7+
output_field = "article"
8+
output_model = Article # a pydantic BaseModel
9+
instructions = "Summarize the document into a title and a paragraph."
10+
tools = [make_schema_validator(Article)]
1111
1212
Everything else — building ``dspy.RLM``, choosing the sandbox, budget caps,
1313
retrying on validation failure, observability — is inherited. A consumer's

0 commit comments

Comments
 (0)