Skip to content

Commit 0a8b000

Browse files
committed
docs(release): prepare v0.2.0 release candidate
1 parent c1ea31f commit 0a8b000

22 files changed

Lines changed: 194 additions & 54 deletions

.env.example

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -14,10 +14,10 @@ GEMINI_LOCATION=global # or us-central1 / europe-west4
1414
# Needed for TIGER-Lab/EditReward-Bench. Set this OR run `huggingface-cli login`.
1515
# HF_TOKEN=hf_xxx
1616

17-
# ---- Optional: override the score-guidelines path ----
18-
# Defaults to <repo>/score-guidelines/. Only set this if you've moved the
19-
# templates or are running the package from an unusual install layout.
20-
# REWARDHARNESS_TEMPLATES_DIR=/absolute/path/to/score-guidelines
17+
# ---- Optional: override packaged score guidelines ----
18+
# Defaults to rewardharness/resources/score_guidelines inside the installed
19+
# package. Set this only for a deliberate custom template bundle.
20+
# REWARDHARNESS_TEMPLATES_DIR=/absolute/path/to/score_guidelines
2121

2222
# ---- Optional: swap in a different Sub-Agent VLM ----
2323
# The OpenAI-compatible model id sent to vLLM /v1/chat/completions.

.github/ISSUE_TEMPLATE/bug_report.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ A one-line description of what went wrong.
1111
## Reproduction
1212
```bash
1313
# Exact command(s). Include `make check` output if relevant.
14+
# Prefer the v0.2 CLI form, for example: rewardharness inspect
1415
```
1516

1617
## Expected vs. actual
@@ -22,6 +23,7 @@ A one-line description of what went wrong.
2223
- Python: `python --version`
2324
- Key package versions: `pip show vllm google-genai openai | grep -E '^(Name|Version)'`
2425
- Output of `make check` (or `python scripts/check_env.py`):
26+
- Import style used: `rewardharness.*` or deprecated `src.*` compatibility layer
2527

2628
## Logs / stack trace
2729
<details>

.github/ISSUE_TEMPLATE/config.yml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,8 @@
11
blank_issues_enabled: false
22
contact_links:
3+
- name: v0.2 migration guide
4+
url: https://github.com/TIGER-AI-Lab/RewardHarness/blob/main/README.md
5+
about: Public package, CLI, Library schema, and compatibility-layer guidance.
36
- name: Paper
47
url: https://arxiv.org/abs/2605.08703
58
about: Read the paper before filing a question about results or methodology.

.github/ISSUE_TEMPLATE/feature_request.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,3 +16,7 @@ What would the smallest version of the fix look like? File paths + behavior, not
1616

1717
## Out of scope
1818
(optional) Things you are NOT proposing — helps reviewers stay focused.
19+
20+
## Compatibility
21+
State whether the proposal changes the public `rewardharness.*` API, Library
22+
schema, checkpoint schema, or CLI output.

.github/dependabot.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
version: 2
22

33
updates:
4-
# Python dependencies (core + vLLM extra)
4+
# Python dependencies (runtime, development/release tooling, and vLLM extra)
55
- package-ecosystem: pip
66
directory: "/"
77
schedule:

.github/workflows/release.yml

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
name: Release
2+
3+
on:
4+
push:
5+
tags: ["v*"]
6+
7+
permissions:
8+
contents: read
9+
10+
jobs:
11+
build:
12+
runs-on: ubuntu-latest
13+
steps:
14+
- uses: actions/checkout@v4
15+
- uses: actions/setup-python@v5
16+
with:
17+
python-version: "3.10"
18+
- run: python -m pip install -r requirements-dev.txt
19+
- name: Verify tag and package version
20+
run: |
21+
package_version=$(python -c 'import rewardharness; print(rewardharness.__version__)')
22+
tag_version=${GITHUB_REF_NAME#v}
23+
test "${tag_version//-/}" = "$package_version"
24+
- run: make release-check
25+
- uses: actions/upload-artifact@v4
26+
with:
27+
name: distributions
28+
path: dist/
29+
if-no-files-found: error
30+
31+
pypi:
32+
needs: build
33+
runs-on: ubuntu-latest
34+
environment: pypi
35+
permissions:
36+
id-token: write
37+
steps:
38+
- uses: actions/download-artifact@v4
39+
with:
40+
name: distributions
41+
path: dist/
42+
- uses: pypa/gh-action-pypi-publish@release/v1
43+
with:
44+
attestations: true
45+
46+
github:
47+
needs: build
48+
runs-on: ubuntu-latest
49+
permissions:
50+
contents: write
51+
steps:
52+
- uses: actions/download-artifact@v4
53+
with:
54+
name: distributions
55+
path: dist/
56+
- name: Create GitHub release
57+
env:
58+
GH_TOKEN: ${{ github.token }}
59+
run: |
60+
prerelease=""
61+
[[ "$GITHUB_REF_NAME" == *-* ]] && prerelease="--prerelease"
62+
gh release create "$GITHUB_REF_NAME" dist/* --generate-notes $prerelease

CHANGELOG.md

Lines changed: 23 additions & 16 deletions
Large diffs are not rendered by default.

CITATION.cff

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
cff-version: 1.2.0
22
message: "If you use this software, please cite the paper."
33
title: "RewardHarness: Self-Evolving Agentic Post-Training"
4-
version: "0.1.2"
5-
date-released: "2026-05-16"
4+
version: "0.2.0-rc1"
5+
date-released: "2026-08-17"
66
type: software
77
authors:
88
- given-names: Yuxuan

CLAUDE.md

Lines changed: 11 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,9 @@
11
# RewardHarness
22

3+
The canonical import namespace is `rewardharness`; `src` contains compatibility
4+
adapters only. New code must not introduce dependencies from `rewardharness`
5+
back to `src`.
6+
37
Self-evolving reward model for image editing quality evaluation.
48
Qwen2.5-VL-7B (vLLM) as Sub-Agent; Gemini `gemini-3.1-pro-preview` as orchestration layer (Router, ChainAnalyzer, Evolver).
59

@@ -12,17 +16,17 @@ Gemini (orch layer) Qwen vLLM (sub-agent + tools)
1216
Evolver._validate_tool()
1317
```
1418

15-
- **Router** (`src/router.py`): selects relevant Skills/Tools from Library per editing prompt
16-
- **ChainAnalyzer** (`src/chain_analyzer.py`): analyzes reasoning chains → improvement signals (skill_updates, tool_updates)
17-
- **Evolver** (`src/evolver.py`): applies signals to Library; validates tool prompts via vLLM; snapshot/restore for rollback
18-
- **SubAgent** (`src/sub_agent.py`): multi-turn Qwen reasoning with `<think>/<tool>/<obs>/<answer>` tags
19-
- **Library** (`src/library/`): Skills (markdown evaluation guidance) + Tools (VLM system_prompts)
20-
- **Pipeline** (`src/pipeline.py`): evolution loop with Phase A (skills) / Phase B (tools) / Phase C (pruning)
19+
- **Router** (`rewardharness/evaluation/router.py`): selects relevant Skills/Tools from Library per editing prompt
20+
- **ChainAnalyzer** (`rewardharness/evolution/analyzer.py`): analyzes reasoning chains → improvement signals (skill_updates, tool_updates)
21+
- **Evolver** (`rewardharness/evolution/evolver.py`): applies signals to Library; validates tool prompts via vLLM; snapshot/restore for rollback
22+
- **SubAgent** (`rewardharness/evaluation/engine.py`): multi-turn Qwen reasoning with `<think>/<tool>/<obs>/<answer>` tags
23+
- **Library** (`rewardharness/resources/library/`): Skills (markdown evaluation guidance) + Tools (VLM system_prompts)
24+
- **Pipeline** (`rewardharness/evolution/pipeline.py`): evolution loop with Phase A (skills) / Phase B (tools) / Phase C (pruning)
2125

2226
## Commands
2327

2428
```bash
25-
# Tests (107 tests, ~2s)
29+
# Tests (138 tests, a few seconds)
2630
python -m pytest tests/ -v
2731

2832
# Evolution (main experiment)

CONTRIBUTING.md

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ Thanks for considering a contribution. RewardHarness is a research codebase, so
1010
make install-dev
1111
make check # env / credentials / endpoints
1212
make test # mocked tests, ~2 s
13+
make quality # Ruff, mypy, rating-integrity and shell syntax gates
1314
```
1415
PRs that break the test suite won't be merged.
1516
3. **Keep PRs focused.** One PR = one logical change. If you're refactoring while adding a feature, split into two PRs.
@@ -45,11 +46,16 @@ One-line subject (≤72 chars), then optional body. Reference issues with `#NNN`
4546

4647
## Tests
4748

48-
- All new code in `src/` should have a corresponding mocked test in `tests/`.
49-
- Tests must not make real network calls. If you need a Gemini response, patch `src.router.call_gemini`. If you need a Sub-Agent completion, patch `src.sub_agent.OpenAI`. See `tests/README.md` for the patterns.
49+
- All new code in `rewardharness/` should have a corresponding mocked test in `tests/`.
50+
- Tests must not make real network calls. Inject or patch the client boundary;
51+
see `tests/README.md` for the current patterns.
5052

5153
## Releases
5254

5355
Versions follow [SemVer](https://semver.org/). Before tagging, maintainers run
5456
`make release-check`, cut a tag + GitHub Release, and add a `## [x.y.z]` block
5557
to `CHANGELOG.md`.
58+
59+
Release candidates use PEP 440 versions such as `0.2.0rc1` and matching tags
60+
such as `v0.2.0-rc1`. The tag workflow publishes the same validated artifacts
61+
to PyPI and GitHub through trusted publishing.

0 commit comments

Comments
 (0)