Skip to content

docs(internal): drop hardcoded home path from bat-story-eval skill #2253

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sergeykad merged 3 commits into from
Aug 23, 2026
Merged

docs(internal): drop hardcoded home path from bat-story-eval skill #2253

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
dedd434
docs(internal): drop hardcoded home path from bat-story-eval skill
Aug 23, 2026
36983e3
fix(internal): resolve uat-stories worktree from the git common dir
Aug 23, 2026
c6b97df
Merge branch 'master' into docs/bat-story-eval-portable-paths
sergeykad Aug 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .claude/skills/bat-story-eval/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ If `$ARGUMENTS` is `--help` or missing `--baseline`, show usage and stop:
### 0a. Compute Diff

```bash
cd /home/julien/github/ha-mcp/worktree/uat-stories
cd "$(dirname "$(git rev-parse --path-format=absolute --git-common-dir)")/worktree/uat-stories"
git diff <baseline>..HEAD -- src/ha_mcp/ --stat
git diff <baseline>..HEAD -- src/ha_mcp/ --name-only
```
Expand Down Expand Up @@ -112,7 +112,7 @@ For EACH agent, run all stories against the **baseline** version. One container
### 1a. Start container with first story

```bash
cd /home/julien/github/ha-mcp/worktree/uat-stories
cd "$(dirname "$(git rev-parse --path-format=absolute --git-common-dir)")/worktree/uat-stories"
uv run python tests/uat/stories/run_story.py \
catalog/<first_story>.yaml \
--agents <agent> --keep-container \
Expand Down Expand Up @@ -370,4 +370,4 @@ Flag >5% total size increase (directly impacts token cost per turn).
- Reuse containers: first story starts it (`--keep-container`), rest use `--ha-url`
- Custom story YAMLs go to `/tmp/` (ephemeral); full details reported in Step 6
- See "Metrics" section in Step 4 for primary vs secondary metric classification
- The working directory MUST be `/home/julien/github/ha-mcp/worktree/uat-stories` for `uv run`
- The working directory MUST be the `worktree/uat-stories` worktree root (where `pyproject.toml` lives) for `uv run`
2 changes: 1 addition & 1 deletion .claude/skills/bat-story-eval/references/evaluation-protocol.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ uv run python tests/uat/stories/scripts/ha_query.py \
Check the exit code first. `ha_query.py` exits non-zero when the agent CLI
itself failed, and prints `[exit N]` (plus stderr, when there is any) after
whatever text the CLI managed to produce — `[exit 124]` for a query that hung
past its timeout. The one non-zero exit without a marker is `Error: <agent>
past its timeout. The one non-zero exit without a marker is `Error: {agent}
CLI not found`, which means the agent is not installed: fix the environment
rather than re-running. A failed query is **not a measurement**: do not score
it as any of the three outcomes below — re-run it, and if it keeps failing,
Expand Down
Loading