Skip to content
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 93 additions & 0 deletions plugins/all-skills/skills/orca-replay/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
---
name: orca-replay
category: development
description: Answers questions about a past agent run from its recording rather than from memory, and replays or forks that run. Use when asked why an earlier run did something, or to reproduce a failure.
---

# OrcaReplay Run Reader

Coding agents are confidently wrong about their own history. A summary written from memory is
missing the tool results, the exit codes, and the files that changed without anyone mentioning
them.

This skill enforces one rule: **when a question is about something that already happened, read the
trace before answering.** Do not reconstruct it. If a recording exists, guessing is the wrong move
even when the guess would have been right.

## When to Use This Skill

- "Why did you delete / overwrite / move X?"
- "What changed this file?" or "Which step broke the build?"
- "Can you reproduce yesterday's failure?"
- "Would a different model have got this right?"

## What This Skill Does

1. **Finds the run** — `orca_list_runs`, newest first, naming the run each fork came from.
2. **Narrows to the cause** — `orca_graph` with `to: <event seq>` returns only the chain that
produced one event. That is the shape of an answer to *why*; the full timeline
(`orca_show_run`) is the shape of an answer to *what*.
3. **Separates evidence from inference** — every edge is labelled `recorded` (the recorder watched
it happen) or `inferred` (derived just now from a named rule). "The trace shows the `rm` at step
14 removed it" and "this looks like the `rm` at step 14, going by timing" are different claims,
and collapsing them into one confident sentence is the exact failure this skill exists to
prevent.
4. **Reproduces before explaining** — `orca_replay` re-runs the recording offline and reports what
could not be reproduced.
5. **Compares models only when asked** — `orca_compare` forks one run onto several models from the
same checkpoint, graded by a shell command whose exit code is the verdict.

## How to Use

### Basic Usage

```
Why did the last run overwrite src/config.ts?
```

The agent reads the trace instead of recalling the session, and answers from the tool call that
did it.

```
Does yesterday's failure still reproduce?
```

## Example

**User**: "Which step broke the build?"

**Output**:
```
Step 14 did. From the recording (recorded edge, not inferred):

14 TOOL file_editor {"command":"str_replace","path":".../tsconfig.json"}
15 SHELL npm run build exit 2
16 FILE tsconfig.json modified +1 -1

The edit at 14 removed "strict": true, and the build at 15 exited 2. I replayed the run
(reused=5/5, divergences=0) and it fails the same way.
```

## Tips

- **Pass `worktree: true` to replay.** Without it, replay restores the recorded filesystem over the
working tree for the duration of the run, so uncommitted work is absent in the meantime — and
stays absent if the replay is interrupted.
- **Check what re-executes before the first replay.** Replay serves the model's answers from the
trace, but the agent process runs again for real, so its shell commands run again too. A run that
only read files and edited the repository is free to replay; one that touched Docker, a database
or a remote host is not.
- **Replay is not a sandbox.** It blocks provider egress, not the network.
- **A matching replay is not a determinism result.** The model is not re-asked, its recorded
answers are served back — so replay cannot tell you whether a *fresh* run would fail the same
way. Say so rather than implying it did.
- **`reused=3/5` is usually not a failure.** Harnesses make calls for themselves — a quota probe, a
session-naming request — and replay does not repeat them.
- **No recording means no answer.** Say that plainly and offer to start one with
`orca record claude`, rather than falling back to a reconstruction.

## Requirements

The `orcareplay` npm package (Node 20+) with its MCP server registered as `orca`, and at least one
recorded run in the project's `.orca/runs` directory. Apache-2.0 —
[Continuum-AI-Corp/OrcaReplay](https://github.com/Continuum-AI-Corp/OrcaReplay).