Skip to content

fix: harden skill-first research workflow #51

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
freshtechbro merged 5 commits into from
May 10, 2026
Merged

fix: harden skill-first research workflow #51

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
965d52c
docs: add skill-first research decision record
freshtechbro May 9, 2026
8f78a27
feat: make research skill evidence gated
freshtechbro May 9, 2026
46ec773
docs: demote public research reliability wording
freshtechbro May 9, 2026
87701d4
fix: guide research recovery for gated providers
freshtechbro May 9, 2026
14e946c
fix: harden skill-first research workflow
freshtechbro May 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -257,7 +257,7 @@ Start every surface check from generated help when you need the current public l

- `screencast / browser replay`: `screencast-start`, `screencast-stop`
- `desktop observation`: `desktop-status`, `desktop-windows`, `desktop-active-window`, `desktop-capture-desktop`, `desktop-capture-window`, `desktop-accessibility-snapshot`
- `computer use / browser-scoped computer use`: `--challenge-automation-mode off|browser|browser_with_helper` on `research run`, `shopping run`, `product-video run`, `inspiredesign run`, and `macro-resolve --execute`; start with `npx opendevbrowser research run --topic "account recovery flow" --source-selection auto --challenge-automation-mode browser --mode json --output-format json` when you need the first entry point, and use `review` plus `session-inspector` as the quickest proof surfaces while the optional helper stays browser-scoped rather than becoming a desktop agent
- `computer use / browser-scoped computer use`: `--challenge-automation-mode off|browser|browser_with_helper` on `research run`, `shopping run`, `product-video run`, `inspiredesign run`, and `macro-resolve --execute`; start with `npx opendevbrowser research run --topic "account recovery flow" --sources web,community --challenge-automation-mode browser --mode json --output-format json` when you need the first entry point, and use `review` plus `session-inspector` as the quickest proof surfaces while the optional helper stays browser-scoped rather than becoming a desktop agent

## Challenge Handling Boundary

Expand Down
13 changes: 8 additions & 5 deletions docs/CLI.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -271,7 +271,7 @@ npx opendevbrowser -v
- A `Find It Fast` block that uses the exact lookup terms `screencast / browser replay`, `desktop observation`, and `computer use / browser-scoped computer use`.
- That block maps replay to `screencast-start` / `screencast-stop`, desktop observation to the public `desktop-*` family, and browser-scoped computer use to `--challenge-automation-mode` on `research run`, `shopping run`, `product-video run`, `inspiredesign run`, and `macro-resolve --execute`, with a concrete `research run --topic ... --challenge-automation-mode browser` entry command.
- An `Agent Quick Start` block that tells agents to start with `opendevbrowser_prompting_guide` or `opendevbrowser_skill_load opendevbrowser-best-practices "quick start"` before low-level browser commands.
- A follow-up `validated_lanes` entry that points agents to `opendevbrowser_skill_load opendevbrowser-best-practices "validated capability lanes"` for the current reliable transcript, research, and shopping runbook.
- A follow-up `validated_lanes` entry that points agents to `opendevbrowser_skill_load opendevbrowser-best-practices "validated capability lanes"` for the current transcript, evidence-gated research, and shopping runbook.
- A direct pointer to `opendevbrowser_skill_list` when an agent needs a different local skill lane.
- A browser-scoped computer-use description that makes the optional helper boundary explicit and does not imply a desktop agent.
- The complete generated CLI command, flag, and `opendevbrowser_*` tool inventories.
Expand All @@ -280,7 +280,7 @@ npx opendevbrowser -v
Quick lookup terms from generated help:
- `screencast / browser replay`: `screencast-start`, `screencast-stop`
- `desktop observation`: `desktop-status`, `desktop-windows`, `desktop-active-window`, `desktop-capture-desktop`, `desktop-capture-window`, `desktop-accessibility-snapshot`
- `computer use / browser-scoped computer use`: `--challenge-automation-mode off|browser|browser_with_helper` on `research run`, `shopping run`, `product-video run`, `inspiredesign run`, and `macro-resolve --execute`; entry command `npx opendevbrowser research run --topic "account recovery flow" --source-selection auto --challenge-automation-mode browser --mode json --output-format json`
- `computer use / browser-scoped computer use`: `--challenge-automation-mode off|browser|browser_with_helper` on `research run`, `shopping run`, `product-video run`, `inspiredesign run`, and `macro-resolve --execute`; entry command `npx opendevbrowser research run --topic "account recovery flow" --sources web,community --challenge-automation-mode browser --mode json --output-format json`

These first-contact assets are also mirrored as release and website inputs through `src/cli/onboarding-metadata.json`, `src/public-surface/generated-manifest.ts`, and `src/public-surface/generated-manifest.json`.

Expand Down Expand Up @@ -424,9 +424,11 @@ and the shared inspiredesign artifact source at `src/inspiredesign/handoff.ts`.

#### Research (`research run`)

`research run` is a low-level, provider-constrained primitive. Load `opendevbrowser-research` before research tasks so planning, evidence review, confidence, limitations, and final synthesis stay evidence-gated.

```bash
npx opendevbrowser research run --topic "browser automation" --days 30 --mode compact
npx opendevbrowser research run --topic "Chrome extension debugging workflows" --days 30 --source-selection auto --browser-mode managed --mode json
npx opendevbrowser research run --topic "Chrome extension debugging workflows" --days 30 --sources web,community --browser-mode managed --mode json
npx opendevbrowser research run --topic "creator tools" --sources web,shopping --include-engagement --limit-per-source 5 --mode context
```

Expand All @@ -450,11 +452,12 @@ Flags:
- `--cookie-policy` (alias of `--cookie-policy-override`)

Notes:
- Use `--source-selection auto` for generic topical research.
- Prefer explicit public source-family examples such as `--sources web,community`; use `--source-selection` only to explain selector semantics.
- Use `--browser-mode extension` when X, Threads, Facebook, Reddit, or another signed-in social provider needs an existing relay-backed browser session; use `managed` for reproducible no-auth reruns.
- In the current contract, `auto` and `all` both stay inside the public topical families (`web`, `community`, `social`).
- In the current contract, `auto` and `all` both stay inside the public topical families (`web`, `community`, `social`), but neither value guarantees reliability.
- Add shopping only with `--source-selection shopping` or explicit `--sources ...shopping...` when the task is deliberately commercial.
- Successful research artifact bundles include human-readable `report.md` alongside `summary.md`, `records.json`, `context.json`, `meta.json`, and `bundle-manifest.json`.
- Inspect `records.json`, `context.json`, `meta.json`, and `report.md` before publishing final claims.
- Research runs fail instead of emitting a successful empty report when providers return only shell records, stale records, or no source evidence; successful runs persist diagnostics in `meta.json`.

#### Shopping (`shopping run`)
Expand Down
8 changes: 4 additions & 4 deletions docs/FIRST_RUN_ONBOARDING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ Expected:
- the block maps replay to `screencast-start` / `screencast-stop`
- the block maps desktop observation to the public `desktop-*` family
- the block maps browser-scoped computer use to `--challenge-automation-mode` on `research run`, `shopping run`, `product-video run`, `inspiredesign run`, and `macro-resolve --execute`
- the block includes a concrete browser-scoped entry command such as `npx opendevbrowser research run --topic "account recovery flow" --source-selection auto --challenge-automation-mode browser --mode json --output-format json`
- the block includes a concrete browser-scoped entry command such as `npx opendevbrowser research run --topic "account recovery flow" --sources web,community --challenge-automation-mode browser --mode json --output-format json`
- help then opens the `Agent Quick Start` block
- the block explicitly points agents to `opendevbrowser_prompting_guide`
- the block explicitly points agents to `opendevbrowser_skill_load opendevbrowser-best-practices "quick start"`
Expand All @@ -86,12 +86,12 @@ Optional terminal proof:
npx --no-install opendevbrowser --help | grep -E "screencast / browser replay|desktop observation|computer use / browser-scoped computer use|screencast-start|desktop-status|challenge-automation-mode"
```

## 2c) Validate the currently reliable workflow lanes
## 2c) Validate the current workflow lanes

These are the most repeatable capability checks from the April 6 validation pass.
These are repeatable capability checks from the April 6 validation pass, with research treated as an evidence-gated primitive. Load `opendevbrowser-research` before research tasks and inspect returned artifacts before final claims.

```bash
npx --no-install opendevbrowser research run --topic "Chrome extension debugging workflows" --days 30 --source-selection auto --mode json --output-format json
npx --no-install opendevbrowser research run --topic "Chrome extension debugging workflows" --days 30 --sources web,community --mode json --output-format json
npx --no-install opendevbrowser shopping run --query "wireless ergonomic mouse" --providers shopping/bestbuy,shopping/ebay --budget 150 --browser-mode managed --mode json --output-format json
npx --no-install opendevbrowser shopping run --query "27 inch 4k monitor" --providers shopping/bestbuy,shopping/ebay --budget 350 --sort lowest_price --browser-mode managed --mode json --output-format json
```
Expand Down
Loading
Loading