Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 45 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -198,6 +198,51 @@ All hooks managed by [prek](https://prek.j178.dev/) (installed via `npm install`
6. Use `./scripts/dev-setup.sh --expose-cli` only with explicit approval for host-visible CLI exposure
7. Run the tests targeted to the behavior you change once per relevant change set; rerun them after later edits or hook autofixes that can affect that behavior

### E2E Selection and Authoring

Use live E2E only for behavior that needs a real shell, installer, process,
Docker, OpenShell, `/proc`, sandbox, external service, or GitHub Actions
boundary. Put deterministic code, parser, registry, workflow-planner, and
fixture logic in unit, integration, package-contract, or `e2e-support` tests
instead. Do not add a live E2E target for a check that can be observed through a
stable local boundary.

Before adding or extending E2E coverage, name the semantic coverage dimension
that is missing. Existing migrated examples show the intended granularity:
catalogue targets pair environment, onboarding profile, expected state, optional
lifecycle, and `suiteIds`; `dashboard-remote-bind` owns install, onboard,
artifacts, and terminal cleanup; `credential-sanitization`,
`telegram-injection`, `messaging-providers`, `messaging-compatible-endpoint`,
and `gpu-e2e` are separate behavior contracts rather than one broad "full" run.
Extend matrix metadata only when it selects an already-defined behavior
dimension. Do not duplicate behavior logic in a second registry, workflow list,
or hand-maintained catalogue; use the typed registry and shared E2E workflow
planner documented in [`test/e2e/README.md`](test/e2e/README.md) and
[`test/e2e/docs/README.md`](test/e2e/docs/README.md).

If a gap is real but not ready for a test, record it as a combinatorial gap
instead of adding speculative coverage. State the missing dimension, the
existing nearest coverage, why a new test would duplicate or overreach current
behavior, and the issue or PR that will make it testable. A gap note must not
change release judgment by itself.

Assert outcomes, state, artifacts, and redacted diagnostics. Do not assert
incidental terminal output, progress wording, spinner frames, ANSI escape
sequences, timing text, or prompt layout unless that text is the product
contract under review. Terminal traces are evidence; they are not stable
behavior unless the issue explicitly makes them the behavior.

Retries require a checked-in bounded policy with a narrow transient signature,
owner, idempotence or reconciliation basis, attempt evidence, and an entry in
[`test/e2e/RETRY_INVENTORY.md`](test/e2e/RETRY_INVENTORY.md). Do not add
unproven retries, ambiguous mutation retries, or broad failed-job reruns. A
mutation retry is allowed only after the test reconciles the external state and
proves repeating the same desired operation is safe. Keep bounded operation
retries separate from complete workflow reruns: `E2E / Main Retry` records
attempts and does not request a broad rerun, while Hosted Runner Recovery owns
at most one full rerun only for authenticated GitHub-hosted runner-loss
evidence.

### Plain Language and Direct Design

- Use existing repository vocabulary and name what a thing does.
Expand Down
28 changes: 28 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -262,6 +262,34 @@ artifact sink. Pass the auto fixture's frozen, canonical `progress` capability
through unchanged; custom, copied, or no-op progress adapters are rejected at
audited subprocess boundaries.

Use live E2E only when the behavior needs a real shell, installer, process,
Docker, OpenShell, `/proc`, sandbox, external service, or GitHub Actions
boundary. Prefer unit, integration, package-contract, or `e2e-support` tests for
deterministic code and workflow-planner logic. Select or extend coverage by
semantic dimension, not by incidental output. Migrated targets such as
`dashboard-remote-bind`, `credential-sanitization`, `telegram-injection`,
`messaging-providers`, `messaging-compatible-endpoint`, and `gpu-e2e` show the
expected shape: each target owns a behavior contract, while the typed registry
keeps environment, onboarding profile, expected state, lifecycle, and `suiteIds`
as matrix metadata. Extend that metadata only to select existing behavior; do
not duplicate behavior logic in workflows, lists, or catalogues. When a
combination is missing but not ready for a test, record a combinatorial gap with
the missing dimension, nearest existing coverage, why a new test would duplicate
or overreach current behavior, the follow-up owner, and the issue or PR that will
make the gap testable. Do not add speculative coverage or change release judgment.

E2E assertions should check outcomes, state, artifacts, and redacted diagnostics.
Do not assert incidental terminal output, progress wording, ANSI escape
sequences, spinner frames, or timing text unless that text is the product
contract. A retry must have a checked-in bounded policy, a narrow transient
signature, idempotence or reconciliation evidence, per-attempt artifacts, and a
matching [`test/e2e/RETRY_INVENTORY.md`](test/e2e/RETRY_INVENTORY.md) entry. Do
not add unproven retries, ambiguous mutation retries, or broad failed-job
reruns. Keep operation-level retries separate from complete workflow reruns:
`E2E / Main Retry` records attempt evidence without requesting a broad rerun,
and Hosted Runner Recovery owns at most one full rerun only for authenticated
GitHub-hosted runner-loss evidence.

### macOS Test Dependencies

Some tests run command-line tools that macOS does not ship.
Expand Down
Loading