Skip to content

testing-checklist.md

Latest commit

 

History

History
55 lines (44 loc) · 2.47 KB

testing-checklist.md

File metadata and controls

55 lines (44 loc) · 2.47 KB
Raw

Testing checklist for optional repair integration

An extension adopting the optional-integration recipe should verify all of the following before release. The reference implementation of a two-branch harness is the pi-repair-layer package smoke test (scripts/package-smoke.mjs in that repo): it compiles the consumer once, then runs the emitted JS in a clean project with and without the package installed.

Adapter branch (package installed)

  1. Activation registers the adapted definition (not the raw object) and emits no fallback note.
  2. Strictly valid input passes through unchanged.
  3. Every configured alias/selector repair produces valid arguments (e.g. {file_path: "/x"} → {path: "/x"} via prepareArguments).
  4. Unconfigured content and similar-but-unknown keys remain unchanged.
  5. Unrepairable input throws before the executor runs.
  6. An existing owner prepareArguments shim is still called.

Fallback branch (package absent)

  1. Activation succeeds — no error escapes the factory.
  2. The registered definition is the raw one (identity: the same object, no prepareArguments added).
  3. Exactly one stderr note is emitted, naming the extension and the tool.
  4. Strictly valid input behaves identically to the adapter branch.
  5. Decide explicitly about invalid input: in fallback mode pi's native validation runs TypeBox Value.Convert first, which silently coerces some invalid values (null → "null") instead of repairing or rejecting. If that is unacceptable for the tool, the dependency should be hard, not optional.

Discrimination

  1. A module-not-found error naming a different (transitive) module is rethrown, not treated as absence — simulate by breaking a transitive dependency of the installed package.

Environment expectations (document, don't test)

  • Compiled (Bun-binary) pi: recipe always falls back safely, even with the package installed — dynamic import() cannot resolve npm siblings there.
  • Git-installed or cross-scope consumers: fallback; offer optionalDependencies.
  • Node-based pi installs with the package npm-installed in the same scope: adapter branch.

Privacy

  1. If onOutcome or feedback is used: telemetry and persisted entries contain no argument values or note text (rule IDs, stages, policy, outcome, and fingerprints only).
  2. If RepairLifecycle feedback is used: concurrent identical calls receive one note each, and stale notes do not leak to later calls.