RFC 008: Spec-neutral local and remote validation architecture

[zkwentz]

What & Why

Validation needs one trust model and report contract across local author checks and independent Hub certification. This RFC defines a spec-neutral adapter, plan, executor, report, and remote-lane architecture while keeping the future task workflow in #898 explicit.

Risk & Review Guidance

• Risk level: MEDIUM — architecture-only, but it constrains every implementation layer that follows
• Task class: docs
• Review focus: rfcs/008-local-remote-validation.md:10 for the spec boundary; rfcs/008-local-remote-validation.md:224 for local semantics; rfcs/008-local-remote-validation.md:280 for remote trust boundaries
• Blast radius: A wrong abstraction would couple validation to one package format or overstate what author-run checks prove.

Decisions & Alternatives

• Normalize through trusted spec adapters — rejected making Harbor the core contract because Spec-agnostic task validation: openenv validate-task for external task.md packages (PostTrain Arena / BenchFlow) #898 needs other execution models.
• Separate author reports from certification — local and author-triggered remote runs always remain uncertified.
• Capability-driven skips — rejected silently treating unavailable remote checks as passes.

Verification

• PASS: PYTHONPATH=src:envs .venv/bin/python -m pytest tests/ -q -m 'not integration' — 1,559 passed, 114 skipped, 31 external integration tests deselected on the complete stack.
• PASS: Fork whole-stack CI — 7/7 checks passed, including Python 3.11/3.12, lint, docs, lock validation, and package smoke tests.
• NOT verified: The planned remote coordinator and certification lanes are architectural only.

Scope & Non-Goals

• No coordinator deployment, ACA lane, model evaluation, or task-package CLI is implemented in this PR.

Stack

Base of the 17-PR RFC 008 stack.

Full 17-PR stack

1. #942 RFC 008: Spec-neutral local and remote validation architecture ← current
2. #943 Validation foundation: spec-neutral contracts and adapters
3. #944 Validation policy: spec-aware check registry and planner
4. #945 Validation executor: hardened shared reports and eligibility
5. #946 Local validation API: execution-model-aware profiles
6. #947 CLI: validation profiles and shared reports
7. #948 HF runtime: dedicated sandbox execution mode
8. #949 HF certification: enforce trusted dedicated isolation
9. #950 [RFC 008] Define remote author validation and publish gating
10. #951 Validation: unify CLI reports and add a strict publish profile
11. #952 Validation: add structured author diagnostics and remediation
12. #953 Validation: run author validation in dedicated HF Sandboxes
13. #954 Validation: isolate submitted code in author Sandboxes
14. #955 Validation: bind author reports to policy and source
15. #956 CLI: scaffold publish requirements
16. #957 CLI: gate pushes on staged remote validation
17. #958 Docs: explain local, remote, and publish validation

All upstream PRs remain draft while RFC 008 is under review.

[thegovind]

@zkwentz - I opened #898 and reviewed RFC 008 plus the stack through #958. These efforts look complementary:

• RFC 008: Spec-neutral local and remote validation architecture #942–CLI: validation profiles and shared reports #947 should own the shared spec adapter, planner, executor, report contract, and served openenv validate flow.
• Spec-agnostic task validation: openenv validate-task for external task.md packages (PostTrain Arena / BenchFlow) #898 should narrow to the one-shot task-package lane that RFC 008 explicitly leaves as follow-up work.

I do not plan to open a parallel validation-core PR. My proposed follow-up split is:

1. A built-in PostTrain/BenchFlow task.md adapter and structural policy on top of Validation foundation: spec-neutral contracts and adapters #943/Validation policy: spec-aware check registry and planner #944.
2. A one-shot local runner for Docker oracle replay plus empty-trial sanity, using the shared report contract and never the served OpenEnv launcher.
3. A separate task-specific CLI, docs, and example on top of CLI: validation profiles and shared reports #947. Per Amy's comment on Spec-agnostic task validation: openenv validate-task for external task.md packages (PostTrain Arena / BenchFlow) #898, this should not overload the existing openenv validate command. I lean toward openenv validate-task (or openenv task validate) rather than validate-env, since openenv validate is already the environment command.
4. Public third-party adapter discovery should be local-author functionality only; official/remote certification should continue to use pinned trusted adapters.

The Harbor/ATIF/ACP trajectory work from my earlier investigation remains separate; it should not expand #898.

My RFC read is that we do not need a competing RFC, but the one-shot lifecycle, reward boundary, and policy should be captured in a focused RFC 008 amendment or follow-up section before the runner lands. I am happy to own that and the #898 implementation.

If we work in parallel before this stack lands, I would base the implementation on #947 (rfc008-06-cli) rather than main or #958; otherwise I will wait for #942–#947 to merge. Does that ownership split and branch base work for you?