Skip to content

feat(agent): tiered validation pipeline (Tiers 2–3) #490

Description

@krokoko

Context: ROADMAP.md → Agent quality → Tiered validation pipeline
Related: #149 (ADR-013 closed), in-pipeline fix-up draft, evaluation pipeline draft


Component

Agent (Python runtime)

Describe the feature

Three post-agent tiers per roadmap:

  1. Tier 1 (already shipped): tool validation—build/test/lint via agent post-hooks and PR body baselines. This issue implements Tiers 2–3 only.
  2. Tier 2: code quality—DRY/SOLID/complexity analysis.
  3. Tier 3: risk and blast radius analysis—feeds PR risk classifier and review routing ((orchestration): Tiered human review routing by PR risk #425).

See ADR-013 tiered validation pyramid.

Use case

Build/lint passing does not imply maintainable or low-risk changes. Operators want graduated validation before PR creation and to feed evaluation metrics.

Proposed solution

  1. Define tier interfaces in agent/ with Blueprint opt-in per repo.
  2. Tier 2: integrate existing linters/analyzers (eslint complexity, ruff, optional semgrep rules) with structured JSON output.
  3. Tier 3: rule-based blast radius (files touched, public API surface, migrations)—output consumed by PR risk classifier draft.
  4. Non-blocking advisory mode first; optional Cedar/policy gates later.
  5. Emit validation results to TaskEventsTable and task record for dashboards.

Other information

  • ADR-013 documents the validation pyramid; this issue tracks implementation.

  • Design context: docs/decisions/ADR-013-tiered-validation-pyramid.md, docs/design/EVALUATION.md.

  • This might be a breaking change

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent-runtimePython agent container: pipeline, runner, hooks, prompts, tools, DockerfileenhancementNew feature or requestorchestrationTask lifecycle, REST API handlers, orchestrator Lambdas, durable execution

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions