Skip to content

[External Validation] quality/quality.external_evaluation_live - quality-ops #20

Description

@tradecatlabs

External Validation Work Item

  • Work item: external-work.e8aee56b238d245f
  • Domain: quality
  • Category: quality.external_evaluation_live
  • Owner: quality-ops
  • Priority: P0
  • Occurrences: external.007c49f25f216465, external.1c55d1a889352e2e, external.5ec80b3706220f4a, external.7e560cc23b8ad5a3, external.b58c00b442803e55, external.b67b5799548c1a61, external.c9e99898d3ba0efb

Required Credentials

  • benchmark artifact access
  • evaluation runner access
  • external benchmark source access
  • human review owner approval
  • model/API credentials when live evaluation uses external model

Required Evidence

  • EvaluationRun summary passed
  • evaluation trend gate passed
  • external benchmark usage/license boundary accepted
  • no prompt/answer/report leakage in artifacts

Operator Commands

  • bash scripts/core-quality-corpus-gate.sh --output-json <core-quality-json>
  • bash scripts/evaluation-trend-gate.sh --history-dir <dir> --output-json <path>
  • bash scripts/evidence-coverage-trend-gate.sh --output-json <coverage-trend-json>
  • bash scripts/external-validation-proof-ref-gate.sh --work-queue-json <work-queue-json> --evidence-json <proof-ref-bundle-json> --output-json <proof-ref-gate-json>
  • bash scripts/mingli-bench-gate.sh --output-json <mingli-bench-json>
  • bash scripts/mingli-bench-gate.sh --output-json <path>
  • bash scripts/run-evaluations.sh --output-dir <dir>

Proof Ref Template

  • Pattern: evidence://external-validation/quality-external-evaluation-live/<run-id>
  • Artifact hash: sha256:<64 lowercase hex artifact digest>
  • Verification command: bash scripts/external-validation-proof-ref-gate.sh --work-queue-json <work-queue-json> --evidence-json <proof-ref-bundle-json> --output-json <proof-ref-gate-json>

Blocking Items

  • category_live_evidence_missing
  • category_live_pending
  • proof_ref_missing
  • stale_owner_pending

Closure Condition

External benchmark summary, corpus trend and evidence coverage gate are attached without question-level leakage.

Non-Claims

  • This issue does not prove live validation has passed.
  • Do not paste token, secret, DSN, endpoint URL, chat id, user input, report body or production logs.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions