-
Notifications
You must be signed in to change notification settings - Fork 43
[External Validation] quality/quality.external_evaluation_live - quality-ops #20
Copy link
Copy link
Open
Labels
category.quality.external_evaluation_liveExternal validation category quality.external_evaluation_liveExternal validation category quality.external_evaluation_livedomain.qualityExternal validation domain qualityExternal validation domain qualityexternal-validationExternal validation tracker work itemExternal validation tracker work itemmeasurement-infrastructureFateCat measurement infrastructure closure workFateCat measurement infrastructure closure workoperator-action-requiredRequires authorized operator actionRequires authorized operator actionowner.quality-opsExternal validation owner quality-opsExternal validation owner quality-ops
Description
Metadata
Metadata
Assignees
Labels
category.quality.external_evaluation_liveExternal validation category quality.external_evaluation_liveExternal validation category quality.external_evaluation_livedomain.qualityExternal validation domain qualityExternal validation domain qualityexternal-validationExternal validation tracker work itemExternal validation tracker work itemmeasurement-infrastructureFateCat measurement infrastructure closure workFateCat measurement infrastructure closure workoperator-action-requiredRequires authorized operator actionRequires authorized operator actionowner.quality-opsExternal validation owner quality-opsExternal validation owner quality-ops
External Validation Work Item
external-work.e8aee56b238d245fqualityquality.external_evaluation_livequality-opsP0external.007c49f25f216465,external.1c55d1a889352e2e,external.5ec80b3706220f4a,external.7e560cc23b8ad5a3,external.b58c00b442803e55,external.b67b5799548c1a61,external.c9e99898d3ba0efbRequired Credentials
benchmark artifact accessevaluation runner accessexternal benchmark source accesshuman review owner approvalmodel/API credentials when live evaluation uses external modelRequired Evidence
Operator Commands
bash scripts/core-quality-corpus-gate.sh --output-json <core-quality-json>bash scripts/evaluation-trend-gate.sh --history-dir <dir> --output-json <path>bash scripts/evidence-coverage-trend-gate.sh --output-json <coverage-trend-json>bash scripts/external-validation-proof-ref-gate.sh --work-queue-json <work-queue-json> --evidence-json <proof-ref-bundle-json> --output-json <proof-ref-gate-json>bash scripts/mingli-bench-gate.sh --output-json <mingli-bench-json>bash scripts/mingli-bench-gate.sh --output-json <path>bash scripts/run-evaluations.sh --output-dir <dir>Proof Ref Template
evidence://external-validation/quality-external-evaluation-live/<run-id>sha256:<64 lowercase hex artifact digest>bash scripts/external-validation-proof-ref-gate.sh --work-queue-json <work-queue-json> --evidence-json <proof-ref-bundle-json> --output-json <proof-ref-gate-json>Blocking Items
category_live_evidence_missingcategory_live_pendingproof_ref_missingstale_owner_pendingClosure Condition
External benchmark summary, corpus trend and evidence coverage gate are attached without question-level leakage.
Non-Claims