fix(runtime-baseline): make single-case thresholds fail closed

Replace incomplete run-level publishability with a strict single-case checker. Validate frozen policy values and case-specific inputs before evaluating every declared threshold, while leaving manifest conservation, repetition, CV, and provenance to the later controller.

Constraint: A single-case Pass is not a publishable baseline-run decision.

Confidence: High; boundary tests cover each threshold and isolated mutations prove the resume-drain and PING checks are observable.

Scope-risk: Limited to runtime-baseline threshold inputs, schema, fixture assembly, license configuration, and baseline documentation.

Tested: WSL cargo test -p runtime-baseline --all-targets; WSL cargo clippy -p runtime-baseline --all-targets -- -D warnings -D clippy::unwrap_used; cargo fmt --all -- --check; SkyWalking Eyes header check; git diff --check.

Not-tested: Fixed-host runtime baseline matrix remains deferred until real thresholds and the controller verifier exist.

Co-authored-by: OmX <omx@oh-my-codex.dev>

Author: AlexStocks

.licenserc.yaml

@@ -39,7 +39,7 @@ header:
 
   paths-ignore:
     - '**/*.md'
-    - '*.json'
+    - '**/*.json'
     - 'LICENSE'
     - 'NOTICE'
     - 'Cargo.lock'

docs/performance/storage-runtime-baseline.md

@@ -7,26 +7,27 @@
 
 - **冻结阈值只能来自固定 Linux/WSL 主机。** 设计规格明确禁止用 Windows / GitHub
   hosted runner 的跑分数字冻结阈值——矩阵的可重现性依赖固定的内核、文件系统与
-  CPU 拓扑。因此本仓的 CI 只在固定 Linux 环境执行完整矩阵；本地/Windows 仅用于
-  开发迭代与 `cargo check` / `clippy` / `cargo test`。
+  CPU 拓扑。当前 hosted CI 只编译 harness 并执行代码检查；完整矩阵未来只能通过
+  `workflow_dispatch` 在固定环境运行，或由维护者在规定的固定 Linux/WSL 主机执行。
 - **不在基线未稳前伪造数字。** `tools/runtime-baseline/src/thresholds.rs` 中的常量
   当前为占位哨兵值（`f64::NAN` / `usize::MAX` / `u64::MAX` / `i64::MAX`），并标注
   `TODO(#351)`。只要任一仍为占位值，`thresholds::all_frozen()` 返回 `false`，
-  `verify_outcome` 会据此把任何 case 判为 `NonPublishable`，杜绝「用占位值假装通过」。
+  `verify_outcome` 会据此返回 `PolicyUnavailable`，杜绝「用占位值假装通过」。
 
-## 本次 PR 交付了什么（#390）
+## 本次 PR 的准备工作（Refs #390）
 
 建立 #351 量化验收门禁的**阈值冻结机制**本身：
 
 | 文件 | 作用 |
 |------|------|
 | `tools/runtime-baseline/src/thresholds.rs` | #351 验收所需的冻结基线阈值常量 + `all_frozen()` 哨兵检测 |
-| `tools/runtime-baseline/src/verify.rs` | `Outcome` 比对器：`verify_outcome` / `verify_outcome_path`，返回 `Publishable` / `NonPublishable` / `Fail` |
-| `tools/runtime-baseline/schema/outcome.schema.json` | verifier 期望的 outcome JSON 结构 |
+| `tools/runtime-baseline/src/verify.rs` | 严格的单 case threshold checker；返回 `Pass` / `PolicyUnavailable` / `Fail` |
+| `tools/runtime-baseline/schema/outcome.schema.json` | 单 case threshold input 的 JSON Schema 2020-12 合同 |
 | 本文件 | 方法论与回填流程说明 |
 
-`verify.rs` 的单元测试 `placeholder_thresholds_make_every_case_non_publishable` 是「不伪造
-数字」的硬保证：占位状态下任何真实数值都判 `NonPublishable`。
+单 case 的 `Pass` 只证明该 case 满足冻结阈值，不代表完整 baseline run 可发布。最终发布
+结论还必须由 controller verifier 核对 manifest case 集守恒、重复轮次、CV、来源身份和
+完整 run outcome。占位状态下 checker 只返回 `PolicyUnavailable`。
 
 ## 为什么 observer 接线与真实数字是 follow-up
 
@@ -35,18 +36,19 @@
   控制器与 `cases.yaml`。这部分改动进入 `src/common/runtime/`，会拉入 `storage` → `rocksdb`，
   必须在能编译/运行 RocksDB 的环境验证——因此作为独立 PR 在 Linux 主机推进。
 - **Baseline Results（真实数字冻结）**：在固定 Linux/WSL 主机按版本化 `cases.yaml` 跑完整
-  矩阵后，把真实数值回填进 `thresholds.rs`，再让 verifier 全量判 `Publishable`。仅此 PR 验收
-  通过后才关闭 #350。
+  矩阵后，把真实数值回填进 `thresholds.rs`，再由完整 verifier 聚合全部 case、稳定性和
+  provenance 证据。只有原始结果、报告和真实阈值满足 #390 Acceptance 后才关闭 #390。
 
 ## 未来回填流程
 
 1. 固定 Linux/WSL 主机，按版本化 `cases.yaml` 跑完整矩阵（基础吞吐 + value 变体 + 单因素扫描
-   + offered-load 4 档 + lifecycle 3 类；每 case 重复 5 次、CV<=5%）。
+   + offered-load 4 档 + lifecycle 3 类；性能 case 重复 5 次、lifecycle case 重复 3 次，
+   CV<=5%）。
 2. 将实测值回填 `thresholds.rs`（替换 `TODO(#351)` 占位）。
-3. `python3 tools/runtime-baseline/run_baseline.py verify ...` 跑 `verify_outcome` 全量
-   判 `Publishable`。
+3. 由后续 `run_baseline.py verify` 对每个 case 执行 threshold check，并核对完整 case 集、
+   重复轮次、CV 和来源身份后产生 run 级发布结论。
 4. 提交原始 JSON/CSV + 环境清单（内核、CPU、文件系统）+ 摘要；冻结 #351 量化验收阈值。
-5. 关闭 #350。
+5. 满足 #390 Acceptance 后关闭 #390，并在 #350 / #351 中引用结果。
 
 ## 本地验证（不依赖 RocksDB）
 

tools/runtime-baseline/schema/outcome.schema.json

@@ -1,31 +1,173 @@
 {
-  "$schema": "http://json-schema.org/draft-07/schema#",
-  "$comment": "Baseline Results PR 回填真实数字、且 thresholds.rs 全部常量脱离占位哨兵值后，outcome 才可 publishable。本 schema 描述 verifier (src/verify.rs) 期望的 outcome 结构；verifier 使用 serde 反序列化，容忍未知字段以便前向兼容。",
-  "title": "BaselineOutcome",
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://arana-db.github.io/kiwi/schemas/runtime-baseline/threshold-outcome.schema.json",
+  "title": "RuntimeBaselineThresholdOutcome",
+  "description": "Strict input for one case threshold check. A passing case is not a publishable baseline run.",
   "type": "object",
+  "additionalProperties": false,
   "required": [
     "case_id",
     "qps",
     "p99_ms",
     "max_latency_ms",
     "error_rate",
-    "queued_plus_running_max"
+    "queued_plus_running_max",
+    "cpu_change_ratio",
+    "rss_change_ratio",
+    "threads_delta",
+    "case"
   ],
   "properties": {
-    "case_id": { "type": "string" },
-    "qps": { "type": "number" },
-    "p99_ms": { "type": "number" },
-    "max_latency_ms": { "type": "number" },
-    "error_rate": { "type": "number" },
-    "overload_120pct_error_rate": { "type": "number" },
-    "queued_plus_running_max": { "type": "integer" },
-    "storage_gate_pause_ms": { "type": "integer" },
-    "storage_gate_resume_drain_ms": { "type": "integer" },
-    "shutdown_drain_ms": { "type": "integer" },
-    "slow_storage_new_conn_ms": { "type": "integer" },
-    "slow_storage_ping_ms": { "type": "integer" },
-    "cpu_change_ratio": { "type": "number" },
-    "rss_change_ratio": { "type": "number" },
-    "threads_delta": { "type": "integer" }
+    "case_id": {
+      "type": "string",
+      "minLength": 1,
+      "pattern": ".*\\S.*"
+    },
+    "qps": {
+      "type": "number",
+      "minimum": 0
+    },
+    "p99_ms": {
+      "type": "number",
+      "minimum": 0
+    },
+    "max_latency_ms": {
+      "type": "number",
+      "minimum": 0
+    },
+    "error_rate": {
+      "type": "number",
+      "minimum": 0,
+      "maximum": 1
+    },
+    "queued_plus_running_max": {
+      "type": "integer",
+      "minimum": 0
+    },
+    "cpu_change_ratio": {
+      "type": "number",
+      "minimum": -1
+    },
+    "rss_change_ratio": {
+      "type": "number",
+      "minimum": -1
+    },
+    "threads_delta": {
+      "type": "integer"
+    },
+    "case": {
+      "oneOf": [
+        {
+          "type": "object",
+          "additionalProperties": false,
+          "required": [
+            "kind",
+            "anchor_qps",
+            "anchor_p99_ms",
+            "anchor_max_latency_ms"
+          ],
+          "properties": {
+            "kind": {
+              "const": "normal_load"
+            },
+            "anchor_qps": {
+              "type": "number",
+              "exclusiveMinimum": 0
+            },
+            "anchor_p99_ms": {
+              "type": "number",
+              "exclusiveMinimum": 0
+            },
+            "anchor_max_latency_ms": {
+              "type": "number",
+              "exclusiveMinimum": 0
+            }
+          }
+        },
+        {
+          "type": "object",
+          "additionalProperties": false,
+          "required": [
+            "kind",
+            "error_kinds"
+          ],
+          "properties": {
+            "kind": {
+              "const": "overload_120pct"
+            },
+            "error_kinds": {
+              "type": "array",
+              "uniqueItems": true,
+              "items": {
+                "type": "string",
+                "minLength": 1,
+                "pattern": "^\\S(?:.*\\S)?$"
+              }
+            }
+          }
+        },
+        {
+          "type": "object",
+          "additionalProperties": false,
+          "required": [
+            "kind",
+            "pause_ms",
+            "resume_drain_ms"
+          ],
+          "properties": {
+            "kind": {
+              "const": "storage_gate"
+            },
+            "pause_ms": {
+              "type": "integer",
+              "minimum": 0
+            },
+            "resume_drain_ms": {
+              "type": "integer",
+              "minimum": 0
+            }
+          }
+        },
+        {
+          "type": "object",
+          "additionalProperties": false,
+          "required": [
+            "kind",
+            "drain_ms"
+          ],
+          "properties": {
+            "kind": {
+              "const": "shutdown"
+            },
+            "drain_ms": {
+              "type": "integer",
+              "minimum": 0
+            }
+          }
+        },
+        {
+          "type": "object",
+          "additionalProperties": false,
+          "required": [
+            "kind",
+            "new_conn_ms",
+            "ping_ms"
+          ],
+          "properties": {
+            "kind": {
+              "const": "slow_storage"
+            },
+            "new_conn_ms": {
+              "type": "integer",
+              "minimum": 0
+            },
+            "ping_ms": {
+              "type": "integer",
+              "minimum": 0
+            }
+          }
+        }
+      ]
+    }
   }
 }