docs: audit remote-only liveness evidence

Author: grandchildrice

docs/paper/ef_severity_analysis.md

@@ -158,6 +158,9 @@ records. All seven records containing both vocabulary classes were also
 reviewed: only one directly evidenced a single-input failure, but it was fixed
 before mainnet, and zero established High. See
 [`liveness_candidate_audit.md`](liveness_candidate_audit.md).
+The three remote-term-only rows were also reviewed; none connects a single
+remote input to an availability failure. In total, 15 targeted liveness rows
+(14 distinct artifacts) have been source-reviewed with zero confirmed High.
 
 ## Generated evidence
 
@@ -173,3 +176,4 @@ before mainnet, and zero established High. See
 - [`tables/liveness_candidate_triage.csv`](tables/liveness_candidate_triage.csv)
 - [`tables/liveness_test_label_audit.csv`](tables/liveness_test_label_audit.csv)
 - [`tables/liveness_both_terms_audit.csv`](tables/liveness_both_terms_audit.csv)
+- [`tables/liveness_remote_only_audit.csv`](tables/liveness_remote_only_audit.csv)

docs/paper/liveness_candidate_audit.md

@@ -129,7 +129,39 @@ selecting records whose source text contains both vocabulary classes did not
 produce any confirmed High labels. Keyword co-occurrence improves review
 priority, but does not replace trigger, deployment, or threshold validation.
 
-## 6. Paper contribution
+## 6. Audit of the three remote-term-only records
+
+The three rows with remote-trigger vocabulary but no availability vocabulary
+were reviewed next.
+
+| Record | Remote evidence | Availability evidence | Result |
+|---|---|---|---|
+| Geth `GetBlockHeaders` overflow | Crafted `Skip` request is explicit | No crash, panic, exhaustion, or unavailability stated | Remote protocol logic defect, not demonstrated DoS |
+| Lighthouse PeerDAS persistence race | No attacker control stated | Data columns fail to persist | Predeployment internal race |
+| Lighthouse gossip amplification | Remote gossip messages are explicit | Amplification after a repeated >256-message cycle; no crash | Multi-message resource amplification, pre-mainnet |
+
+The Geth record demonstrates why “remote” alone is insufficient: one crafted
+request reaches defective arithmetic, but the source and regression tests show
+incorrect traversal handling rather than an availability failure.
+
+The gossip record merged on 2020-01-31, before Beacon Chain mainnet, explicitly
+discusses non-malicious validators, and requires a repeated cycle of more than
+256 unique messages. The LLM rationale changes this into a remote attacker
+crashing the client, which is not supported by the source.
+
+Results:
+
+- attacker-controlled input evidenced: 1/3;
+- remote input evidenced: 2/3;
+- single-input availability failure evidenced: 0/3;
+- confirmed exact High: 0/3.
+
+Across the three completed targeted strata—five `label=test` rows, seven
+both-term rows, and three remote-only rows—15 candidate rows representing 14
+distinct artifacts have now been source-reviewed. None establishes exact High.
+This remains a deliberately selected audit, not an unbiased precision estimate.
+
+## 7. Paper contribution
 
 > In 89 estimated liveness-DoS High rows, source text mentions availability in
 > 67 cases but a remote trigger in only ten, and only seven mention both.
@@ -144,14 +176,12 @@ priority, but does not replace trigger, deployment, or threshold validation.
 This is not a precision estimate for all 89 rows because the first audited
 stratum was deliberately selected for suspicious protocol labels.
 
-## 7. Next audit strata
+## 8. Next audit strata
 
 The remaining queue should be reviewed in this order:
 
-1. the three remote-term-only records, where the claimed availability outcome
-   needs verification;
-2. the 19 records containing neither term class;
-3. the 60 availability-term-only records, stratified by client, blast radius,
+1. the 19 records containing neither term class;
+2. the 60 availability-term-only records, stratified by client, blast radius,
    and deployed/predeployment state.
 
 ## Generated evidence
@@ -160,3 +190,4 @@ The remaining queue should be reviewed in this order:
 - [`tables/liveness_candidate_triage.csv`](tables/liveness_candidate_triage.csv)
 - [`tables/liveness_test_label_audit.csv`](tables/liveness_test_label_audit.csv)
 - [`tables/liveness_both_terms_audit.csv`](tables/liveness_both_terms_audit.csv)
+- [`tables/liveness_remote_only_audit.csv`](tables/liveness_remote_only_audit.csv)

docs/paper/tables/liveness_candidate_summary.csv

@@ -15,3 +15,6 @@ test_label_confirmed_high,0,5,none establish the EF >33% threshold,0.0
 both_terms_audited,7,89,all rows containing both availability and remote-trigger terms,7.865
 both_terms_single_input_failure_evidenced,1,7,one source directly states a handcrafted transaction triggers panic,14.286
 both_terms_confirmed_high,0,7,none establish deployed >33% impact at the fix date,0.0
+remote_only_audited,3,89,all rows containing remote-trigger but no availability vocabulary,3.371
+remote_only_single_input_failure_evidenced,0,3,none connect one remote input to an availability failure,0.0
+remote_only_confirmed_high,0,3,none establish the EF >33% threshold,0.0

docs/paper/tables/liveness_remote_only_audit.csv

@@ -0,0 +1,4 @@
+id,source_platform,title,source_url,severity_estimated,severity_analysis_label,audit_verdict,availability_evidence,attacker_controlled_input_evidenced,remote_input_evidenced,single_input_failure_evidenced,deployment_context,confirmed_high,audit_reason,blast_radius,authority_tier,root_cause,attack_path,label
+lighthouse:networking:PR#832,lighthouse,Prevent duplicate gossipsub loops/message amplification,https://github.com/sigp/lighthouse/pull/832,High,tier-uncertain,pre_mainnet_multi_message_amplification_no_crash,resource_amplification,False,True,False,pre_mainnet,False,"The source describes duplicate gossip loops after more than 256 unique messages and explicitly discusses non-malicious validators. It does not report a crash, requires a repeated multi-message cycle, and merged on 2020-01-31 before Beacon Chain mainnet.",client_specific,B_corroborated,missing_input_validation,malicious_attestation,beacon-chain:attestation
+6fad56ffd2f16be8,lighthouse,Fix data columns not persisting for PeerDAS due to a `getBlobs` race condition (#6756),https://github.com/sigp/lighthouse/commit/dd7591f7123dfe072631c0deb0abc1b78cc82733,High,tier-uncertain,predeployment_internal_race_no_remote_attack,functional_failure,False,False,False,predeployment,False,"The source establishes a PeerDAS getBlobs race that prevents data-column persistence. It does not identify attacker control, one-message failure, or a deployed network impact; the commit dates to 2025-01-15.",client_specific,B_corroborated,integer_overflow_underflow,crafted_state,data-availability-sampling
+e84e13f52af9dacd,geth,eth/protocols/eth: Fix overflows in  `GetBlockHeaders`,https://github.com/ethereum/go-ethereum/pull/31522,High,tier-uncertain,remote_protocol_logic_defect_no_availability_outcome,no_failure,True,True,False,deployed_code,False,"The source explicitly identifies crafted GetBlockHeaders Skip values and fixes overflow/underflow traversal. It does not report crash, panic, resource exhaustion, or node unavailability from the single request.",client_specific,B_corroborated,integer_overflow_underflow,malformed_input,p2p

scripts/audit_liveness_candidates.py

@@ -136,6 +136,43 @@
     ),
 }
 
+REMOTE_ONLY_DECISIONS = {
+    "e84e13f52af9dacd": (
+        "remote_protocol_logic_defect_no_availability_outcome",
+        "no_failure",
+        True,
+        True,
+        False,
+        "deployed_code",
+        "The source explicitly identifies crafted GetBlockHeaders Skip values and fixes "
+        "overflow/underflow traversal. It does not report crash, panic, resource "
+        "exhaustion, or node unavailability from the single request.",
+    ),
+    "6fad56ffd2f16be8": (
+        "predeployment_internal_race_no_remote_attack",
+        "functional_failure",
+        False,
+        False,
+        False,
+        "predeployment",
+        "The source establishes a PeerDAS getBlobs race that prevents data-column "
+        "persistence. It does not identify attacker control, one-message failure, or a "
+        "deployed network impact; the commit dates to 2025-01-15.",
+    ),
+    "lighthouse:networking:PR#832": (
+        "pre_mainnet_multi_message_amplification_no_crash",
+        "resource_amplification",
+        False,
+        True,
+        False,
+        "pre_mainnet",
+        "The source describes duplicate gossip loops after more than 256 unique messages "
+        "and explicitly discusses non-malicious validators. It does not report a crash, "
+        "requires a repeated multi-message cycle, and merged on 2020-01-31 before "
+        "Beacon Chain mainnet.",
+    ),
+}
+
 
 def pct(count: int, denominator: int) -> float:
     return round(100 * count / denominator, 3) if denominator else 0.0
@@ -329,6 +366,64 @@ def main() -> int:
         args.output_dir / "liveness_both_terms_audit.csv", index=False
     )
 
+    remote_only_rows = candidates[
+        candidates["evidence_screen"].eq("remote_term_only")
+    ].copy()
+    actual = set(remote_only_rows["id"])
+    expected = set(REMOTE_ONLY_DECISIONS)
+    if actual != expected:
+        raise ValueError(
+            f"Remote-only decisions mismatch; "
+            f"missing={sorted(actual - expected)}, extra={sorted(expected - actual)}"
+        )
+    remote_only_rows["audit_verdict"] = remote_only_rows["id"].map(
+        lambda row_id: REMOTE_ONLY_DECISIONS[row_id][0]
+    )
+    remote_only_rows["availability_evidence"] = remote_only_rows["id"].map(
+        lambda row_id: REMOTE_ONLY_DECISIONS[row_id][1]
+    )
+    remote_only_rows["attacker_controlled_input_evidenced"] = remote_only_rows[
+        "id"
+    ].map(lambda row_id: REMOTE_ONLY_DECISIONS[row_id][2])
+    remote_only_rows["remote_input_evidenced"] = remote_only_rows["id"].map(
+        lambda row_id: REMOTE_ONLY_DECISIONS[row_id][3]
+    )
+    remote_only_rows["single_input_failure_evidenced"] = remote_only_rows[
+        "id"
+    ].map(lambda row_id: REMOTE_ONLY_DECISIONS[row_id][4])
+    remote_only_rows["deployment_context"] = remote_only_rows["id"].map(
+        lambda row_id: REMOTE_ONLY_DECISIONS[row_id][5]
+    )
+    remote_only_rows["audit_reason"] = remote_only_rows["id"].map(
+        lambda row_id: REMOTE_ONLY_DECISIONS[row_id][6]
+    )
+    remote_only_rows["confirmed_high"] = False
+    remote_only_rows[
+        [
+            "id",
+            "source_platform",
+            "title",
+            "source_url",
+            "severity_estimated",
+            "severity_analysis_label",
+            "audit_verdict",
+            "availability_evidence",
+            "attacker_controlled_input_evidenced",
+            "remote_input_evidenced",
+            "single_input_failure_evidenced",
+            "deployment_context",
+            "confirmed_high",
+            "audit_reason",
+            "blast_radius",
+            "authority_tier",
+            "root_cause",
+            "attack_path",
+            "label",
+        ]
+    ].sort_values(["audit_verdict", "id"]).to_csv(
+        args.output_dir / "liveness_remote_only_audit.csv", index=False
+    )
+
     exact_duplicate_rows = int(candidates["exact_diff_duplicate"].sum())
     duplicate_groups = len(duplicate_fingerprints)
     distinct_artifacts = len(candidates) - (exact_duplicate_rows - duplicate_groups)
@@ -434,6 +529,24 @@ def main() -> int:
             len(both_rows),
             "none establish deployed >33% impact at the fix date",
         ),
+        (
+            "remote_only_audited",
+            len(remote_only_rows),
+            len(candidates),
+            "all rows containing remote-trigger but no availability vocabulary",
+        ),
+        (
+            "remote_only_single_input_failure_evidenced",
+            int(remote_only_rows["single_input_failure_evidenced"].sum()),
+            len(remote_only_rows),
+            "none connect one remote input to an availability failure",
+        ),
+        (
+            "remote_only_confirmed_high",
+            int(remote_only_rows["confirmed_high"].sum()),
+            len(remote_only_rows),
+            "none establish the EF >33% threshold",
+        ),
     ]
     summary_frame = pd.DataFrame(
         summary, columns=["metric", "rows", "denominator", "definition"]

tests/test_security_dataset.py

@@ -60,6 +60,9 @@
 LIVENESS_BOTH_TERMS_AUDIT = (
     ROOT / "docs" / "paper" / "tables" / "liveness_both_terms_audit.csv"
 )
+LIVENESS_REMOTE_ONLY_AUDIT = (
+    ROOT / "docs" / "paper" / "tables" / "liveness_remote_only_audit.csv"
+)
 
 BOILERPLATE = re.compile(r"critical update required|urgency guidelines|high-urgency", re.I)
 REQUIRED_COLS = {
@@ -372,3 +375,18 @@ def test_liveness_candidate_screen_and_first_audit_stratum():
         "deployed_code": 2,
         "deployment_unclear": 1,
     }
+
+    remote_only = pd.read_csv(LIVENESS_REMOTE_ONLY_AUDIT)
+    assert len(remote_only) == 3
+    assert int(remote_only["attacker_controlled_input_evidenced"].sum()) == 1
+    assert int(remote_only["remote_input_evidenced"].sum()) == 2
+    assert not remote_only["single_input_failure_evidenced"].any()
+    assert not remote_only["confirmed_high"].any()
+    assert remote_only["availability_evidence"].value_counts().to_dict() == {
+        "no_failure": 1,
+        "functional_failure": 1,
+        "resource_amplification": 1,
+    }
+
+    audited_ids = set(audit["id"]) | set(both["id"]) | set(remote_only["id"])
+    assert len(audited_ids) == 15