fix(kernel validation): represent exact-agreement PSNR without rejecting infinity

[morluto]

Severity

High numerical-evidence compatibility — the strict kernel-validation contract cannot represent the mathematically correct PSNR result for exact agreement, so a producer must either reject its strongest pass or replace it with an arbitrary finite cap

Validated against main at 0a60b288342cd96e2608302fa9fd2917e59d7e58.

Audit only; no implementation change is included.

Summary

flameox.kernel-validation.v1 supports psnr as a higher-is-better numerical validation metric.

KernelValidationMetric applies the same finite-number rule to every metric:

_require_finite(self.value, "metric value")
_require_finite(self.threshold, "metric threshold")

For PSNR, exact agreement has:

MSE = 0
PSNR = 10 log10(peak^2 / MSE) = +infinity

This is not malformed data. It is the natural extended-real result of the declared metric when prediction and reference are identical.

The current schema therefore rejects a valid exact-agreement document such as:

{
  "name": "psnr",
  "value": "Infinity",
  "comparator": ">=",
  "threshold": 60.0,
  "unit": "dB",
  "status": "pass"
}

JSON itself cannot encode IEEE infinity as a standards-compliant number, and the Pydantic model also rejects non-finite floats. A producer is forced to:

• omit PSNR and mark it inconclusive/unsupported despite exact equality;
• invent a finite sentinel such as 100 dB;
• clamp to an implementation-specific maximum;
• or fail import.

All of those weaken or distort the retained numerical evidence.

Metric definition

The standard formula is:

PSNR(I, J) = 10 * log10(MAX^2 / MSE(I, J))

where:

MSE(I, J) = average((I - J)^2)

References:

• TorchMetrics PSNR documentation: https://lightning.ai/docs/torchmetrics/v1.8.2/image/peak_signal_noise_ratio.html
• MathWorks PSNR algorithm: https://www.mathworks.com/help/images/ref/psnr.html

For exact agreement I == J:

MSE(I, J) = 0
MAX^2 / 0 = +infinity
log10(+infinity) = +infinity

Thus the PSNR ordering result is unambiguous even though the finite scalar representation is not.

Direct code evidence

MetricName includes:

"psnr"

and _HIGHER_IS_BETTER includes it.

KernelValidationMetric.coherent_result() then executes:

_require_finite(self.value, "metric value")
_require_finite(self.threshold, "metric threshold")
...
passed = self.value >= self.threshold

https://github.com/morluto/flameox/blob/0a60b288342cd96e2608302fa9fd2917e59d7e58/src/flameox/adapters/kernel_validation.py

_require_finite() is:

if value is not None and not math.isfinite(value):
    raise ValueError(...)

There is no PSNR-specific exact-agreement variant or finite representation policy.

The physical kernel_validation_metrics.value field is float64, so the normalized table also lacks a typed exact/unbounded representation.

Deterministic counterexample

Take one validation output where every tensor element matches the reference exactly:

actual = expected

Then:

max_abs_error = 0
mse = 0
rmse = 0
cosine_similarity = 1  (when defined)
PSNR = +infinity

A producer can validly emit the first four supported metric values, but cannot emit the fifth under Flameox's contract.

Suppose the preregistered validation policy is:

PSNR >= 60 dB

The result should be a decisive pass stronger than any finite value. Current allowed states are:

Attempt A: encode float("inf")

Rejected by _require_finite and by strict JSON compatibility.

Attempt B: encode no value and status inconclusive

Accepted structurally, but scientifically false: the result is conclusive and exceeds the threshold.

Attempt C: encode an arbitrary cap such as 100.0

Accepted but no longer observed evidence. A different producer may choose 200, 300, or the maximum finite dtype value; cross-run comparison becomes implementation-dependent.

Attempt D: omit PSNR

The artifact no longer satisfies a preregistered PSNR validation contract despite exact agreement.

There is no faithful representation.

Why this should not be solved by permitting arbitrary non-finite floats

NaN and negative infinity must remain invalid for ordinary validation metrics. The correct design is a typed semantic variant, not turning off finite validation globally.

For example, a metric value may distinguish:

finite(value)
positive_infinity(reason="zero_mse_exact_agreement")
unavailable/unsupported/inconclusive

The positive-infinity variant must be legal only for metrics/profiles where the mathematical definition permits it and the producer supplies the evidence needed to establish the condition.

Additional PSNR contract gap

The schema stores only:

metric name
value
unit
threshold

PSNR also depends on the signal/data range (MAX or data_range). Two producers can compute different dB values from the same error with different range assumptions while both use unit="dB".

The exact range/profile belongs in the metric identity and comparison compatibility. This is relevant to #225's exact metric-contract comparison work.

Impact

• Exact numerical equality cannot be preserved through a supported metric.
• Producers may introduce arbitrary caps that look observed and comparable.
• Confirmatory PSNR thresholds can become inconclusive or unsupported for the strongest possible result.
• Cross-provider and cross-run results can differ solely because of sentinel convention.
• Later comparison code cannot distinguish true finite 100 dB from “infinite, capped at 100.”

Required invariant

Every admitted metric must have a total, lossless representation over its mathematically valid output domain.

For PSNR:

MSE == 0
=> exact positive-infinity semantic value
=> pass for every finite lower threshold

The representation must not require an arbitrary finite approximation.

Proposed direction

1. Add a tagged metric-value domain

Conceptually:

finite
positive_infinity
unavailable
unsupported
inconclusive

Bind legal variants per metric definition.

2. Bind the PSNR profile

Record at least:

data/peak range
base
reduction/case semantics
reference identity
zero-MSE convention

so finite and infinite values are comparable only under the same definition.

3. Preserve exact normalized evidence

Use an explicit value kind in kernel_validation_metrics rather than overloading nullable float64. Queries should order +infinity above every finite value without converting it to a sentinel.

4. Keep strict rejection elsewhere

NaN, negative infinity, and impossible infinity variants for MSE/RMSE/max error/cosine similarity remain invalid.

Acceptance criteria

• Exact-agreement PSNR is representable without an arbitrary finite cap.
• A zero-MSE PSNR result passes every finite >= threshold.
• NaN and illegal infinity values remain rejected.
• The JSON and normalized evidence schemas use an explicit tagged value, not non-standard JSON numbers.
• PSNR data range/profile participates in metric identity and comparison compatibility.
• Tests cover exact equality, very high finite PSNR, zero/nonzero MSE boundary, producer sentinel attempts, and changed data range.
• Compare compatible kernel-validation metrics across experiment treatments #225 comparisons preserve and correctly order the exact-agreement variant.
• Historical finite-only PSNR rows have an explicit compatibility policy.

Relationship to existing issues

• Compare compatible kernel-validation metrics across experiment treatments #225 requires exact kernel-validation metric identity and cross-treatment comparison.
• fix(evidence): reserve common provenance columns and remove duplicate extractor-version field #207 concerns authoritative evidence schema ownership.
• [Architecture] Ban non-validating mutation of ContractModel state #158 protects typed metric variants after construction.
• [Architecture] Add an explicit migration contract for every persisted schema #157 owns physical/schema migration.