fix(accelerator): report added and removed launch regions instead of intersecting them away

[morluto]

Severity

High comparison correctness — a complete execution phase that appears or disappears can produce no comparison row at all, while the result reports available evidence

Validated against main at 0a60b288342cd96e2608302fa9fd2917e59d7e58.

Audit only; no implementation change is included.

Summary

AcceleratorRecipes.accelerator_launches() computes launch-structure deltas only for region names present in both inputs:

for region in sorted(primary_by_region.keys() & comparison_by_region.keys())

Regions present only in the baseline or only in the comparison input are silently absent from comparisons.

The result still returns both complete region lists and can set:

evidence.status = available

when each side independently contains runtime launches and accelerator kernels.

Therefore a major structural change such as:

baseline:  decode region exists
candidate: prefill region exists

can yield:

comparisons = ()
coverage complete on both sides
evidence available

There is no added/removed-region disposition or limitation saying the region sets do not overlap.

Direct code evidence

After independently summarizing both inputs, the code builds:

primary_by_region = {item.region: item for item in regions}
comparison_by_region = {item.region: item for item in comparison_regions}
comparisons = tuple(
    AcceleratorLaunchComparison(...)
    for region in sorted(primary_by_region.keys() & comparison_by_region.keys())
)

flameox/src/flameox/analysis/recipe_accelerator.py

Lines 60 to 120 in 0a60b28

	if cursor is not None and start > cursor:
	gaps.append(start - cursor)
	cursor = max(cursor if cursor is not None else start, start + duration)
	return gaps
	class AcceleratorRecipes(RecipeContext):
	def accelerator_launches(
	self,
	input_id: str,
	*,
	comparison_input_id: str | None = None,
	phase: str | None = None,
	limit: int | None = None,
	corpus_commit_id: str | None = None,
	) -> AcceleratorLaunchAnalysisResult:
	corpus_commit_id = self._pinned_commit_id(corpus_commit_id)
	bounded = self._limit(limit)
	with self._open_snapshot(corpus_commit_id) as snapshot:
	regions, total, coverage, limitations = self._accelerator_launch_regions(
	snapshot,
	input_id,
	phase=phase,
	limit=bounded,
	)
	comparison_regions: tuple[AcceleratorLaunchRegion, ...] = ()
	comparison_coverage: dict[str, bool] | None = None
	if comparison_input_id is not None:
	(
	comparison_regions,
	_,
	comparison_coverage,
	comparison_limitations,
	) = self._accelerator_launch_regions(
	snapshot,
	comparison_input_id,
	phase=phase,
	limit=bounded,
	)
	limitations = (
	*limitations,
	*(f"Comparison input: {item}" for item in comparison_limitations),
	)
	primary_by_region = {item.region: item for item in regions}
	comparison_by_region = {item.region: item for item in comparison_regions}
	comparisons = tuple(
	AcceleratorLaunchComparison(
	region=region,
	direct_launch_count_delta=(
	comparison_by_region[region].direct_launch_count
	- primary_by_region[region].direct_launch_count
	),
	graph_launch_count_delta=(
	comparison_by_region[region].graph_launch_count
	- primary_by_region[region].graph_launch_count
	),
	kernel_count_delta=(
	comparison_by_region[region].kernel_count
	- primary_by_region[region].kernel_count
	),
	kernel_duration_delta_ns=(

The public model contains only numeric deltas:

class AcceleratorLaunchComparison(ContractModel):
    region: str
    direct_launch_count_delta: int
    graph_launch_count_delta: int
    kernel_count_delta: int
    kernel_duration_delta_ns: int
    runtime_launch_gap_total_delta_ns: int
    idle_gap_total_delta_ns: int

There is no added, removed, baseline_only, candidate_only, or optional-side variant.

flameox/src/flameox/analysis/recipe_models.py

Lines 175 to 220 in 0a60b28

	class KernelNameCount(ContractModel):
	name: str
	count: int
	class AcceleratorStreamSummary(ContractModel):
	identity: str
	device: str | None = None
	context: str | None = None
	stream: str | None = None
	track_id: str | None = None
	kernel_count: int
	kernel_duration_ns: int
	idle_gap_count: int
	idle_gap_total_ns: int
	idle_gap_max_ns: int
	class AcceleratorLaunchRegion(ContractModel):
	region: str
	region_start_ns: int
	region_end_ns: int
	region_duration_ns: int
	selection_rule: str
	direct_launch_count: int
	direct_launch_duration_ns: int
	graph_launch_count: int
	graph_launch_duration_ns: int
	kernel_count: int
	kernel_duration_ns: int
	kernel_names: tuple[KernelNameCount, ...]
	kernel_names_truncated: bool
	correlated_kernel_count: int
	runtime_launch_gap_count: int
	runtime_launch_gap_total_ns: int
	runtime_launch_gap_max_ns: int
	idle_gap_count: int
	idle_gap_total_ns: int
	idle_gap_max_ns: int
	stream_count: int
	streams: tuple[AcceleratorStreamSummary, ...]
	streams_truncated: bool
	class AcceleratorLaunchComparison(ContractModel):
	region: str

Evidence status is determined from per-input track coverage, not comparison coverage of region identity:

available if primary and comparison each have runtime launches and kernels

An empty intersection does not make the comparison partial or inconclusive.

Deterministic counterexample

Publish two runs with complete trace evidence.

Baseline run B

One runtime launch and one correlated kernel, both in phase decode:

regions(B) = {decode}
coverage(B).runtime_launches = true
coverage(B).accelerator_kernels = true

Candidate run C

One runtime launch and one correlated kernel, both in phase prefill:

regions(C) = {prefill}
coverage(C).runtime_launches = true
coverage(C).accelerator_kernels = true

Call:

accelerator_launches(B, comparison_input_id=C)

Current set algebra:

keys(B) ∩ keys(C) = {decode} ∩ {prefill} = ∅

Therefore:

result.regions = (decode summary,)
result.comparison_regions = (prefill summary,)
result.comparisons = ()
result.evidence.status = available

The most important comparison fact—that decode disappeared and prefill appeared—is not represented.

A downstream agent that reads only comparisons can conclude that no launch-structure deltas were found. Even a careful consumer must manually rediscover the set difference despite the API claiming to provide the comparison.

Stronger same-name-filter counterexample

Without an explicit phase filter, a candidate may add a new unscoped/cycle region while retaining one common region:

baseline regions  = {decode}
candidate regions = {decode, warmup}

The API reports only the decode delta. The added warmup phase can contain thousands of kernels and dominate runtime, yet no comparison row or limitation mentions it.

The reverse removal case is symmetric.

Why treating missing as zero must be explicit

One possible descriptive comparison is:

baseline-only region -> candidate values treated as zero
candidate-only region -> baseline values treated as zero

But that choice is a semantic rule and should be represented as an explicit disposition. A missing region could also mean:

• genuinely added/removed behavior;
• phase annotation renamed;
• coverage/truncation difference;
• extraction/provider incompatibility;
• artifact/session selection mismatch under fix(accelerator): scope trace timing and correlation to one artifact/session #275.

The result needs to distinguish these cases rather than silently drop them.

Additional consequences

• Region-count changes are not summarized anywhere.
• total and returned describe only the primary input; there is no comparison total or truncated flag for the region comparison population.
• A comparison may be marked available when the region-name intersection is empty.
• Phase renaming can erase all deltas even when event counts/timing are otherwise identical.
• Added/removed cycle artifacts such as cycle_0001/decode disappear from the comparison.

Violated invariant

A set comparison over region identities must be total over the union:

regions_to_compare = baseline_regions ∪ candidate_regions

For every region identity, the result must state one of:

paired
baseline_only / removed
candidate_only / added
unpairable / incompatible

No region may disappear merely because it lacks a peer.

Proposed direction

1. Use a discriminated comparison variant

Conceptually:

PairedRegionDelta
AddedRegion
RemovedRegion
UnpairableRegion

Each carries the relevant full summary and coverage/provenance.

2. Validate region identity compatibility

After #275 scopes regions by artifact/session/provider/clock, define whether phase/cycle names are directly comparable. Renames or profile changes should be unpairable, not zero-filled automatically.

3. Make evidence status reflect comparison completeness

An empty or partial region match should yield a typed limitation/status even if each input separately has complete tracks.

4. Expose both totals/truncation states

Return baseline, candidate, paired, added, removed, and unpairable counts so bounded output cannot hide omitted regions.

Acceptance criteria

• Comparison operates over the union of region identities.
• Candidate-only regions are returned as added.
• Baseline-only regions are returned as removed.
• The decode versus prefill counterexample reports both changes.
• A new high-cost warmup region cannot be absent from the comparison result.
• Missing regions are not silently treated as zero without an explicit compatible-identity rule.
• Evidence status/limitations reflect unmatched or truncated comparison populations.
• Result counts identify paired, added, removed, and unpairable regions.
• Tests cover disjoint regions, partial overlap, phase rename, cycle addition/removal, per-side truncation, and one ordinary paired region.

Relationship to existing issues

• fix(accelerator): scope trace timing and correlation to one artifact/session #275 defines artifact/session-scoped region and correlation identity; that must land before reliable cross-input matching.
• fix(comparisons): reject ambiguous measurement dimensions instead of pooling them #249/fix(comparisons): require identical paired block coverage instead of analyzing the intersection silently #254 express the analogous requirement for complete measurement/block populations.
• fix(evidence): reserve common provenance columns and remove duplicate extractor-version field #207 governs the trace-event fields needed for stable region identity.
• [Architecture] Add an explicit migration contract for every persisted schema #157 owns result-schema migration.