Complete Tracks by Length (nee. track accuracy over time)

docs/source/index.md

@@ -38,6 +38,7 @@ metrics/track_overlap
 metrics/divisions
 metrics/chota
 metrics/complete_tracks
+metrics/lineage-accuracy
 ```
 
 ```{toctree}

docs/source/metrics/lineage-accuracy.md

@@ -0,0 +1,53 @@
+(lineage-accuracy-metrics)=
+# Tracklet and Lineage Accuracy Metrics
+
+Track accuracy over N frames measures what fraction of ground truth tracklet or lineage segments that span N frames are correctly reconstructed.
+For lineage segments, all branches are included - a segment is only correct if all branches are correct.
+Skip edges that span multiple frames count toward their actual frame difference. For example, a skip edge from t=0 to t=3 contributes a segment of size 3, not size 1.
+
+A segment is counted as correct if there are no errors anywhere in it. In our current implementation, this includes checking:
+- The starting node is a true positive
+- All edges are true positives (edge TP implies endpoint nodes are TP)
+
+Important counting rules:
+- Isolated nodes (nodes with no outgoing edges) are NOT counted as segments
+- Ground truth tracks shorter than window size N do contribute 1 to the total for window N
+
+In the extreme case where N equals the number of frames in the dataset, this is the same as the CTC-BIO [Complete Tracks](complete-tracks).
+
+Tracklet accuracy over N frames is the same, but counts each tracklet (segment between divisions) independently rather than full lineages. Division edges are not included in this version of the metric.
+
+## Usage
+
+```python
+from traccuracy.metrics import TrackAccuracyOverTime
+from traccuracy.matchers import IOUMatcher
+
+# Match ground truth and predicted tracking graphs
+matcher = IOUMatcher(iou_threshold=0.5)
+matched = matcher.compute_mapping(gt_graph, pred_graph)
+
+# Compute lineage accuracy over windows spanning 1-50 frames
+metric = TrackAccuracyOverTime(max_window=50, lineages=True, error_type="basic")
+result = metric.compute(matched)
+
+# Results contain three lists (index 0 = window 1, index 1 = window 2, etc.):
+#   "correct" - number of correctly reconstructed segments at each window size
+#   "total"   - total number of GT segments at each window size
+#   "accuracy"- correct/total (or NaN if total is 0)
+print(f"Window 1: {result.results['accuracy'][0]:.2%}")
+print(f"Window 10: {result.results['accuracy'][9]:.2%}")
+print(f"Window 50: {result.results['accuracy'][49]:.2%}")
+
+# For tracklet accuracy (segments between divisions):
+tracklet_metric = TrackAccuracyOverTime(max_window=50, lineages=False)
+tracklet_result = tracklet_metric.compute(matched)
+```
+
+## References
+
+This metric is based on the track accuracy evaluation from linajea:
+
+- **Paper:** Malin-Mayor, C., Hirsch, P., Guber, L. et al. Automated reconstruction of whole-embryo cell lineages by learning from sparse annotations. *Nat Biotechnol* 41, 44–49 (2023). [https://doi.org/10.1038/s41587-022-01427-7](https://www.nature.com/articles/s41587-022-01427-7)
+
+- **Original implementation:** [`linajea.evaluation.evaluator.get_perfect_segments`](https://github.com/funkelab/linajea/blob/master/linajea/evaluation/evaluator.py#L409)

docs/source/metrics/metrics.md

@@ -26,4 +26,5 @@ Unless otherwise noted, metrics are written assuming that you have dense ground
 | [Cell Cycle Accuracy (CCA)](cca)| `one-to-one`, `many-to-one`| One of the CTC-Bio metrics. Measures the ability of a method to identify a distribution of cell cycle lengths that matches the distribution present in the ground truth.|
 | [Complete Tracklets and Lineages](complete-tracks) |  `one-to-one`, `many-to-one`| Extends the "Complete Tracks" from the CTC-Bio metrics. Measures the fraction of tracklets and lineages that are fully correct in the prediction.|
 | [Track Overlap Metrics](track-overlap-metrics): Track Purity (TP), Target Effectiveness (TE), Track Fractions (TF) | `one-to-one`, `many-to-one` , `one-to-many`| A set of metrics that compute the maximum overlap for each track, where track is defined as the region between divisions. Target effectiveness (TE) measures how much of each ground truth track is covered by the most overlapping predicted track, weighted by track length. Track Purity (TP) is the inverse of TE, and Track Fractions (TF) is the unwighted average of TE. |
+| [Lineage and Tracklet Accuracy](lineage-accuracy-metrics) | `one-to-one`, `many-to-one` | A set of metrics that computes the fraction of tracklets or lineages that are fully correct over a sliding window of N frames. |
 | [Cell-specific Higher Order Tracking Accuracy (CHOTA)](chota-metric) |`one-to-one`, `many-to-one`, `one-to-many`, `many-to-many` | A metric between 0 and 1 that unifies local correctness, global coherence, and lineage tracking. Higher scores are better.| 

src/traccuracy/matchers/_iou.py

@@ -134,6 +134,7 @@ def _construct_time_to_seg_id_map(
     Raises:
         AssertionError: If two nodes in a time frame have the same segmentation_id
     """
+    assert graph.label_key is not None
     time_to_seg_id_map: dict[int, dict[Hashable, Hashable]] = {}
     for node_id, data in graph.nodes(data=True):
         time = data[graph.frame_key]

src/traccuracy/metrics/__init__.py

@@ -5,6 +5,7 @@
 from ._ctc import AOGMMetrics, CTCMetrics
 from ._divisions import DivisionMetrics
 from ._results import Results
+from ._track_accuracy_over_time import TrackAccuracyOverTime
 from ._track_overlap import TrackOverlapMetrics
 
 __all__ = [
@@ -16,5 +17,6 @@
     "CompleteTracks",
     "DivisionMetrics",
     "Results",
+    "TrackAccuracyOverTime",
     "TrackOverlapMetrics",
 ]

src/traccuracy/metrics/_complete_tracks.py

@@ -1,17 +1,16 @@
 from __future__ import annotations
 
-import itertools
 import warnings
 from typing import TYPE_CHECKING
 
 import numpy as np
 
-from traccuracy._tracking_graph import EdgeFlag, NodeFlag
 from traccuracy.track_errors._basic import classify_basic_errors
 from traccuracy.track_errors._ctc import evaluate_ctc_events
 from traccuracy.track_errors._divisions import evaluate_division_events
 
 from ._base import Metric
+from ._compute_track_accuracy import _has_fp_division, _is_edge_correct, _is_node_correct
 
 if TYPE_CHECKING:
     from collections.abc import Hashable
@@ -55,6 +54,7 @@ def __init__(self, error_type: str = "basic"):
         if error_type not in ["ctc", "basic"]:
             raise ValueError(f"Unrecognized error type {error_type}. Should be 'ctc' or 'basic'")
         self.error_type = error_type
+        self.is_ctc = error_type == "ctc"
 
     def _compute(
         self, matched: Matched, relax_skips_gt: bool = False, relax_skips_pred: bool = False
@@ -135,11 +135,12 @@ def _compute(
                 for tracklet_start in tracklet_starts
             ]
             div_edges_correct = [
-                self._check_gt_edge_correct(
+                _is_edge_correct(
                     div_edge,
                     matched,
-                    relax_skips_gt=relax_skips_gt,
-                    relax_skips_pred=relax_skips_pred,
+                    self.is_ctc,
+                    relax_skips_gt,
+                    relax_skips_pred,
                 )
                 for div_edge in div_edges
             ]
@@ -162,86 +163,26 @@ def _compute(
             else np.nan,
         }
 
+    def _check_node_and_div(self, node: Hashable, matched: Matched, relax_skips_pred: bool) -> bool:
+        """Check if a GT node is correct and has no false positive division."""
+        return _is_node_correct(
+            node, matched, self.is_ctc, relax_skips_pred
+        ) and not _has_fp_division(node, matched, self.is_ctc)
+
     def _check_tracklet_correct(
         self, start_node: Hashable, matched: Matched, relax_skips_gt: bool, relax_skips_pred: bool
     ) -> bool:
-        if not self._check_gt_node_correct(start_node, matched, relax_skips_pred=relax_skips_pred):
+        if not self._check_node_and_div(start_node, matched, relax_skips_pred):
             return False
         out_edges = list(matched.gt_graph.graph.out_edges(start_node))
         while len(out_edges) == 1:
             out_edge = out_edges[0]
-            if not self._check_gt_edge_correct(out_edge, matched, relax_skips_gt, relax_skips_pred):
+            if not _is_edge_correct(
+                out_edge, matched, self.is_ctc, relax_skips_gt, relax_skips_pred
+            ):
                 return False
             curr_node = out_edge[1]
-            if not self._check_gt_node_correct(
-                curr_node, matched, relax_skips_pred=relax_skips_pred
-            ):
+            if not self._check_node_and_div(curr_node, matched, relax_skips_pred):
                 return False
             out_edges = list(matched.gt_graph.graph.out_edges(curr_node))
         return True
-
-    def _check_gt_node_correct(
-        self, node: Hashable, matched: Matched, relax_skips_pred: bool
-    ) -> bool:
-        node_tp = NodeFlag.TRUE_POS if self.error_type == "basic" else NodeFlag.CTC_TRUE_POS
-        gt_track = matched.gt_graph
-        # check if this gt node is a true pos
-        if node_tp in gt_track.nodes[node]:
-            # check if it is not matched to a FP-DIV, if applicable
-            if self.error_type == "basic":
-                matched_nodes = matched.get_gt_pred_matches(node)
-                for pred_node in matched_nodes:
-                    if NodeFlag.FP_DIV in matched.pred_graph.nodes[pred_node]:
-                        return False
-            return True
-        else:
-            # if skip edges are relaxed, check if the node is between skip tps
-            # (enough to check that one prev edge is a skip TP)
-            if relax_skips_pred:
-                for prev_edge in gt_track.graph.in_edges(node):
-                    if EdgeFlag.SKIP_TRUE_POS in gt_track.edges[prev_edge]:
-                        return True
-        # it's not a TP or between skip edges, so it's just wrong
-        return False
-
-    def _check_gt_edge_correct(
-        self,
-        edge: tuple[Hashable, Hashable],
-        matched: Matched,
-        relax_skips_gt: bool,
-        relax_skips_pred: bool,
-    ) -> bool:
-        gt_track = matched.gt_graph
-        pred_track = matched.pred_graph
-        edge_data = gt_track.edges[edge]
-        # check if it is a TP
-        if self.error_type == "ctc":
-            # the ctc errors don't annotate edge TPs, so instead we check for absence of
-            # all the error types. Wrong semantic are only annotated on the pred graph,
-            # so we need to find the matched edge and check it
-            tp = True
-            if EdgeFlag.CTC_FALSE_NEG in edge_data:
-                tp = False
-            else:
-                matched_sources = matched.get_gt_pred_matches(edge[0])
-                matched_targets = matched.get_gt_pred_matches(edge[1])
-                matched_edges = [
-                    (source, target)
-                    for source, target in itertools.product(matched_sources, matched_targets)
-                    if pred_track.graph.has_edge(source, target)
-                ]
-                for matched_edge in matched_edges:
-                    if EdgeFlag.WRONG_SEMANTIC in pred_track.graph.edges[matched_edge]:
-                        tp = False
-                        break
-            if tp:
-                return True
-        else:
-            if EdgeFlag.TRUE_POS in edge_data:
-                return True
-        is_skip_edge = gt_track.is_skip_edge(edge)
-        if is_skip_edge and relax_skips_gt and EdgeFlag.SKIP_TRUE_POS in edge_data:
-            return True
-        if (not is_skip_edge) and relax_skips_pred and EdgeFlag.SKIP_TRUE_POS in edge_data:
-            return True
-        return False