Supporting lifted edges would enable better modeling of long range information across time. Lifted edges only provide a contribution to the cost, but not to the connectivity.
Motivation
We sometimes have knowledge about long-range connectivity across time in tracking problems. For example:
- Transformer-based tracking models like the GTR predict affinities between detections across multiple timeframes.
- In heterogeneous cell populations (e.g. mixed cell lines or different cells with specific stains) only the same kinds should be connected via tracks. Since the attribution is probabilistic (e.g. cell type classification) and may not be available in each frame (e.g. cell line classification not possible during divisions) long-range edges area a good way to model this.
Details
The concept of edges was introduced for the multicut graph partitioning problem in Hornakova et al., and is explained in a less formal (and more approachable) way in Chapter 2.26 of my thesis.
The key idea behind lifted edges is that they only contribute to the cost of the solution, but not to the connectivity. This fits the problems described in the motivation well, because we want nodes connected by an active lifted edge to be also connected by "local" edges, i.e. edges that connect detections in adjacent frames.
I made this figure to illustrate the concept for tracking and the difference to normal edges:
In the optimal solution for a normal edge the nodes are not connected by a local path across time, because the edge induces conductivity. In the optimal solution for the lifted edge the nodes are connected, because the lifted edge only contributes to the cost of the solution and does not induce conductivity.
(Note: I assume that negative costs are attractive here, which I think matches the convention of motile.)
Implementation
I am not familiar with the implementation of motile / ilpy yet, but would be happy to help in the implementation of this. I already discussed this with @funkey and @ben, who both expressed interest in this feature and in helping to implement it.
It would probably best if you could give some high-level ideas on how to implement this @bentaculum or @funkey and then we can think about how to tackle this in more detail.
Supporting lifted edges would enable better modeling of long range information across time. Lifted edges only provide a contribution to the cost, but not to the connectivity.
Motivation
We sometimes have knowledge about long-range connectivity across time in tracking problems. For example:
Details
The concept of edges was introduced for the multicut graph partitioning problem in Hornakova et al., and is explained in a less formal (and more approachable) way in Chapter 2.26 of my thesis.
The key idea behind lifted edges is that they only contribute to the cost of the solution, but not to the connectivity. This fits the problems described in the motivation well, because we want nodes connected by an active lifted edge to be also connected by "local" edges, i.e. edges that connect detections in adjacent frames.
I made this figure to illustrate the concept for tracking and the difference to normal edges:
In the optimal solution for a normal edge the nodes are not connected by a local path across time, because the edge induces conductivity. In the optimal solution for the lifted edge the nodes are connected, because the lifted edge only contributes to the cost of the solution and does not induce conductivity.
(Note: I assume that negative costs are attractive here, which I think matches the convention of
motile.)Implementation
I am not familiar with the implementation of
motile/ilpyyet, but would be happy to help in the implementation of this. I already discussed this with @funkey and @ben, who both expressed interest in this feature and in helping to implement it.It would probably best if you could give some high-level ideas on how to implement this @bentaculum or @funkey and then we can think about how to tackle this in more detail.