Ek/docs buff#8
Conversation
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
There was a problem hiding this comment.
This is an excellent summary of the approach! I'm just leaving this as a "to do" for myself, because I need to add a description of LayerNormDetached and how that works.
|
|
||
| The locally linear property can be verified empirically. For any sample, the sum of feature contributions should equal the embedding exactly. In practice, Glass Box UMAP achieves reconstruction errors on the order of $3 \times 10^{-14}$—machine precision for 64-bit floating point. | ||
|
|
||
| Glass Box UMAP's attributions are exact, not estimates. |
There was a problem hiding this comment.
I plan to add some more disclaimers to this statement. Something like this:
However, glass-box UMAP is alos subject to all of the caveats that come with using the original UMAP. If the network does not learn an acceptable clustering of the data, the resulting feature attributions will be exact but uninformative about the underlying data. Preprocessing is entirely up to the user, and changing any steps from a previous run will almost certainly affect the feature attributions. Initializing with PCA or learning the UMAP embedding directly from the data will most likely produce different feature attributions. Ideally, the user should fit the same architecture several times with different random seeds and plot error bars for the features, and high-magnitude contributions with small error bars will be the primary features of interest. Fits with any change in the pipeline will produce different feature attributions.
|
|
||
| The key insight is that certain neural network architectures are *locally linear*: for any input, the network's output can be expressed exactly as a matrix multiplication of that input. Glass Box UMAP exploits this property by using PReLU activations and zero-bias linear layers, allowing us to compute the Jacobian of the embedding with respect to input features. Multiplying this Jacobian by the input gives exact feature contributions that sum precisely to the embedding coordinates. | ||
|
|
||
| Glass Box UMAP's feature attributions are mathematically exact, validated to near machine precision. This makes it possible to understand exactly which features drive the structure in your embeddings. |
There was a problem hiding this comment.
although i will also list caveats here as well!
| X_np = _to_numpy(X) | ||
| self._mean = X_np.mean(axis=0) | ||
| assert self._mean is not None | ||
| X_centered = X_np - self._mean |
There was a problem hiding this comment.
With gene expression data in AnnData objects, it is likely that users will prefer to do PCA within AnnData. In that case the mean should already be zero, but just something of which we should be aware.
james-golden-arcadia
left a comment
james-golden-arcadia
left a comment
There was a problem hiding this comment.
Documentation is beautiful. Thank you for adding PCA.
The base branch was changed.
|
Just changed the base to main. If it looks good to you, please approve so we can get this merged. We can create a different PR that more completely populates the docs. |
2 participants
Makes PR makes the docs look beautiful. All that's missing is content.