Publish results and model descriptions

Adds Mermaid style evaluation flowchart, results, the
head-invariance table/figure, and the data figures. docs/models.md describes
each model operationally (input, normalization, scoring contract, coverage),
cross-linking the adapter contract and the companion manuscript.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: lagillenwater

README.md

@@ -2,9 +2,7 @@
 
 Foundation-model evaluation harness for patient-derived tumor organoid (PDTO) drug-response prediction. Realizing the benefits of foundation models requires careful evaluations that map the boundaries of generalization.
 
-The harness produces a registry-backed report comparing three models against three out-of-distribution split strategies on two PDTO datasets, with negative/positive controls, bootstrap confidence intervals, and a pretraining-leakage exposure profile per result.
-
-See [docs/fm-pdo-evaluator-plan.md](docs/fm-pdo-evaluator-plan.md) for the 3-week plan.
+The harness trains a fixed downstream head on cell-line drug screens (GDSC2) and tests it, frozen, on sarcoma organoids (Soragni 2024), comparing foundation-model embeddings (Stack) against simple expression baselines under negative and positive controls. It is the companion code for *Prospective Evaluation of Foundation Model Performance in Precision Medicine* (greenelab/fm-pm-eval-manuscript).
 
 ## Quickstart
 
@@ -19,11 +17,76 @@ uv sync --extra dev
 uv run pytest
 ```
 
+## Evaluation design
+
+A model is scored as a **representation** (PCA/NMF of expression, or a Stack embedding) fed to a swappable **head** (linear ridge or RBF kernel ridge). The head is trained on GDSC2 cell lines and frozen, then asked to predict held-out Soragni organoids. Negative (within-drug permutation) and positive (planted interaction) controls bracket every result. See [docs/models.md](docs/models.md) for each model and [docs/adapter_contract.md](docs/adapter_contract.md) for the encoder interface.
+
+```mermaid
+flowchart LR
+    G[GDSC2 sarcoma<br/>cell lines] --> R
+    S[Soragni PDTOs<br/>organoids] --> R
+    subgraph R[Representation]
+        E[log1p CPM<br/>expression PCA/NMF]
+        K[Stack-Large<br/>embedding]
+    end
+    R --> H
+    subgraph H[Head]
+        L[linear ridge<br/>SimpleProbe]
+        N[RBF kernel ridge<br/>KernelProbe]
+        B[bilinear<br/>z, g, z⊗g]
+    end
+    H --> T[train on GDSC2,<br/>predict frozen on Soragni]
+    T --> M[metrics:<br/>interaction rho,<br/>within-drug rho,<br/>regret@k]
+    M --> C{controls}
+    C --> NEG[negative:<br/>within-drug permutation]
+    C --> POS[positive:<br/>planted interaction]
+```
+
 ## Datasets
 
-- **Soragni 2024** sarcoma PDTOs ([Synapse PDTOSarcoma](https://www.synapse.org/PDTOSarcoma))
-- **Yang 2024** primary liver cancer PDOs ([Cancer Cell](https://www.cell.com/cancer-cell/fulltext/S1535-6108(24)00089-8))
+- **Soragni 2024** sarcoma PDTOs ([Synapse PDTOSarcoma](https://www.synapse.org/PDTOSarcoma)) — 17 matched organoids, 21 drugs shared with GDSC2 by PubChem CID
+- **GDSC2** sarcoma cell lines (DepMap RNA-seq + GDSC2 screen) — 28 lines, the powered training cohort
+- **Yang 2024** primary liver cancer PDOs ([Cancer Cell](https://www.cell.com/cancer-cell/fulltext/S1535-6108(24)00089-8)) — deferred to v2
+
+The cohorts and the shared drug panel (the raw inputs, no model):
+
+![Cohort composition](docs/figures/cohort_composition.png)
+![Response distributions](docs/figures/response_distributions.png)
+![Soragni organoid x drug viability](docs/figures/soragni_response_heatmap.png)
+![Shared drug panel](docs/figures/shared_panel.png)
+
+Regenerate with `uv run python scripts/plot_data.py`.
+
+## Results
 
+**A Stack-Large embedding gives no advantage over PCA of expression for sarcoma drug response.** In-distribution (within GDSC2 sarcoma), the embedding signal beyond the drug mean is, if anything, weaker than expression PCA (interaction rho: expression 0.224 > Stack 0.199 > subtype 0.138). Out of distribution (GDSC2 → Soragni, frozen), the transcriptome models collapse toward the drug-mean prior, and the predictive signal lives in the functional organoid screens rather than the transcriptome. Positive controls confirm each apparatus has full power when signal is planted in its own representation, so the null is biological, not a dead pipeline.
+
+### Head-invariance
+
+The headline comparison rides on a *linear* head, so the obvious objection is that a foundation-model embedding might only pay off under a *nonlinear* head. Running the same frozen GDSC2 → Soragni transfer through linear ridge, RBF kernel ridge, and the bilinear model shows the result holds across head families — the nonlinear head does **not** rescue Stack:
+
+| head | representation | interaction rho | p |
+|---|---|--:|--:|
+| linear | expression PCA | +0.130 | 0.105 |
+| linear | expression NMF | +0.184 | 0.035 |
+| linear | Stack | +0.130 | 0.085 |
+| kernel | expression PCA | **+0.225** | 0.005 |
+| kernel | Stack | +0.076 | 0.225 |
+| bilinear | 16 shared drugs | −0.019 | 0.420 |
+
+Under the linear head expression and Stack tie; under the kernel head **expression rises to 0.225 while Stack stays at 0.076** — the nonlinear capacity benefits expression, not the embedding. Stack ≤ expression is head-invariant.
+
+![Head-invariance](docs/figures/head_invariance.png)
+
+Regenerate the table and figure:
+
+```bash
+uv run python scripts/transfer_gdsc_soragni.py --all-heads \
+    --stack-gdsc stack_gdsc.csv --stack-soragni stack_soragni.csv \
+    --out results/head_invariance.csv
+uv run python scripts/transfer_pharmaformer_lite.py --out results/head_invariance.csv
+uv run python scripts/plot_data.py
+```
 
 ## Affiliation
 

docs/models.md

@@ -0,0 +1,54 @@
+# Models under evaluation
+
+Operational description of every model/representation the harness scores, kept next to
+the code so it stays in sync. For the encoder interface see [adapter_contract.md](adapter_contract.md);
+for how the comparison is read see the Results section of the [README](../README.md). The
+scholarly description of each model (architecture, citations, rationale) lives in the
+companion manuscript (greenelab/fm-pm-eval-manuscript), which cross-references this file.
+
+## Two layers: representation and head
+
+A prediction is `response(sample, drug) = a_drug + f(representation(sample))`. Two choices
+are independent:
+
+- **Representation** — the per-sample vector a model produces: PCA/NMF of expression, or a
+  foundation-model embedding (Stack). Swapping the representation is how we ask "what does the
+  encoder capture."
+- **Head** — the function `f` fit on top: a linear ridge slope (`SimpleProbe`) or an RBF
+  kernel ridge (`KernelProbe`). Swapping the head is how we ask whether a finding is
+  *head-invariant* (a linear head could under-read a representation that only pays off
+  nonlinearly).
+
+Both heads share the per-drug mean and the PCA/NMF reduction
+([`probe/base.py`](../src/fmharness/probe/base.py)) and the same `fit(emb, drugs, y)` /
+`predict_parts(emb, drugs) -> (base, residual)` contract, so any representation can be scored
+under either head apples-to-apples. The bilinear and biomarker models below are standalone
+(they do not use this head interface).
+
+## Catalogue
+
+| Model | Representation | Head / form | Input + normalization | Where |
+|---|---|---|---|---|
+| Expression PCA | top-k PCA of log1p CPM expression | linear ridge (`SimpleProbe`) | bulk RNA-seq, CPM then log1p; per-drug ridge over k PCs | [`probe/simple.py`](../src/fmharness/probe/simple.py) |
+| Expression NMF | non-negative gene programs of log1p CPM | linear ridge | same; NMF requires non-negative input | `probe/simple.py` (`reducer="nmf"`) |
+| Stack-Large | 1600-dim Stack embedding (PCA-reduced) | linear ridge | CPM expression fed to Stack as pseudo-cells; embedding CSV indexed by sample id | [`stack_panel.py`](../src/fmharness/stack_panel.py), `probe/simple.py` |
+| Kernel (any representation) | same PCA/NMF reduction, score-standardized | RBF kernel ridge (`KernelProbe`) | identical to the linear head's input; `(alpha, gamma)` by per-drug leave-one-out | [`probe/kernel.py`](../src/fmharness/probe/kernel.py) |
+| Bilinear (PharmaFormer-lite) | expression PCA `z` + Morgan-fingerprint PCA `g` | ridge on `[z, g, z⊗g]` | log1p CPM expression + 1024-bit Morgan fingerprint; predicts unseen drugs via chemistry | [`scripts/transfer_pharmaformer_lite.py`](../scripts/transfer_pharmaformer_lite.py), [`bilinear.py`](../src/fmharness/bilinear.py) |
+| Biomarker | pre-specified genomic rules (e.g. PIK3CA→everolimus) | rule offsets on the GDSC2 drug-mean prior | WES SNV/CNV calls + drug identity | [`scripts/biomarker_anchored.py`](../scripts/biomarker_anchored.py) |
+| Drug-mean baseline | none | `a_drug` only (`n_components=0`) | response labels only | `probe/simple.py` |
+
+## Shared guarantees
+
+- **Graceful degradation.** Every learned head shrinks its embedding term toward 0 when the
+  representation is uninformative (ridge penalty / large kernel alpha), so an uninformative
+  model reduces to the drug-mean baseline rather than injecting noise — this is what makes a
+  null result a real null, not a weak model.
+- **`base + residual` split.** `predict_parts` returns the per-drug mean and the embedding
+  residual separately so the metrics (`interaction_rho`, `within_drug_rho` in
+  [`evaluation.py`](../src/fmharness/evaluation.py)) score the embedding part alone, avoiding
+  the leave-one-out drug-base artifact.
+- **Coverage.** The per-drug transfers (expression, Stack) score only drugs shared between
+  GDSC2 and Soragni (~15–21 by PubChem CID); the bilinear scores the full fingerprinted panel.
+  Uncovered drugs fall back to the GDSC2 drug-mean.
+- **Determinism.** All heads are closed-form (RidgeCV / KernelRidge) and seeded; repeated fits
+  are bit-identical.