Skip to content

Add anchor and hyperparameter-init controls to posterior_mean_function#829

Draft
kalama-ai wants to merge 1 commit into
feature/posterior_mean_propertyfrom
feature/posterior_mean_configurations
Draft

Add anchor and hyperparameter-init controls to posterior_mean_function#829
kalama-ai wants to merge 1 commit into
feature/posterior_mean_propertyfrom
feature/posterior_mean_configurations

Conversation

@kalama-ai

Copy link
Copy Markdown
Collaborator

Add anchors and mean_kernel_init controls to posterior_mean_function

What this does

The base branch added GaussianProcessSurrogate.posterior_mean_function, which turns a trained GP's posterior mean into a mean module you can plug into a second GP. That gave us mean transfer, but with exactly one fixed behavior. This PR opens up that behavior with two keyword arguments:

def posterior_mean_function(
    self,
    searchspace,
    objective,
    measurements,
    *,
    anchors: Literal["pretrained", "new", "combined"] = "pretrained",
    mean_kernel_init: Literal["freeze", "warmstart", "discard"] = "freeze",
) -> GPyTorchMean

The defaults (anchors="pretrained", mean_kernel_init="freeze") reproduce the existing behavior exactly, so nothing changes unless you opt in.

The two knobs

anchors decides which data the inner GP is conditioned on when computing the transferred mean:

  • "pretrained" — the source GP's own training data (recovered in raw space). This is the "pure transfer" case.
  • "new" — the new GP's measurements only.
  • "combined" — both, concatenated.

mean_kernel_init decides what happens to the inner GP's mean/kernel/likelihood once transfer is set up:

  • "freeze" — deep-copy the pretrained modules and lock them (requires_grad=False). The inner mean is a fixed, static prior.
  • "warmstart" — deep-copy them but leave them trainable, so the outer MLL can keep adjusting them.
  • "discard" — throw away the pretrained hyperparameters and start the inner modules fresh from the factories, trainable.

Importantly, discard resets all three modules (mean, kernel, and likelihood), not just the kernel, because all three feed into the posterior mean.

Guards and a warning

Not every combination is sound, so the method validates up front instead of letting things fail later:

  • anchors="new" + mean_kernel_init="discard" is rejected with a ValueError — it transfers no pretrained information at all, so it would just be a plain GP refit dressed up as transfer learning.
  • mean_kernel_init="warmstart" with anchors in {"new", "combined"} emits a warning. When the inner anchors include the new targets and the inner mean is free to move, the same y_new ends up driving both the inner prior mean and the outer marginal likelihood. The flexible inner mean can then interpolate y_new, the outer residual collapses, and the MLL drives the outer noise toward zero — overconfident posteriors. freeze is always safe here, since a frozen inner mean can't chase y_new.

- `anchors`: choose pretrained / new / combined data for the inner GP
- `mean_kernel_init`: freeze, warmstart or discard the inner hyperparameters
- Reject the no-op (new, discard) combo; warn on warmstart with new targets
- Split out _resolve_anchors and _build_inner_gp as module-level helpers
- Cover the new behaviour in tests/test_posterior_mean_function.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant