feat: unified ModelRegistry keyed on (backend, provider)

.github/pages/index.html

@@ -330,7 +330,7 @@ <h3>Vault &amp; Secret Management</h3>
       <div class="feature">
         <div class="feature-icon">&#129302;</div>
         <h3>Model Registry</h3>
-        <p>Periodic provider model list cache with SQLite persistence and ETS write-through for low-latency reads. GenServer-managed refresh cycles for Anthropic, OpenAI, and Google APIs with graceful degradation on failures.</p>
+        <p>Backend-keyed model registry with per-backend probes, SQLite persistence, and ETS read-through cache. Four write sources (baseline, probe, session hook, on-demand), ETS heir crash survival, and exponential backoff with graceful degradation on failures.</p>
       </div>
     </div>
   </section>

.gitignore

@@ -8,7 +8,7 @@
 /deps/
 
 # Where 3rd-party dependencies like ExDoc output generated docs.
-/doc/
+/doc*/
 
 # Ignore .fetch files in case you like to edit your project deps locally.
 /.fetch

README.md

@@ -81,7 +81,7 @@ Clean separation of concerns, connected through a public Elixir API.
 | **Notifications** | `MonkeyClaw.Notifications`       | Event-driven notification system — routes telemetry events to user-facing alerts via PubSub (real-time) and email (async), with workspace-scoped rules, severity thresholds, and ETS-cached routing |
 | **Channels** | `MonkeyClaw.Channels`                    | Bi-directional platform adapters — Slack, Discord, Telegram, WhatsApp, Web — with adapter behaviour, message recording, webhook verification, and async agent dispatch |
 | **Vault** | `MonkeyClaw.Vault`                           | Encrypted secret and OAuth token storage with `@secret:name` opaque references — model never sees plaintext; AES-256-GCM encryption at rest with HKDF-derived keys |
-| **ModelRegistry** | `MonkeyClaw.ModelRegistry`             | Periodic provider model list cache — GenServer with SQLite persistence, ETS write-through, and configurable refresh intervals |
+| **ModelRegistry** | `MonkeyClaw.ModelRegistry`             | Backend-keyed model registry — GenServer with per-backend probes, SQLite persistence, ETS read-through cache, and configurable refresh intervals per `(backend, provider)` pair |
 
 Contexts (`MonkeyClaw.Assistants`, `MonkeyClaw.Workspaces`, `MonkeyClaw.Webhooks`, `MonkeyClaw.Notifications`, `MonkeyClaw.Channels`, `MonkeyClaw.Vault`) provide the
 public CRUD API. `MonkeyClaw.AgentBridge` translates domain objects into
@@ -529,30 +529,35 @@ cached models grouped by provider, trigger refresh).
 
 ### Model Registry
 
-Periodic refresh of available AI models from provider APIs, with
-SQLite persistence and ETS write-through cache for low-latency reads.
+Unified model cache keyed on `(backend, provider)` with per-backend
+probes, SQLite persistence, and ETS read-through for low-latency reads.
 
-Two-layer architecture:
+Five-layer architecture:
 
 | Layer | Module | Owns |
 |-------|--------|------|
-| **ModelRegistry** | `MonkeyClaw.ModelRegistry` | GenServer — ETS table lifecycle, periodic refresh timer, serialized writes |
-| **Provider** | `MonkeyClaw.ModelRegistry.Provider` | HTTP fetching via Req for Anthropic, OpenAI, and Google APIs |
-
-The GenServer is justified because it manages concurrent state (ETS
-table ownership), periodic work (configurable refresh interval), and
-serialized writes (preventing concurrent refresh races). Reads bypass
-the GenServer entirely — ETS with `:read_concurrency` enabled.
-
-Graceful degradation: provider API failures log warnings and preserve
-stale cache. Vault resolution failures skip that provider. The
-GenServer never crashes on refresh failure. The LiveView handles
-a missing ModelRegistry process (disabled in test config) by showing
+| **ModelRegistry** | `MonkeyClaw.ModelRegistry` | GenServer — ETS table lifecycle, tick scheduler, per-backend probe dispatch, serialized writes via single upsert funnel |
+| **CachedModel** | `MonkeyClaw.ModelRegistry.CachedModel` | Ecto schema — `(backend, provider)` unique key, embedded model list, trust-boundary changeset validation |
+| **Baseline** | `MonkeyClaw.ModelRegistry.Baseline` | Boot seed loader — reads baseline model entries from `runtime.exs`, cold-start availability |
+| **EtsHeir** | `MonkeyClaw.ModelRegistry.EtsHeir` | ETS crash survival — heir process reclaims the table when the registry crashes and re-transfers on restart |
+| **Provider** | `MonkeyClaw.ModelRegistry.Provider` | HTTP fetching via Req for Anthropic, OpenAI, and Google APIs; called by the BeamAgent backend adapter |
+
+Four independent writers populate the cache: **Baseline** (boot seed),
+**Probe** (periodic per-backend tasks via `TaskSupervisor`),
+**Session hook** (authenticated cast from `AgentBridge.Session`), and
+**on-demand refresh** (`refresh/1`, `refresh_all/0`). All four funnel
+through a single validated upsert path with conditional precedence on
+`(refreshed_at, refreshed_mono)`.
+
+Graceful degradation: probe failures trigger exponential backoff
+(5s initial, 5min cap) and preserve stale cache. Baseline guarantees
+a floor of known models at boot even if SQLite is unavailable. The
+GenServer never crashes on refresh failure. The LiveView handles a
+missing ModelRegistry process (disabled in test config) by showing
 an empty state.
 
 Runtime reconfiguration via `ModelRegistry.configure/1` allows changing
-the workspace ID and provider secret mappings without restarting the
-process.
+backends, intervals, and backend configs without restarting the process.
 
 ### Dashboard
 

config/config.exs

@@ -96,12 +96,15 @@ config :monkey_claw, MonkeyClaw.Extensions,
     ]
   }
 
-# Model registry — periodic refresh of available models from provider APIs.
-# Disabled by default; configure workspace_id and provider_secrets to enable.
+# Model registry — per-backend probe of available models from provider APIs.
+# Configure :backends and :backend_configs to enable periodic probes.
+# See MonkeyClaw.ModelRegistry moduledoc for full option descriptions.
 config :monkey_claw, MonkeyClaw.ModelRegistry,
-  refresh_interval_ms: 3_600_000,
-  workspace_id: nil,
-  provider_secrets: %{}
+  backends: [],
+  default_interval_ms: 3_600_000,
+  backend_intervals: %{},
+  backend_configs: %{},
+  workspace_id: nil
 
 # Configure Elixir's Logger
 config :logger, :default_formatter,

config/runtime.exs

@@ -103,3 +103,42 @@ if config_env() == :prod do
   #
   # See https://hexdocs.pm/swoosh/Swoosh.html#module-installation for details.
 end
+
+# ── MonkeyClaw.ModelRegistry Baseline ────────────────────────
+#
+# Baseline entries seed the registry at boot so the agent has a
+# floor of known models before any probe runs. Entries are
+# structurally validated by MonkeyClaw.ModelRegistry.Baseline.load!/0
+# and then trust-boundary validated by CachedModel.changeset/2 inside
+# the registry's upsert funnel. Users can override or extend this
+# list in their own runtime.exs without rebuilding the release.
+config :monkey_claw, MonkeyClaw.ModelRegistry.Baseline,
+  entries: [
+    %{
+      backend: "claude",
+      provider: "anthropic",
+      models: [
+        %{model_id: "claude-opus-4-6", display_name: "Claude Opus 4.6", capabilities: %{}},
+        %{model_id: "claude-sonnet-4-6", display_name: "Claude Sonnet 4.6", capabilities: %{}},
+        %{
+          model_id: "claude-haiku-4-5-20251001",
+          display_name: "Claude Haiku 4.5",
+          capabilities: %{}
+        }
+      ]
+    },
+    %{
+      backend: "codex",
+      provider: "openai",
+      models: [
+        %{model_id: "gpt-5", display_name: "GPT-5", capabilities: %{}}
+      ]
+    },
+    %{
+      backend: "gemini",
+      provider: "google",
+      models: [
+        %{model_id: "gemini-2.5-pro", display_name: "Gemini 2.5 Pro", capabilities: %{}}
+      ]
+    }
+  ]

lib/monkey_claw/agent_bridge/backend.ex

@@ -44,6 +44,37 @@ defmodule MonkeyClaw.AgentBridge.Backend do
   @type thread_info :: map()
   @type permission_mode :: :default | :accept_edits | :bypass_permissions | :plan | :dont_ask
 
+  @typedoc """
+  Options for listing models. Adapter-specific keys are permitted.
+
+  Common keys used by MonkeyClaw adapters:
+
+    * `:workspace_id` — Vault workspace for secret resolution
+    * `:secret_name` — Vault secret name for the backend's API key
+    * `:probe_deadline_ms` — Hard wall-clock deadline for the probe
+  """
+  @type list_models_opts :: %{
+          optional(:workspace_id) => Ecto.UUID.t(),
+          optional(:secret_name) => String.t(),
+          optional(:probe_deadline_ms) => pos_integer(),
+          optional(atom()) => term()
+        }
+
+  @typedoc """
+  Single model descriptor returned by `list_models/1`.
+
+  The `:provider` field MUST be present on every entry so the
+  registry can fan multi-provider backends out into one row per
+  `(backend, provider)` pair.
+  """
+  # Loose shape by design — CachedModel.changeset/2 performs full trust-boundary validation.
+  @type model_attrs :: %{
+          provider: String.t(),
+          model_id: String.t(),
+          display_name: String.t(),
+          capabilities: map()
+        }
+
   @doc """
   Start a new agent session.
 
@@ -52,6 +83,27 @@ defmodule MonkeyClaw.AgentBridge.Backend do
   """
   @callback start_session(opts :: map()) :: {:ok, session_pid()} | {:error, term()}
 
+  @doc """
+  List the models this backend currently supports.
+
+  Called by `MonkeyClaw.ModelRegistry` during boot (baseline delta),
+  periodic probes, and on-demand refreshes. Does NOT require a live
+  session — adapters decide internally how to satisfy the request
+  (HTTP API call, transient CLI init handshake, local manifest
+  read, etc.).
+
+  Implementations should respect their own deadline; the registry
+  also enforces a hard outer deadline via `Task.shutdown/2` as a
+  safety net.
+
+  Returns a flat list of `model_attrs` maps. A single adapter may
+  return models from multiple providers in one list (e.g., Copilot
+  routing both OpenAI and Anthropic); the registry groups by
+  `:provider` at write time.
+  """
+  @callback list_models(opts :: list_models_opts()) ::
+              {:ok, [model_attrs()]} | {:error, term()}
+
   @doc """
   Stop an agent session.
 
@@ -155,15 +207,16 @@ defmodule MonkeyClaw.AgentBridge.Backend do
   # ── Checkpoint Operations (Experiment Support) ───────────────
 
   @doc """
-  Save a checkpoint of the current session state.
+  Snapshot the given files for later rollback.
 
-  Returns a checkpoint identifier that can be used with
-  `checkpoint_rewind/2` to restore the session to this point.
+  Captures the content, permissions, and existence of each file in
+  `file_paths` so that `checkpoint_rewind/2` can restore them.
+  Returns a checkpoint identifier (UUID) for the snapshot.
 
-  Used by the experiment Runner to snapshot state before each
+  Used by the experiment Runner to snapshot scoped files before each
   iteration, enabling rollback on rejection.
   """
-  @callback checkpoint_save(session_pid(), label :: String.t()) ::
+  @callback checkpoint_save(session_pid(), label :: String.t(), file_paths :: [String.t()]) ::
               {:ok, checkpoint_id :: String.t()} | {:error, term()}
 
   @doc """

lib/monkey_claw/agent_bridge/backend/beam_agent.ex

@@ -14,6 +14,8 @@ defmodule MonkeyClaw.AgentBridge.Backend.BeamAgent do
 
   @behaviour MonkeyClaw.AgentBridge.Backend
 
+  alias MonkeyClaw.ModelRegistry.Provider
+
   @impl true
   def start_session(opts), do: BeamAgent.start_session(opts)
 
@@ -80,29 +82,62 @@ defmodule MonkeyClaw.AgentBridge.Backend.BeamAgent do
   @impl true
   def thread_list(pid), do: BeamAgent.Threads.thread_list(pid)
 
-  # ── Checkpoint Operations ────────────────────────────────────
+  @impl true
+  def list_models(opts) when is_map(opts) do
+    backend = Map.get(opts, :backend)
+    provider = backend_to_provider(backend)
+
+    provider_opts =
+      opts
+      |> Map.to_list()
+      |> Keyword.take([:workspace_id, :secret_name, :api_key, :base_url])
 
-  # BeamAgent.Checkpoint may not yet export these functions.
-  # Suppress Dialyzer warnings; function_exported?/3 guard
-  # ensures runtime safety until the API is available.
+    case Provider.fetch_models(provider, provider_opts) do
+      {:ok, models} ->
+        {:ok, Enum.map(models, &annotate_provider(&1, provider))}
+
+      {:error, _} = error ->
+        error
+    end
+  end
+
+  # Map the MonkeyClaw backend identifier to the upstream provider name.
+  # Static table — future SDK and local backends extend this.
+  defp backend_to_provider("claude"), do: "anthropic"
+  defp backend_to_provider("codex"), do: "openai"
+  defp backend_to_provider("gemini"), do: "google"
+  defp backend_to_provider("opencode"), do: "anthropic"
+  defp backend_to_provider("copilot"), do: "github_copilot"
+  defp backend_to_provider(nil), do: "anthropic"
+  defp backend_to_provider(other) when is_binary(other), do: other
+
+  defp annotate_provider(%{model_id: id, display_name: name, capabilities: caps}, provider) do
+    %{
+      provider: provider,
+      model_id: id,
+      display_name: name,
+      capabilities: caps
+    }
+  end
+
+  # ── Checkpoint Operations ────────────────────────────────────
 
   @impl true
-  def checkpoint_save(pid, label) do
-    if function_exported?(BeamAgent.Checkpoint, :save, 2) do
-      # credo:disable-for-next-line Credo.Check.Refactor.Apply
-      apply(BeamAgent.Checkpoint, :save, [pid, label])
-    else
-      {:error, :not_supported}
+  def checkpoint_save(pid, label, file_paths) do
+    with {:ok, info} <- BeamAgent.session_info(pid) do
+      uuid = "#{label}-#{:erlang.unique_integer([:positive, :monotonic])}"
+
+      case BeamAgent.Checkpoint.snapshot(info.session_id, uuid, file_paths) do
+        {:ok, _cp} -> {:ok, uuid}
+        {:error, _} = error -> error
+      end
     end
   end
 
   @impl true
   def checkpoint_rewind(pid, checkpoint_id) do
-    if function_exported?(BeamAgent.Checkpoint, :rewind, 2) do
-      # credo:disable-for-next-line Credo.Check.Refactor.Apply
-      apply(BeamAgent.Checkpoint, :rewind, [pid, checkpoint_id])
-    else
-      {:error, :not_supported}
+    with {:ok, info} <- BeamAgent.session_info(pid) do
+      BeamAgent.Checkpoint.rewind(info.session_id, checkpoint_id)
     end
   end
 end