postgres backend bypasses get_embedding_function(): ignores MEMPALACE_EMBEDDING_MODEL, skips the #1068 ORT thread cap, and rebuilds the ONNX session on every call

[Lcstyle]

Summary

mempalace/backends/postgres.py::_embed constructs its own
chromadb.utils.embedding_functions.DefaultEmbeddingFunction() and never calls
mempalace.embedding.get_embedding_function(). Because every embedding-layer
improvement in the project lands inside that resolver, the PostgreSQL backend
silently misses all of them. Three separate defects compound in one eight-line
function.

1. MEMPALACE_EMBEDDING_MODEL is ignored on the PostgreSQL write path

With MEMPALACE_EMBEDDING_MODEL=openai-compat and a reachable endpoint, the
config layer resolves correctly — describe_device() returns
openai-compat (http://…) and get_embedding_function() returns
OpenAICompatEmbeddingFunction. But postgres.py::_embed never consults it, so
every vector on both the write path (_prepare_write_inputs) and the query path
(query) is computed by a local CPU embedder.

Observed on two long-running installs: the configured remote embedding server
served 9 texts in 18.3 hours of near-continuous mining.

This is the same class of failure as MemPalace#2324 (plugin manifest → MCP server
silently falls back to ChromaDB's default embedder; same dimensionality, nothing
raises), and the support added in MemPalace#1559 for external embedding APIs never
reaches this backend.

2. The ONNX Runtime thread cap from MemPalace#1068 is bypassed

MemPalace#1068 ("mine pins 400–500% CPU — ORT intra_op pool ignores OMP env vars") was
fixed in PR MemPalace#1836 by _build_ef_class() / _MempalaceONNX, which sets
SessionOptions.intra_op_num_threads. That fix is wired up only inside
get_embedding_function(). Calling DefaultEmbeddingFunction() directly skips
it entirely.

Measured on a 48-core host: instantiating one such embedder adds ~49 OS
threads; a second adds ~47. Daemon CPU averaged 210–265% over 18 hours and
was sampled at 2550% mid-mine.

3. DefaultEmbeddingFunction rebuilds the ONNX session on every call

This one appears to be unreported. In chromadb 1.5.9, DefaultEmbeddingFunction
is not ONNXMiniLM_L6_V2 — it is a separate class in
chromadb/api/types.py whose entire __call__ body is:

return ONNXMiniLM_L6_V2()(input)

A fresh instance per call, and ONNXMiniLM_L6_V2.model is a per-instance
cached_property. So the module-global _embedder in postgres.py caches an
object that holds no model, and every _embed() call reconstructs the entire
ONNX session.

Measured, three consecutive calls with identical short inputs:

via DefaultEmbeddingFunction :  0.706s / 0.711s / 0.713s   <- perfectly flat, no warm-up ever
via a reused instance        :  1.257s / 0.532s / 0.539s

With DRAWER_UPSERT_BATCH_SIZE = 1000 that is one full session rebuild per 1000
chunks, each spawning and discarding a ~47-thread pool.

Why the stated rationale no longer holds

The docstring reads:

Reuse Chroma's default local embedding function so the PostgreSQL backend
matches the zero-API embedding model already used by the default backend
without adding a second ML dependency stack.

Delegating to get_embedding_function() satisfies that goal completely: with no
embedding configuration set, the resolver still returns a local ONNX MiniLM
embedder. It adds no dependency and does not change default behaviour. It
diverges only when an operator has explicitly configured something else — which
is the intent of MEMPALACE_EMBEDDING_MODEL.

Suggested fix

def _embed(texts: list[str]) -> list[list[float]]:
    if not texts:
        return []
    from ..embedding import get_embedding_function

    ef = get_embedding_function()
    vectors = ef(input=texts)
    return [
        v.tolist() if hasattr(v, "tolist") else [float(x) for x in v]
        for v in vectors
    ]

get_embedding_function() already caches under a lock, so the module-global
_embedder becomes unnecessary.

A note on the requires_explicit_embeddings capability as an alternative.
Declaring it is architecturally tidier — pgvector, qdrant, milvus,
sqlite_exact and chroma all do — but on its own it is not sufficient,
because it only takes effect at palace.get_collection(), and at least two
important paths do not go through that wrap point:

• mcp_server._get_collection_postgres() calls PostgresBackend.get_collection
  directly (it re-implements the KG write-through attach by hand), so MCP
  searches and writes bypass it;
• convo_miner.mine_sessions() imports mcp_server._get_collection, so the
  main mining path bypasses it too.

Consequently, adding the capability and making _embed raise would break both
the MCP server and the miner. Delegating inside _embed fixes every route,
wrapped or not, and cannot break a caller that already worked. Both changes
together are fine; the delegation is the one that must be there.

Vector-space safety

Switching embedders silently re-points a palace at a different vector space —
the risk described in MemPalace#1561. Before applying this locally we compared, on 500
real stored documents, chromadb's DefaultEmbeddingFunction against an
openai-compat endpoint serving the same all-MiniLM-L6-v2:

min cosine 0.999999488   p1 0.999999520   mean 0.999999692
both sides L2-normalized (min = mean = max = 1.000000)
k-NN top-20 id-set overlap: mean 0.997, min 0.950

Identical vector space, so no re-embedding of existing rows was required. That
will not be true for every configured endpoint, so the change should probably be
accompanied by the identity enforcement discussed in MemPalace#1561 / MemPalace#1724.

Result after delegating

Per-batch embed time 0.706s → 0.016s, the remote GPU endpoint began
receiving the work, and the per-call ~47-thread ONNX pool disappeared.

Environment

• mempalace tracking develop (3.8.0 unreleased), backend postgres
• chromadb 1.5.9, Python 3.14, Linux
• Two installs, ~760k and ~1.36M drawers

Related

MemPalace#1068 (thread cap, fixed but bypassed here) · MemPalace#1559 (external embedding APIs,
lands only in the resolver) · MemPalace#2324 (same silent-fallback shape) · MemPalace#1561 / MemPalace#1724
(embedder identity) · PR MemPalace#665 (the origin of this file; the _embed body and
capability set are identical there)