Add the WM-side knowledge service and a lookup-literal lint rule (ADR-0072, ADR-0028)

goals.md §4.10 puts the WM in charge of the logical fact store and the
memo table, but nothing before this owned either: an ER had no path to
send `knowledge/query` or `knowledge/registerSchema` requests, and the
DAG needed to memoize a query and refresh a stale bucket on demand did
not exist anywhere in the WM.

- services/knowledge_service.py: loads the fact store, runs R11's
  confirmation walk, holds the schema an ER registers, executes queries
  over the interpreter, and memoizes them keyed to the fact DAG's
  revision. Mode.VERIFIED re-checks tracked inputs before answering;
  Mode.CACHED skips that walk for a live WM's latency-sensitive callers
  (ADR-0014 D6). A stale bucket a walk discovers is refreshed by
  dispatching a scoped `extract_knowledge` run into an ER and ingesting
  the facts it returns, deduped per bucket and capped in flight so
  concurrent queries needing the same bucket cost one round trip.
- runner/knowledge_bridge.py: the installable slot (ADR-0072) an ER's
  JSON-RPC client calls into, so the runner layer never imports the
  service that sits above it in the WM's layer stack.
- finecode_knowledge/query/validate.py: adds `_warn_lookup_literals`,
  the mirror of `_warn_unconstrained_key_bindings` -- it flags a FIELD
  literal that exists only to bind a head `Prov`, which in a conjunctive
  language is an existence test the author never asked for. ADR-0028 is
  the case this closes: `expected_in` was bound by adding a `def_path`
  literal to two rules, so a project missing that fact produced no
  finding at all even though the real violation held.

Author: Aksem

finecode_knowledge/src/finecode_knowledge/query/validate.py

@@ -27,6 +27,7 @@
     from finecode_knowledge.model.registry import SchemaRegistry
 
 __all__ = [
+    "LookupLiteralWarning",
     "UnconstrainedKeyBindingWarning",
     "validate_body",
     "validate_head_parameters",
@@ -65,6 +66,7 @@ def validate_body(
     _check_negation_safety(body, context=context)
     _check_range_restriction(body, context=context, projected=projected)
     _warn_unconstrained_key_bindings(body, context=context, projected=projected)
+    _warn_lookup_literals(body, context=context, projected=projected)
 
 
 def _check_literals_against_schema(
@@ -298,3 +300,83 @@ def _warn_unconstrained_key_bindings(
 
 class UnconstrainedKeyBindingWarning(UserWarning):
     """See ``_warn_unconstrained_key_bindings``."""
+
+
+def _warn_lookup_literals(
+    body: Conjunction, *, context: str, projected: typing.Sequence[object]
+) -> None:
+    """Warn where a FIELD literal exists only to bind provenance for the head.
+
+    **The mirror of ``_warn_unconstrained_key_bindings``.** That one catches a
+    body that addresses an entity and never requires it to exist -- possibly
+    under-constrained. This one catches the opposite: a body that *requires a
+    fact to exist* only because the author wanted its provenance -- possibly
+    over-constrained.
+
+    In a conjunctive language every literal is a filter, so retrieving a value
+    and requiring it are the same act. A literal added to fetch something the
+    head reports silently narrows the rule's extension, and it narrows it toward
+    **under-reporting** -- a rule that stops reporting a real violation, which is
+    the direction ADR-0013 D4.3 and ADR-0002 C3 refuse. ADR-0028 is the case
+    this was written for: ``expected_in`` was bound by adding
+    ``ProjectFields.def_path(subject, _, at=expected_in)`` to two rules, and a
+    project with no ``def_path`` fact therefore produced no finding at all.
+
+    The rule it enforces:
+
+        A ``Prov`` in the head must ride on a literal the body would contain
+        anyway. If you had to *add* a literal to bind it, the value is not part
+        of the rule's truth -- it is reporting metadata, and it belongs to
+        whoever displays the finding.
+
+    So the shape flagged is a literal whose removal would leave every one of its
+    variables still bound: the entity is constrained elsewhere, the value is a
+    throwaway variable read by nobody, and the only thing the literal yields is
+    a head-bound ``Prov``. Its whole contribution to the answer is an existence
+    test the author never asked for.
+
+    A warning rather than an error, for ``_warn_unconstrained_key_bindings``'
+    reason: requiring the fact is occasionally what the author meant. When it is,
+    the value term is usually wanted too, or the entity is bound here and nowhere
+    else -- both of which this check already declines to flag.
+    """
+    used_elsewhere: dict[int, int] = {}
+    for literal in body:
+        for term in literal.terms:
+            if isinstance(term, Var):
+                used_elsewhere[id(term)] = used_elsewhere.get(id(term), 0) + 1
+
+    exempt = {id(term) for term in projected if isinstance(term, Var)}
+    flagged: list[str] = []
+    for literal in body:
+        if literal.negated or literal.kind is not LiteralKind.FIELD:
+            continue
+        # Only a head-bound provenance makes the literal a lookup. With no `at=`
+        # the existence test is the literal's whole point, which is deliberate.
+        if not isinstance(literal.at, Prov) or id(literal.at) not in exempt:
+            continue
+        entity, value = literal.terms[0], literal.terms[1]
+        # A value the body reads, or projects, is a real constraint.
+        if not isinstance(value, Var) or id(value) in exempt:
+            continue
+        if used_elsewhere.get(id(value), 0) > 1:
+            continue
+        # If the entity is bound only here, the literal is load-bearing.
+        if not isinstance(entity, Var) or used_elsewhere.get(id(entity), 0) < 2:
+            continue
+        flagged.append(literal.predicate)
+
+    if flagged:
+        warnings.warn(
+            f"{context}: {len(flagged)} field literal(s) -- {', '.join(sorted(flagged))} -- "
+            "require a fact to exist only so their provenance can bind a head parameter. "
+            "In a conjunction that is a filter, so a subject missing the fact yields no "
+            "finding at all. Bind the head from a literal the rule needs anyway, or let "
+            "whoever displays the finding look the location up (ADR-0028).",
+            LookupLiteralWarning,
+            stacklevel=3,
+        )
+
+
+class LookupLiteralWarning(UserWarning):
+    """See ``_warn_lookup_literals``."""

finecode_knowledge/tests/test_cypher.py

@@ -16,6 +16,8 @@
 
 from __future__ import annotations
 
+import warnings
+
 import pytest
 from libcat import LIBCAT_SCHEMA, Author, Book, BookFields, Rel, Shelf
 
@@ -25,6 +27,7 @@
     compile_query,
     compile_rule,
 )
+from finecode_knowledge.query.validate import LookupLiteralWarning
 
 
 @q.derived
@@ -48,22 +51,31 @@ def _(src: q.Var[Shelf], shelf: q.Var[Shelf]) -> q.Body:
     LIBCAT_SCHEMA.register_predicate(_predicate)
 
 
-@q.rule
-def shelved_book_not_borrowed(
-    subject: q.Var[Author],
-    missing: q.Var[str],
-    asserted_at: q.Prov,
-    expected_in: q.Prov,
-) -> q.Body:
-    """author {subject} shelved a book but never borrowed {missing}"""
-    book, shelf = q.var(Book), q.var(Shelf)
-    return q.all_(
-        Rel.wrote(subject, book, at=asserted_at),
-        Rel.shelved_on(book, shelf),
-        Shelf.key(shelf, code=missing),
-        BookFields.title(book, q.var(str), at=expected_in),
-        q.not_(_borrowed_isbn(subject, missing)),
-    )
+with warnings.catch_warnings():
+    # `title` is bound only to feed `expected_in`, which is exactly the shape
+    # `LookupLiteralWarning` exists to flag (ADR-0028) -- and here it is the point:
+    # `test_a_field_facts_provenance_binds_to_a_fact_node` below needs a rule that
+    # compiles a field fact's provenance to a `Fact` node, so this fixture must keep
+    # the literal a real rule should not have. Suppressed at the definition rather
+    # than filtered suite-wide, so the check still guards every other rule here.
+    warnings.simplefilter("ignore", LookupLiteralWarning)
+
+    @q.rule
+    def shelved_book_not_borrowed(
+        subject: q.Var[Author],
+        missing: q.Var[str],
+        asserted_at: q.Prov,
+        expected_in: q.Prov,
+    ) -> q.Body:
+        """author {subject} shelved a book but never borrowed {missing}"""
+        book, shelf = q.var(Book), q.var(Shelf)
+        return q.all_(
+            Rel.wrote(subject, book, at=asserted_at),
+            Rel.shelved_on(book, shelf),
+            Shelf.key(shelf, code=missing),
+            BookFields.title(book, q.var(str), at=expected_in),
+            q.not_(_borrowed_isbn(subject, missing)),
+        )
 
 
 # ---- the dialect's constraints show up in the output -------------------

finecode_knowledge/tests/test_lookup_literal_warning.py

@@ -0,0 +1,135 @@
+"""A field literal that exists only to bind provenance for the head (ADR-0028).
+
+The defect this closes: in a conjunction every literal is a filter, so retrieving
+a value and requiring it are the same act. A literal added to fetch something the
+head *reports* narrows the rule's extension -- toward **under-reporting**, which
+is the direction ADR-0013 D4.3 and ADR-0002 C3 refuse.
+
+Concretely, two shipped rules bound ``expected_in`` by adding
+``ProjectFields.def_path(subject, _, at=expected_in)``, and a project with no
+``def_path`` fact then produced no finding at all -- every real condition held.
+
+This is the **mirror** of ``UnconstrainedKeyBindingWarning``
+(``tests/test_key_literal.py``): that one catches a body that never requires an
+entity to exist, this one a body that requires a fact to exist by accident.
+
+Written against ``libcat`` (``tests/fixtures/libcat``), like every other engine
+test -- the engine must not know FineCode's vocabulary (R18/R19).
+"""
+
+from __future__ import annotations
+
+import pytest
+from libcat import LIBCAT_SCHEMA, Author, Book, BookFields, Rel
+
+from finecode_knowledge import query as q
+from finecode_knowledge.query.validate import LookupLiteralWarning, validate_body
+
+
+def _lookup_warnings(recwarn: pytest.WarningsRecorder) -> list:
+    return [w for w in recwarn if issubclass(w.category, LookupLiteralWarning)]
+
+
+def test_a_field_literal_bound_only_to_a_head_prov_warns() -> None:
+    """The ADR-0028 shape: `title` is required to exist purely so `at` can bind.
+
+    `book` is already bound by the edge, and the value term is a throwaway
+    variable nobody reads -- so removing the literal would leave every variable
+    still bound, and its whole contribution to the answer is an existence test.
+    """
+    author, book, at = q.var(Author), q.var(Book), q.Prov()
+    body = q.all_(
+        Rel.wrote(author, book),
+        BookFields.title(book, q.var(str), at=at),
+    )
+
+    with pytest.warns(LookupLiteralWarning, match="provenance can bind a head parameter"):
+        validate_body(body, LIBCAT_SCHEMA, context="test", projected=(author, book, at))
+
+
+def test_a_prov_riding_on_a_literal_the_rule_needs_does_not_warn(
+    recwarn: pytest.WarningsRecorder,
+) -> None:
+    """`asserted_at`'s shape, and the rule the warning enforces.
+
+    Binding provenance off an edge the body needs anyway adds no filter -- the
+    edge was already required. That is the legitimate half of ADR-0028.
+    """
+    author, book, at = q.var(Author), q.var(Book), q.Prov()
+    body = q.all_(Rel.wrote(author, book, at=at))
+
+    validate_body(body, LIBCAT_SCHEMA, context="test", projected=(author, book, at))
+
+    assert not _lookup_warnings(recwarn)
+
+
+def test_a_field_literal_whose_value_the_body_reads_does_not_warn(
+    recwarn: pytest.WarningsRecorder,
+) -> None:
+    """A value another literal consumes is a real join, not a throwaway."""
+    author, book, title, at = q.var(Author), q.var(Book), q.var(str), q.Prov()
+    other = q.var(Book)
+    body = q.all_(
+        Rel.wrote(author, book),
+        BookFields.title(book, title, at=at),
+        BookFields.label(other, title),
+    )
+
+    validate_body(body, LIBCAT_SCHEMA, context="test", projected=(author, book, at))
+
+    assert not _lookup_warnings(recwarn)
+
+
+def test_a_field_literal_whose_value_is_projected_does_not_warn(
+    recwarn: pytest.WarningsRecorder,
+) -> None:
+    """`locations.project_locations`' shape: the caller wants the value itself."""
+    book, title, at = q.var(Book), q.var(str), q.Prov()
+    body = q.all_(BookFields.title(book, title, at=at))
+
+    validate_body(body, LIBCAT_SCHEMA, context="test", projected=(book, title, at))
+
+    assert not _lookup_warnings(recwarn)
+
+
+def test_a_field_literal_with_no_provenance_binding_does_not_warn(
+    recwarn: pytest.WarningsRecorder,
+) -> None:
+    """Without `at=` the existence test is the literal's whole point.
+
+    "Books that were actually scanned" is a thing a rule may legitimately ask,
+    and ADR-0019 D4 keeps the field spelling available precisely to ask it.
+    """
+    author, book = q.var(Author), q.var(Book)
+    body = q.all_(Rel.wrote(author, book), BookFields.title(book, q.var(str)))
+
+    validate_body(body, LIBCAT_SCHEMA, context="test", projected=(author, book))
+
+    assert not _lookup_warnings(recwarn)
+
+
+def test_a_field_literal_that_alone_binds_its_entity_does_not_warn(
+    recwarn: pytest.WarningsRecorder,
+) -> None:
+    """Load-bearing: drop this literal and `book` is unbound, so it is not a lookup."""
+    book, at = q.var(Book), q.Prov()
+    body = q.all_(BookFields.title(book, q.var(str), at=at))
+
+    validate_body(body, LIBCAT_SCHEMA, context="test", projected=(book, at))
+
+    assert not _lookup_warnings(recwarn)
+
+
+def test_a_prov_that_is_not_in_the_head_does_not_warn(
+    recwarn: pytest.WarningsRecorder,
+) -> None:
+    """The warning is about feeding the *head*. A body-internal `at` reports nothing."""
+    author, book, at = q.var(Author), q.var(Book), q.Prov()
+    body = q.all_(
+        Rel.wrote(author, book),
+        BookFields.title(book, q.var(str), at=at),
+    )
+
+    validate_body(body, LIBCAT_SCHEMA, context="test", projected=(author, book))
+
+    assert not _lookup_warnings(recwarn)

src/finecode/wm_server/runner/knowledge_bridge.py

@@ -0,0 +1,69 @@
+"""Slot through which whoever owns the fact store serves the runner layer.
+
+``knowledge/query`` and ``knowledge/registerSchema`` arrive on the **runner**'s
+JSON-RPC client. The code that answers them lives in
+``services/knowledge_service.py``, because it owns the fact store and the memo
+table -- and services sit *above* the runner in the WM's layer stack. See ADR-0072
+for why this is a slot the owner fills on import rather than an upward import.
+"""
+
+from __future__ import annotations
+
+import typing
+
+if typing.TYPE_CHECKING:
+    from finecode.wm_server import context
+
+__all__ = ["KnowledgeHandlers", "handlers", "install", "reset"]
+
+
+class KnowledgeHandlers(typing.Protocol):
+    """What the runner needs from whoever owns the store."""
+
+    async def register_schema(self, snapshot: dict) -> bool:
+        """Install *snapshot* as the schema the WM's store is read against."""
+
+    async def run_query(
+        self,
+        ws_context: context.WorkspaceContext,
+        query: dict,
+        *,
+        mode: str,
+        limit: int | None,
+    ) -> dict:
+        """Execute a serialized query and return ``{"rows": ..., "freshness": ...}``."""
+
+    async def fetch_records(
+        self, ws_context: context.WorkspaceContext, refs: list[dict]
+    ) -> dict:
+        """Read whole entity records and return ``{"records": [...]}``.
+
+        The read a query cannot express,
+        routed here for the same reason ``run_query`` is: it touches the store, so
+        it happens where the store is.
+        """
+
+
+_installed: KnowledgeHandlers | None = None
+
+
+def install(implementation: KnowledgeHandlers) -> None:
+    """Nominate *implementation* as the answer to knowledge requests from an ER."""
+    global _installed
+    _installed = implementation
+
+
+def reset() -> None:
+    """Forget the installed implementation. Tests only."""
+    global _installed
+    _installed = None
+
+
+def handlers() -> KnowledgeHandlers | None:
+    """The installed implementation, or ``None`` if the service was never imported.
+
+    ``None`` is a real state rather than a defect: a WM built without the
+    knowledge service answers ``knowledge/query`` with a method error, which is
+    what an ER asking a WM that cannot serve it should hear.
+    """
+    return _installed