Skip to content

feat(gossip): add gossip protocol MVP and MCP tools - #2239

Closed
jjcav84 wants to merge 6 commits into
MemPalace:developfrom
jjcav84:feat/gossip-mvp
Closed

feat(gossip): add gossip protocol MVP and MCP tools#2239
jjcav84 wants to merge 6 commits into
MemPalace:developfrom
jjcav84:feat/gossip-mvp

Conversation

@jjcav84

@jjcav84 jjcav84 commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add gossip protocol core module (mempalace/gossip.py) with chatter nodes, message propagation, TTL-bounded KG triples, and topic/priority detection.
  • Add mempalace_gossip and mempalace_gossip_status MCP tools.
  • Add tests/test_gossip.py and TestGossipTools in tests/test_mcp_server.py.
  • Update website/reference/mcp-tools.md and README.md to 46 tools.

Test plan

  • uv run pytest tests/ -v passes (4313 passed, 31 skipped on develop).
  • uv run ruff check mempalace/gossip.py mempalace/mcp_server.py tests/test_gossip.py tests/test_mcp_server.py passes.

@jjcav84
jjcav84 changed the base branch from develop to main August 12, 2026 23:25
@jjcav84
jjcav84 force-pushed the feat/gossip-mvp branch 2 times, most recently from 7dedcd0 to 4501b8e Compare August 13, 2026 00:16
@jjcav84
jjcav84 changed the base branch from main to develop August 13, 2026 00:16
Implements mempalace/gossip.py with ChatterNode, GossipMessage,
GossipProtocol, default chatter-node config, topic/priority detection,
propagation through palace graph tunnels, and TTL-bounded derived KG triples.
Adds tests/test_gossip.py with 16 unit tests.

Full suite: 4261 passed, 31 skipped.
@jjcav84

jjcav84 commented Aug 13, 2026

Copy link
Copy Markdown
Contributor Author

Hi maintainers — this PR is rebased onto the latest develop and the full test suite passes. Ready for review.

@jjcav84

jjcav84 commented Aug 14, 2026

Copy link
Copy Markdown
Contributor Author

@milla-jovovich @igorls this branch is up to date with develop and the full test suite passes. It is ready for final review and merge. @fatkobra if you have bandwidth, a look would be appreciated too.

@fatkobra fatkobra left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for building this first gossip slice. The overall separation between
configuration, selection, propagation, and reporting is promising.

I found three blocking issues.

1. The automatic default configuration is specific to one project

DEFAULT_GOSSIP_CONFIG contains project-specific wings, halls, specialties,
and radii, including names such as orkid, brutal-marketing,
past-performance, and negentropy.

When no gossip configuration exists, that topology is automatically written
to the user's palace. An unrelated palace therefore silently acquires another
project's organizational model, and subsequent propagation can write derived
facts according to that model.

Please make the built-in default neutral. Reasonable options would be:

  • an empty configuration that requires explicit setup before propagation;
  • a minimal topology derived from the current palace;
  • or a clearly labelled example configuration that is never installed
    automatically.

Please add a regression test using a palace whose wings and rooms share none
of the example project's names, and verify that enabling or inspecting gossip
does not persist unrelated topology or create destination facts for those
names.

2. A node can be reported as propagated even when every KG write failed

The per-target kg.add_triple() exceptions are caught, but the node is still
appended to chatter_nodes and the message is still included in the
propagation report.

As a result, the caller can receive a successful-looking propagation result
even when zero destination facts were written.

Please make write outcomes observable. At minimum, the result should
distinguish:

  • attempted targets;
  • successful writes;
  • failed writes and their errors;
  • nodes with at least one successful propagation;
  • and nodes for which every write failed.

A node should not be reported simply as propagated when all of its writes
failed.

Please add regression coverage for:

  1. every add_triple() call raising;
  2. a mixture of successful and failed targets;
  3. all writes succeeding.

3. mempalace_gossip is missing from the daemon write classifier

The MCP server includes mempalace_gossip in _MUTATING_TOOLS, but the
daemon's service.WRITE_TOOLS does not include it.

The generic daemon mcp_tool execution path accepts only tools classified as
writes, so gossip is currently treated as an unknown or non-write operation
rather than as the mutation that the MCP server says it is.

Please add mempalace_gossip to the daemon write classification and prevent
the two sets from drifting again.

A table-driven invariant should assert that every member of
_MUTATING_TOOLS is classified as a write unless it is explicitly listed
and documented as an exclusive maintenance operation.

Once these three issues are addressed, I will be happy to re-review the
parent gossip implementation.

- Make DEFAULT_GOSSIP_CONFIG neutral/empty; add EXAMPLE_GOSSIP_CONFIG for tests.
- Add config parameter to GossipProtocol and gossip() for explicit setup.
- Report attempted/successful/failed targets and only mark a node propagated when at least one write succeeds.
- Add mempalace_gossip to service.WRITE_TOOLS and invariant test.

Refs MemPalace#2239
@jjcav84

jjcav84 commented Aug 15, 2026

Copy link
Copy Markdown
Contributor Author

Hi @fatkobra — thanks for the review. I believe all three blockers are now addressed:

  1. Neutral default config: DEFAULT_GOSSIP_CONFIG now ships with empty chatter_nodes and topics (no project-specific wings/halls). The populated orkid/brutal-marketing/past-performance/negentropy example has been moved to EXAMPLE_GOSSIP_CONFIG, which is only used in tests/docs and is never auto-installed. Added test_default_config_does_not_leak_project_topology to lock this in.

  2. Observable write outcomes: GossipProtocol.propagate now reports attempted_targets, successful_targets, and failed_targets (with wing/room/error) for each node. A chatter node is only added to propagated if at least one kg.add_triple() call succeeded; all-failures is correctly not reported as propagation. Tests: test_propagate_report_distinguishes_failed_writes and test_propagate_report_records_partial_write_success.

  3. Daemon write classification: mempalace_gossip is present in mcp_server._MUTATING_TOOLS, service.WRITE_TOOLS, and service.classify_tool("mempalace_gossip") returns "write". Test: test_gossip_tool_classified_as_write.

Verification:

  • pytest tests/test_gossip.py — 19 passed
  • pytest tests/test_mcp_server.py -k gossip — 3 passed

Could you re-review when you have a moment?

@fatkobra fatkobra left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks — I re-reviewed c611528, and my three blocking concerns are
resolved.

  • The installed default configuration is now neutral, with empty chatter
    nodes and topics. The project-specific topology has been moved to an
    example configuration that is not installed automatically.
  • Propagation now exposes attempted, successful, and failed target outcomes,
    including individual errors, and a chatter node is reported as propagated
    only when at least one KG write succeeds.
  • mempalace_gossip is now classified as a write by the daemon and has
    regression coverage for that classification.

The classification test in this branch is specific to the gossip tool rather
than the complete table-driven invariant I originally described. However,
#1992 already implements that repository-wide parity check and the associated
fail-closed handling for classifier drift, so I do not think it should be
duplicated here.

My blockers on this PR are resolved. Approved.

@jjcav84

jjcav84 commented Aug 18, 2026

Copy link
Copy Markdown
Contributor Author

Merged latest upstream/develop to resolve the UNSTABLE merge state. Branch is now up to date — ready for re-review/approval.

@jjcav84

jjcav84 commented Aug 22, 2026

Copy link
Copy Markdown
Contributor Author

@fatkobra — all the open PRs have been rebased onto latest develop and the requested changes from the earlier round are pushed. Could you take a look when you have a moment?

Ready for re-review / merge:

Requested changes addressed (need re-approval):

New / no reviews yet:

@igorls

igorls commented Aug 22, 2026

Copy link
Copy Markdown
Member

Thanks for the persistence on this one, and for turning fatkobra's three
blockers around quickly — the neutral default, the write-outcome reporting,
and the daemon classifier fix were all handled properly.

I read the whole stack rather than just this PR (#2239#2247#2248
#2249#2250#2251#2255 is strictly linear — each head contains every
prior commit, so #2255's tip is the entire feature, ~2,900 lines). I want to
give you an honest read rather than let it keep accumulating: this isn't
close to something we can merge in its current shape.
Two reasons, one of
which is a hard blocker regardless of everything else.

The name has to change

In MemPalace, "gossip" is already taken, and it means the P2P thing. Two
places on develop today:

  • mempalace/mcp_server.py# replicas this node only knows transitively (gossip carriers).
  • website/guide/shared-brain.md — "…and any replicas known only through
    gossip. The same payload is the mempalace_mesh_peers MCP tool."

That's epidemic dissemination in the standard sense: a replica learns about a
third machine it has never exchanged credentials with because a common peer
carried the news. RFC 004 (docs/rfcs/004-replicated-palace.md) builds on
that with anti-entropy sync over the transport seam, and the MeshGuard
binding brings SWIM membership with real gossip underneath.

So a top-level mempalace/gossip.py plus a mempalace_gossip MCP tool would
claim that name for something with no network in it at all — propagate()
never leaves the process. Someone who reads the shared-brain guide and then
finds mempalace_gossip will reasonably expect it to talk to peers.

This is fixable and worth fixing: what you've built is relevance routing /
topic fanout
— score a fact against declared interest profiles, pick top-N,
write derived edges. chatter.py / fanout.py / resonance.py all describe
it honestly and leave "gossip" free for the layer that earns it. Please
rename regardless of where the rest of the discussion lands.

One related thing, and I flag it purely as a courtesy to you: #2255 checks
docs/MEMPALACE_GOSSIP_PROTOCOL_IMPLEMENTATION.md into the repo, and it
still documents your own palace's wings, halls, and specialties as the
canonical configuration. That's your private project data going into a public
tree. Worth scrubbing before it lands anywhere.

The harder question: what this writes into the knowledge graph

This is the part I'd want settled before more code gets built on top.

Every propagation writes derived triples into the user's KG — one per
(wing, room) target, confidence=0.8, TTL-bounded, gated by
random.random() < probability. Those are statements nobody made, generated
non-deterministically, landing in the same table as real facts.
KnowledgeGraph.query_entity() has no source_file filter, so gossip://
triples come back from ordinary KG queries mixed in with genuine ones.

The KG is entity-first and keyed by real names — people, projects. After
gossip_on_gossip() runs, it also contains entities called
gossip://meta/topic, gossip://meta/fact, and gossip://meta/network,
sitting alongside them.

And the loop closes: gossip_on_gossip() writes meta-facts with
source_file = "gossip://meta", while query_gossip_triples() matches
source_file LIKE 'gossip://%' — so the analytics pass reads its own output.
The docstring says this outright ("so the next analytics pass can observe them
too"). Thresholds, intervals, and hysteresis bound the rate, but it's still a
self-feeding write loop into someone's memory, with a background daemon
(#2248) re-propagating multi-hop on a thread.

Set against the mission — verbatim always, never summarize or paraphrase,
100% recall as the design requirement — synthetic probabilistic facts that
decay on a timer are the opposite thing. Recall precision is what this system
is for, and this dilutes it by construction.

I don't think the underlying idea is bad. "Which parts of the palace should
this new fact reach?" is a genuinely interesting question. But the answer
probably can't be "write speculative edges into the KG." A version that
computed routing as a derived, queryable view — no writes into the fact
store, or a physically separate store that recall never reads — would be a
much easier conversation.

Scope

Six stacked PRs adding one feature, where each depends on the one before, is
hard to review and impossible to merge incrementally — and it means the
design question above compounds five times before anyone can weigh in on it.
If we get past the naming and the KG-write question, I'd rather see one PR
with a tight core than the stack.

Two small concrete things while I'm here:

  • The tip registers 47 MCP tools (44 on develop + 3), but README still says
    46 — feat(gossip): add gossip protocol MVP and MCP tools #2239 bumped it for two tools and feat(gossip): meta-gossip analytics and MCP tool #2249's third never updated it.
    test_readme_claims.py only checks that every tool is documented, not the
    count, so it passes.
  • With the default config now neutral (correctly — that fix was right), the
    whole feature is inert until a user hand-authors a chatter topology in JSON,
    and nothing generates one for them. Worth thinking about what the
    out-of-the-box experience is.

Where I'd go from here

The rename is unconditional. On the rest — before you write more code, could
you open an issue (or take it to a discussion) laying out the case for
derived facts in the KG and how it squares with verbatim-always? If there's a
design where routing doesn't mutate the fact store, I'm genuinely interested
in seeing it. I'd rather spend the round-trip on that than on another layer
of the stack.

Thanks again for the work and for staying responsive through review — I know
this isn't the answer you were hoping for on eight open PRs.

jjcav84 added a commit to jjcav84/mempalace that referenced this pull request Aug 22, 2026
- Make DEFAULT_GOSSIP_CONFIG neutral/empty; add EXAMPLE_GOSSIP_CONFIG for tests.
- Add config parameter to GossipProtocol and gossip() for explicit setup.
- Report attempted/successful/failed targets and only mark a node propagated when at least one write succeeds.
- Add mempalace_gossip to service.WRITE_TOOLS and invariant test.

Refs MemPalace#2239
jjcav84 added a commit to jjcav84/mempalace that referenced this pull request Aug 22, 2026
- Make DEFAULT_GOSSIP_CONFIG neutral/empty; add EXAMPLE_GOSSIP_CONFIG for tests.
- Add config parameter to GossipProtocol and gossip() for explicit setup.
- Report attempted/successful/failed targets and only mark a node propagated when at least one write succeeds.
- Add mempalace_gossip to service.WRITE_TOOLS and invariant test.

Refs MemPalace#2239
jjcav84 added a commit to jjcav84/mempalace that referenced this pull request Aug 22, 2026
- Make DEFAULT_GOSSIP_CONFIG neutral/empty; add EXAMPLE_GOSSIP_CONFIG for tests.
- Add config parameter to GossipProtocol and gossip() for explicit setup.
- Report attempted/successful/failed targets and only mark a node propagated when at least one write succeeds.
- Add mempalace_gossip to service.WRITE_TOOLS and invariant test.

Refs MemPalace#2239
jjcav84 added a commit to jjcav84/mempalace that referenced this pull request Aug 22, 2026
- Make DEFAULT_GOSSIP_CONFIG neutral/empty; add EXAMPLE_GOSSIP_CONFIG for tests.
- Add config parameter to GossipProtocol and gossip() for explicit setup.
- Report attempted/successful/failed targets and only mark a node propagated when at least one write succeeds.
- Add mempalace_gossip to service.WRITE_TOOLS and invariant test.

Refs MemPalace#2239
jjcav84 added a commit to jjcav84/mempalace that referenced this pull request Aug 22, 2026
- Make DEFAULT_GOSSIP_CONFIG neutral/empty; add EXAMPLE_GOSSIP_CONFIG for tests.
- Add config parameter to GossipProtocol and gossip() for explicit setup.
- Report attempted/successful/failed targets and only mark a node propagated when at least one write succeeds.
- Add mempalace_gossip to service.WRITE_TOOLS and invariant test.

Refs MemPalace#2239
jjcav84 added a commit to jjcav84/mempalace that referenced this pull request Aug 22, 2026
- Make DEFAULT_GOSSIP_CONFIG neutral/empty; add EXAMPLE_GOSSIP_CONFIG for tests.
- Add config parameter to GossipProtocol and gossip() for explicit setup.
- Report attempted/successful/failed targets and only mark a node propagated when at least one write succeeds.
- Add mempalace_gossip to service.WRITE_TOOLS and invariant test.

Refs MemPalace#2239
@jjcav84

jjcav84 commented Aug 22, 2026

Copy link
Copy Markdown
Contributor Author

Opened #2335 to make the design case per @fatkobra's request.

TL;DR: gossip becomes a routing layer, not a fact layer. No KG writes. Propagation state goes to a sidecar table that recall never reads. The mempalace_gossip MCP tool becomes read-only (returns routing suggestions, doesn't write).

The child PRs (#2247#2251) keep their core logic; only their storage target changes. Will rewrite once there's sign-off on the derived-view approach.

jjcav84 added a commit to jjcav84/mempalace that referenced this pull request Aug 23, 2026
Replace project-specific wing/room names (orkid, brutal-marketing,
past-performance, negentropy) with neutral placeholders (alpha, beta,
gamma, delta) in EXAMPLE_GOSSIP_CONFIG, test fixtures, and the
implementation report doc.

Per igorls review on MemPalace#2239: private palace topology should not
land in the public tree.

Tests pass (38 gossip, 3 MCP gossip, 44 KG). Ruff clean.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@jjcav84

jjcav84 commented Aug 23, 2026

Copy link
Copy Markdown
Contributor Author

Closing this PR and the stacked gossip children (#2247#2251, #2255) in favor of a simpler approach.

After further reflection and @igorls's feedback, the write-time propagation model is the wrong abstraction for the problem we're trying to solve. The actual problem is that search is wing-siloed by default — the LLM searches one wing and misses context in related wings. Propagating synthetic triples into the KG at write time doesn't fix that; it just moves the silo problem earlier and pollutes the verbatim fact store.

The wing-affinity logic in this PR (chatter node topology, hallway-aware selection, affinity scoring) is worth keeping and will be extracted into a query-time search expansion PR instead — making mempalace search automatically expand to relevant wings using existing hallways + palace_graph infrastructure, rather than pre-distributing facts at write time.

Thanks to @fatkobra and @igorls for the thorough reviews — the design feedback directly shaped this decision.

@jjcav84 jjcav84 closed this Aug 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants