Skip to content

CONTRIBUTING.md

Latest commit

 

History

History
370 lines (278 loc) · 18 KB

CONTRIBUTING.md

File metadata and controls

370 lines (278 loc) · 18 KB
Raw

Contributing to marm-memory

You do not need to write code to contribute. Testing MARM with your client setup, reporting what broke, sharing your workflow in Discussions, or jumping into Discord to help someone get unstuck are all real contributions.

If you do want to go deeper, MARM is focused on the MCP server, local memory workflows, the code and concept knowledge graphs, Docker/STDIO transports, IDE and client integrations, and marm-console for inspecting local memory data. This guide covers that practical development workflow. For project history and community recognition, see ACKNOWLEDGMENTS.md.

Questions or Ideas

Drop a message in MARM Discord or reach out directly at support@marmemory.com.

Getting Started

git clone https://github.com/Lyellr88/marm-memory.git
cd marm-memory

Install the MCP server in editable mode:

cd marm-mcp-server
pip install -e ".[dev]"

For source development, fetch the bundled extraction model into package data:

python scripts/bundle-concept-model.py

Run the HTTP server:

python -m marm_mcp_server

Run the STDIO server:

python -m marm_mcp_server.server_stdio

Generate an API key when using exposed HTTP mode or Docker HTTP:

python -m marm_mcp_server --generate-key

Development

Project Structure

marm-mcp-server/
  marm_mcp_server/
    __main__.py                # `python -m marm_mcp_server` entry, delegates to cli
    cli.py                     # HTTP CLI, dependency checks, and server factory
    server.py                  # FastAPI HTTP composition root
    server_stdio.py            # STDIO bootstrap and core-tool registration
    config/settings.py         # Paths, host/port, auth, feature flags
    core/
      memory.py                # MARMMemory facade and public memory object wiring
      memory_utils.py          # Shared memory helpers, chunking, and encoding utilities
      memory_db.py             # SQLite schema, connection pool, and DB maintenance routines
      memory_scoring.py        # Semantic, FTS, temporal, and chunk-aware recall scoring
      memory_ops.py            # Store/update/recall/delete/list memory operations
      memory_recall.py         # Recall orchestration across the scoring lanes
      memory_delete.py         # Delete paths and their cascade handling
      write_queue.py           # Serialized write queue for SQLite writer stability
      consolidation.py         # Content-hash and semantic write-time consolidation
      compaction.py            # Background compaction candidate detection and nudges
      compaction_scheduler.py  # Optional compaction maintenance scheduler
      docs_db.py               # Indexed copy of the shipped docs served to agents
      concept_db.py            # Concept graph schema and isolated SQLite pool
      concept_extraction.py    # spaCy entity/relationship extraction (bundled model)
      concept_queue.py         # Durable outbox: one indexing task per stored memory
      concept_worker.py        # Background worker draining that queue into the graph
      concept_build_lock.py    # Concept-graph binding of the cross-process lease
      lease_lock.py            # Leased-row mutual exclusion, shared by both graphs
      graph_supervisor.py      # Lazy singleton supervisor for the embedded graph engine
      graph_client.py          # Concept graph's in-process link into the code graph
      graph_index_lock.py      # The one gate every code-graph store mutation takes
      graph_index_worker.py    # Git-signature poller that keeps code graphs current
      runtime_flags.py         # Persisted on/off switches and watch suppressions
      runtime_manager.py       # Local runtime discovery and background start/stop
      protocol_delivery_state.py  # Bounded HTTP protocol-delivery state
      models.py                # Shared Pydantic request/response models
      events.py                # Internal event hooks
      rate_limiter.py          # Rate limiting primitives
      response_limiter.py      # MCP response size controls
      shutdown_manager.py      # Graceful shutdown handling
      stdio_logging.py         # STDERR-only STDIO logger setup
      stdio_tool_lifecycle.py  # STDIO protocol, logging, and compaction wrapper
    endpoints/
      session.py               # Session tools
      logging.py               # Log tools
      reasoning.py             # Reasoning/deep-dive tools
      notebook.py              # Notebook tools
      memory.py                # Recall/search tools
      compaction.py            # Unified compaction tool and hidden helper routes
      graph.py                 # 5 bundled code-graph tools (routed through marm_graph)
      concepts.py              # 2 concept-graph tools (build + recall)
      system.py                # Health/system tools
    middleware/
      auth.py                  # Bearer auth for HTTP mode
      protocol_injection.py    # HTTP MCP protocol/compaction response injection
      rate_limiting.py         # HTTP rate limiting middleware
    services/
      analytics.py             # Best-effort local usage analytics
      documentation.py         # Startup documentation loading
      automation.py            # Event handler registration
      notebook.py              # Notebook dispatch service
      recall.py                # Shared smart-recall response logic
      summary.py               # Shared session summary formatting
      graph_context.py         # Bounded read-only concept context for recall
      log_entry.py             # Shared log-entry/notebook data ops, both transports
      compaction_apply.py      # Atomic compaction apply transaction
      compaction_summarize.py  # Compaction cluster summarization helpers
      stdio_entry_tools.py     # STDIO log entry/show/delete workflow bodies
      stdio_graph_tools.py     # STDIO graph/concept bodies and registration helper
      cli_parser.py            # Argument parsers for the product and legacy CLIs
      cli_output.py            # Human-readable status/doctor/maintenance rendering
      product_help.py          # Terminal-aware root help rendering
      product_workflows.py     # High-level local workflows (start, upgrade, uninstall)
      product_logs.py          # Bounded managed-runtime log display
      runtime_status.py        # Read-only status aggregation for diagnostics
      projects_cli.py          # `projects` code-index commands
      graph_auto_cli.py        # `projects auto` / `knowledge auto` on-off switches
      key_management.py        # Persistent local API-key operations
      package_management.py    # Installer detection and registry checks
      skill_install.py         # Installs the bundled marm-init skill into agent folders
      docker_cli.py            # Docker parser registration and dispatch
      docker_commands.py       # Safe Docker command planning and execution
    utils/
      dependency_check.py      # Runtime dependency validation
      helpers.py               # Shared helpers
      logging_filters.py       # Process logging noise filters
      multiprocess_guard.py    # Unsupported multi-worker runtime warning
      security.py              # API key generation
      embedding_state.py       # Inspect persisted embedding compatibility, no runtime init
      embedding_migration.py   # Resumable stopped-server embedding vector migration
      chunk_backfill.py        # Stopped-server backfill of memory_chunks after config change
  marm_graph/                  # Embedded marm-graph wrapper: subprocess JSON-RPC client,
                               #   tool router, and backend verification for the pinned
                               #   codebase-memory-mcp binary
  tests/                       # MCP server test suite
  Dockerfile                   # One image, HTTP default, STDIO override
  pyproject.toml               # Package metadata and console scripts

docs/                          # User-facing docs and project docs
scripts/                       # Local validation, release, and maintenance helpers
marm-console/                  # Standalone local Console app in active development (:8002)

Key Patterns

HTTP and STDIO are separate transports

HTTP mode lives in marm_mcp_server/server.py and is mounted through FastAPI/FastApiMCP at /mcp.

STDIO mode lives in marm_mcp_server/server_stdio.py and uses the official MCP Python SDK over standard input/output. It owns the FastMCP app and registers the seven core tools first; services/stdio_graph_tools.py supplies the graph/concept tool bodies through explicit registration so tools/list order remains stable. STDIO must keep stdout clean for JSON-RPC messages; logs and incidental print() output belong on stderr.

If a tool behavior changes, check whether the HTTP endpoint and STDIO tool both need the same update.

Separate transports means separate processes. Both run the same background indexers (concept extraction and code re-indexing) against the same databases, so anything they touch needs mutual exclusion that reaches across processes: a leased row in the memory database, not an asyncio.Lock or a module-level threading.Lock. core/lease_lock.py is that primitive, and a run of the test suite is not enough to catch a mistake here, since one interpreter never exercises the boundary. The two-process tests in tests/test_concept_two_process.py and tests/test_graph_auto_index.py spawn a real second interpreter for exactly that reason.

Docker HTTP requires an API key

Docker HTTP binds inside the container with SERVER_HOST=0.0.0.0, and host requests arrive through Docker bridge networking rather than 127.0.0.1. Always pass MARM_API_KEY for Docker HTTP.

docker run -d --name marm-mcp-server `
  -p 127.0.0.1:8001:8001 `
  -e SERVER_HOST=0.0.0.0 `
  -e MARM_API_KEY=your-generated-key `
  -v ${HOME}\.marm:/home/marm/.marm `
  lyellr88/marm-mcp-server:latest

Docker STDIO does not use an HTTP key

Docker STDIO launches a one-client process and does not expose an HTTP listener.

docker run -i --rm `
  -v ${HOME}\.marm:/home/marm/.marm `
  lyellr88/marm-mcp-server:latest `
  python -m marm_mcp_server.server_stdio

Use Docker HTTP for shared or multi-agent workflows. Use STDIO for private single-client local workflows. Multiple STDIO containers can point at the same mounted SQLite database, but heavy concurrent writes may hit normal SQLite lock contention.

Local HTTP defaults to loopback

SERVER_HOST defaults to 127.0.0.1. Local loopback HTTP is intended for same-machine use and does not require a key unless MARM_API_KEY is set.

If SERVER_HOST=0.0.0.0, MARM requires a key. When no key is provided, the settings layer auto-generates one and stores it in ~/.marm/.env.

SQLite schema changes need extra care

MARM uses a local SQLite database under ~/.marm/ by default. Tool behavior depends on specific tables for sessions, log entries, notebook entries, memories, compaction staging, and analytics.

Memory, log, and notebook rows can carry nullable project and platform attribution columns. When changing write paths, recall filters, consolidation, or schema migrations, keep these fields aligned across HTTP, STDIO, tests, and docs.

Do not rename fields, move data between tables, or change date/session parsing behavior without updating HTTP tools, STDIO tools, tests, smoke scripts, and docs together.

Retired features stay retired unless re-scoped

Current supported connection paths are HTTP and STDIO. Do not reintroduce retired transports or auth shims unless there is a new spec and implementation plan for them.

Adding or Changing MCP Tools

  1. Find the current HTTP behavior in marm_mcp_server/endpoints/.
  2. Find the matching STDIO behavior in marm_mcp_server/server_stdio.py.
  3. Keep request/response field names aligned where possible.
  4. Prefer parameterized actions for closely related operations, following existing tools such as marm_notebook(action=...), marm_delete(type=...), and marm_compaction(action=...).
  5. Update or add focused tests in marm-mcp-server/tests/.
  6. Update docs if the command shape, transport setup, auth behavior, or user-facing workflow changes.
  7. Run the local test runner before submitting changes.

Testing

Run the known-good local test checks from the repo root:

python scripts/run-tests.py

This runs:

  • Python compile check for marm_mcp_server and tests
  • Pytest suite with a controlled temp directory

For targeted ad hoc pytest runs, use --basetemp C:\tmp\... or clean repo-local pytest artifacts afterward:

powershell -ExecutionPolicy Bypass -File scripts\clean-pytest-artifacts.ps1

Current expectations:

  • Failed: 0 required before submitting a PR
  • Docker tests may skip automatically when Docker or the smoke image is unavailable
  • Warnings should be reviewed, but a known dependency warning may not block a PR

Run release preflight before a push or release:

python scripts/release-preflight.py

This runs the version scan, stale docs scan, known-good test runner, optional Docker smoke test, and a git status summary.

Run Docker smoke directly when changing Docker, transport setup, auth, or startup behavior:

python scripts/test-scripts/docker-smoke.py

For changes to Docker bind mounts, container users, HOME, cache paths, or data persistence, also run the Linux-only smoke test. It verifies that a host-owned mounted database can be written through HTTP and survives a container restart.

Run it from a native Linux host or WSL2 with Docker Desktop WSL integration enabled. The script creates its temporary mounted data directory under Linux /tmp; do not change that location to /mnt/c, or the UID/GID assertion is no longer meaningful:

bash scripts/test-scripts/docker-linux-bind-mount-smoke.sh

The script uses the latest official image by default. To test a locally built image instead:

MARM_DOCKER_SMOKE_IMAGE=marm-mcp-server:smoke bash scripts/test-scripts/docker-linux-bind-mount-smoke.sh

Documentation

Update docs when changing:

  • Install commands
  • Docker behavior
  • Client transport commands
  • API key behavior
  • Tool request/response fields
  • Database behavior
  • Version numbers
  • Roadmap or support status

Useful maintenance scripts:

python scripts/find-versions.py
python scripts/find-dead-code.py

find-versions.py is interactive and can update active version references. It intentionally avoids changing CHANGELOG.md because that file contains historical versions. find-dead-code.py looks for unused functions and classes in the MCP server codebase. Review its findings carefully before removing any code, as some utilities may be used in dynamic ways or reserved for future features.

Submitting Changes

MARM uses a PR-first workflow for normal development. Do not push feature, fix, or release-prep work directly to MARM-main.

  1. Create a focused branch from MARM-main.
  2. Keep the change scoped to one feature, fix, or doc cleanup.
  3. Follow existing file patterns before adding new abstractions.
  4. Run python scripts/run-tests.py.
  5. Run Docker smoke if the change touches Docker, HTTP/STDIO startup, auth, or transports.
  6. Update docs and changelog when user-facing behavior changes.
  7. Push the branch and open a PR into MARM-main.
  8. Wait for CodeRabbit and GitHub checks, then address review findings before merge.

No formal style guide beyond this: keep code readable, preserve current behavior unless the PR is explicitly changing it, and avoid broad refactors mixed into feature work.

Branch Naming

Use short, descriptive branch names:

feature/notebook-polish
fix/dependency-range
docs/install-cleanup
release/v2.6.3

Release Flow

Publishing is tag-driven. Merging a PR into MARM-main does not publish PyPI, Docker, or the MCP Registry by itself.

Release sequence:

branch → PR → CodeRabbit review → merge to MARM-main → tag vX.Y.Z → publish workflow

After the PR is merged and MARM-main is clean:

git checkout MARM-main
git pull
git tag -a vX.Y.Z -m "Release vX.Y.Z"
git push origin vX.Y.Z

The v* tag triggers the publish workflow for:

  • PyPI package publish
  • MCP server Docker image
  • MCP Registry publish

Use normal branch pushes for review. Use tag pushes only for intentional releases.

Project Documentation

Usage Guides

  • README.md - Complete MCP server usage guide with commands, workflows, and examples
  • PROTOCOL.md - Quick start commands and protocol reference
  • FAQ.md - Answers to common questions about using MARM

MCP Server Installation

Project Information