You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Headline: engramx becomes the **extensible context spine**. Any MCP
16
+
server plugs in via a 10-line plugin file; every provider's output is
17
+
budget-weighted, mistake-boosted, and streamed progressively via SSE;
18
+
the mistakes moat grows two new capabilities (bi-temporal validity +
19
+
pre-mortem warnings); `engram gen` emits both `CLAUDE.md` AND `AGENTS.md`
20
+
by default. **Real-world benchmark: 89.1% measured savings** on engramx's
21
+
own 87-file sample (committed report in `bench/results/`).
22
+
23
+
Contributor credit: [@mechtar-ru](https://github.com/mechtar-ru) for PR #6
24
+
(OOM fixes on large codebases — cherry-picked with preserved authorship).
25
+
26
+
### Added — v3.0 "Spine" track
27
+
28
+
**Pillar 1 — Capabilities to add to it (extensibility foundation)**
29
+
-**Generic MCP-client aggregator** (`src/providers/mcp-client.ts`). Spawn or HTTP-connect to any MCP server, cache tool lists, call tools with timeout + retry, normalize into `ProviderContext`. Config at `~/.engram/mcp-providers.json`. Per-provider budgets, graceful degradation, process shutdown hooks. Uses `@modelcontextprotocol/sdk` v1.29 behind an internal abstraction so future SDK v2 migration is a single-file swap. Stdio transport ships; HTTP path stubbed pending post-3.0 Host/Origin hardening integration.
30
+
-**Provider plugin contract v2** (`src/providers/plugin-loader.ts`). Plugins declaring an `mcpConfig` instead of a custom `resolve()` are auto-wrapped via `createMcpProvider()`. Classic plugins with hand-rolled `resolve()` still work unchanged. Custom `resolve()` wins if both are present. 10-line plugins are now possible.
31
+
-**Budget-weighted resolver + mistakes-boost reranking** (`src/providers/resolver.ts`). Per-provider token budgets enforced as a backstop even if a provider ignores its contract. Results whose content mentions a known-mistake label get confidence × 1.5 (capped at 1.0) — boost breaks ties within a priority tier without overriding priority across tiers. Case-insensitive label matching.
32
+
33
+
**Pillar 2 — Save proper context**
34
+
-**Anthropic Auto-Memory bridge** (`src/providers/anthropic-memory.ts`). Reads Claude Code's auto-managed `~/.claude/projects/<encoded>/memory/MEMORY.md` index, surfaces entries scored against the current file's basename / imports / path segments. Tier 1, runs under 10 ms, max 1 MB hard-cap on index size. Override via `ENGRAM_ANTHROPIC_MEMORY_PATH` for tests + advanced users. Inserted at `PROVIDER_PRIORITY[3]` between mistakes and mempalace.
35
+
-**Streaming partial context packets via SSE** (`/context/stream?file=<path>` endpoint + `resolveRichPacketStreaming()` generator). Emit one SSE frame per provider as it resolves. Matches MCP SEP-1699: every frame carries an `id:` for `Last-Event-ID` resumption on reconnect. Client disconnect mid-stream aborts the generator cleanly. Inherits existing auth + Host + Origin guards.
36
+
-**Serena plugin reference** at `docs/plugins/examples/serena-plugin.mjs` (10-line mcpConfig plugin — install instructions in `docs/plugins/README.md`).
37
+
38
+
**Pillar 3 — Really help users (mistakes moat)**
39
+
-**Bi-temporal validity on mistake nodes**: schema migration 8 adds `valid_until` and `invalidated_by_commit` columns plus a partial index `idx_nodes_validity`. Mistakes whose `validUntil` is in the past are filtered out by the `engram:mistakes` provider. Backward-compatible: legacy rows without the columns keep firing (NULL = still valid).
40
+
-**Pre-mortem mistake-guard** (`src/intercept/handlers/mistake-guard.ts`). Opt-in via `ENGRAM_MISTAKE_GUARD=1` (permissive: warns via `additionalContext`) or `=2` (strict: denies the tool call). Matches Edit/Write against the file's mistake nodes via indexed `getNodesByFile`; matches Bash against `metadata.commandPattern` substrings and `sourceFile` mentions in the command. Respects the bi-temporal filter. Zero overhead when unset.
41
+
42
+
**Hygiene / ecosystem**
43
+
-`engram gen` emits BOTH `CLAUDE.md` AND `AGENTS.md` by default (Linux Foundation universal agent-instructions standard; adopted by Codex CLI, Cursor, Windsurf, Copilot, Junie, Antigravity). Explicit `--target=claude|cursor|agents` preserves single-file behavior.
44
+
- README opens with **"What engramx is not"** section — disarms collision with Go-Engram (Gentleman-Programming/engram), DeepSeek's "Engram" paper (Jan 2026), and MemPalace in the first 30 seconds of any new visitor read.
45
+
- PR #6 (`@mechtar-ru`) cherry-picked ourselves with preserved authorship: `MAX_DEPTH=100` in ast-miner's directory walk, `MAX_FILES_PER_COMMIT=50` in git-miner's co-change analysis, expanded default skip dirs. Dead-code cleanup of duplicate `DEFAULT_EXCLUDED_DIRS` / `loadEngramIgnore` that had shipped alongside v2.1's newer `DEFAULT_SKIP_DIRS` / `loadIgnorePatterns`. Closes issue #5.
46
+
47
+
### Proof — real-world benchmark (new, committed)
48
+
49
+
`bench/real-world.ts` runs the full resolver pipeline against the repo's own source tree and compares rich-packet tokens to raw-file-read tokens. Latest run (2026-04-24, 100-file scale-out, 87 files actually sampled after skip rules):
50
+
51
+
| Metric | Value |
52
+
|---|---|
53
+
| Baseline tokens (raw Read of every file) | 163,122 |
54
+
| engramx tokens (rich packets) | 17,722 |
55
+
| Aggregate savings |**89.1%**|
56
+
| Median per-file savings | 84.2% |
57
+
| Files where engramx saved tokens | 85 of 87 |
58
+
| Best case (`src/cli.ts`) | 98.4% (18,820 → 306) |
59
+
60
+
Reproducible by anyone, on any project: `npx tsx bench/real-world.ts --project . --files 50`.
61
+
62
+
### Changed
63
+
64
+
-`autogen()` return type: `{ file: string }` → `{ files: string[] }` (single caller in `cli.ts` updated). Consumers of the programmatic API who called `result.file` must read `result.files[0]` instead (or use `--target` to keep single-file semantics).
65
+
-`PROVIDER_PRIORITY` gains `anthropic:memory` at index 3 — downstream test that hard-coded the array order was updated.
66
+
-`MIGRATIONS` (src/db/migrate.ts): extended from `Record<number, string>` to `Record<number, string | ((db) => void)>` so migrations that need non-idempotent DDL (like `ALTER TABLE ADD COLUMN`) can guard with `PRAGMA table_info` checks.
**v2.1 → v3.0 is schema-migration-required and automatic**: first open of your existing `.engram/graph.db` triggers migration 8. A `.bak-v7` backup is written alongside. Legacy mistake rows survive unchanged (NULL `validUntil` = still valid). Verified on a simulated v2.1 DB during release audit.
72
+
73
+
**API consumers of `autogen()`** must update call sites: `result.file` (single string) → `result.files` (array). CLI callers are unaffected.
74
+
75
+
### Tests
76
+
77
+
771 → 876 passing (+105 new). CI green Ubuntu+Windows × Node 20+22. TypeScript `--noEmit` clean, lint clean.
Thanks for your interest in improving engram. Here's how to help.
3
+
Thanks for wanting to help. EngramX is Apache-2.0, welcomes every kind of contribution, and aims to stay meticulously honest about what works and what doesn't. This doc is short because the rules are few.
4
4
5
5
## Quick Start
6
6
@@ -12,38 +12,74 @@ npm run build
12
12
npm test
13
13
```
14
14
15
-
## What's Most Valuable
15
+
You need Node 20+. No native toolchain — EngramX's SQLite is sql.js WASM, all tree-sitter grammars are bundled as WASM. Zero system libraries required.
16
16
17
-
**Worked examples** are the highest-impact contribution. Run `engram init` on a real codebase, evaluate what the graph got right and wrong, and share the results in an issue or PR.
17
+
## Highest-impact contributions
18
18
19
-
**Language extraction bugs** — if engram misses a function, class, or import in a language it supports, open an issue with the source file and what was missed.
19
+
In rough order of "what helps the most":
20
20
21
-
**New language support** — add regex patterns to `src/miners/ast-miner.ts` following the existing pattern. Include test fixtures.
21
+
1.**Worked examples** — run `engram setup` on a real codebase, record what the graph got right and wrong, open an issue. Honest feedback from actual use is more valuable than any patch.
22
+
2.**Reproducible bench results** — run `npx tsx bench/real-world.ts --project . --files 50` on your project and share the numbers (especially if you see <50% savings — we want to understand why).
23
+
3.**Plugin submissions** — a 10-line MCP plugin file for a service we don't have yet. Drop in `docs/plugins/examples/` + mention the coverage in a PR.
24
+
4.**Language extraction bugs** — if `engram init` misses a function/class/import in a supported language, open an issue with the source file and what was missed.
25
+
5.**Windows-specific fixes** — EngramX CI covers Ubuntu × Node 20/22 AND Windows × Node 20/22. Windows-path bugs are real. We welcome patches that harden cross-platform behaviour.
26
+
6.**New language support** — tree-sitter grammar wiring in `src/miners/ast-miner.ts`, plus a test fixture.
22
27
23
-
## Development
28
+
## Development loop
24
29
25
30
```bash
26
-
npm run dev # Watch mode (auto-rebuild)
27
-
npx vitest # Run tests in watch mode
28
-
npx vitest run # Run tests once
29
-
npm run build # Production build
31
+
npm run dev # watch mode — rebuilds on every save
32
+
npx vitest # tests in watch mode
33
+
npx vitest run # tests once
34
+
npm run build # production build (bundled WASM grammars)
35
+
npm run lint # TypeScript strict check (tsc --noEmit)
36
+
npx tsx bench/real-world.ts # sanity-check your changes against the savings bench
30
37
```
31
38
32
-
## Before Submitting a PR
39
+
## Before you open a PR
33
40
34
-
1.`npm run build` passes
35
-
2.`npx vitest run` passes
36
-
3. If you changed extraction logic, add a test fixture and test case
37
-
4. Keep PRs focused — one change per PR
41
+
1.`npm run build` passes (TypeScript strict).
42
+
2.`npx vitest run` passes all suites (currently 878 on v3.0).
43
+
3. If you changed extraction logic, add a fixture + test case.
44
+
4.**If you touched anything that builds a filesystem path, assert with `path.join()` / `path.resolve()`, never hand-write `/` separators.** We shipped a Windows-CI regression on v3.0's first pass because of this. Tests that build an expected path via `path.join()` (matching the implementation) work on every platform — regex assertions with `\/` do not.
45
+
5. Keep PRs focused — one change per PR.
38
46
39
-
## Code Style
47
+
## Code style
40
48
41
-
- TypeScript strict mode
42
-
- ESM imports (`import`, not `require`)
43
-
- Immutable patterns (spread, not mutation)
44
-
- Functions under 50 lines
45
-
- No `console.log` in library code (only in CLI)
49
+
- TypeScript strict mode.
50
+
- ESM imports (`import`, not `require`). Vitest's CommonJS interop hides bare-`require()` bugs that crash in production — always use top-level ESM imports.
51
+
- Immutable patterns (spread, not mutation).
52
+
- Functions under ~50 lines.
53
+
- No `console.log` in library code — only in CLI entry points (`src/cli.ts`) and the bench runner.
54
+
- Every new test that exercises filesystem paths should explicitly include a Windows-native-path case so regressions surface locally, not only on CI.
55
+
56
+
## Plugin authors
57
+
58
+
Writing a context provider is ~10 lines. See [`docs/plugins/README.md`](docs/plugins/README.md) for the full spec. Two shapes are supported:
59
+
60
+
-**MCP-backed** — declare an `mcpConfig` and the loader spawns/connects to the MCP server for you. Any MCP server becomes an EngramX provider in one `.mjs` file.
61
+
-**Classic** — write your own `resolve()` + `isAvailable()` for full control.
0 commit comments