Skip to content

Commit da67604

Browse files
ashu17706ashu17706
committed
docs: upstream merge plan and outcome (May 2026)
Records the fast-forward sync of fork main from d58fedf to ddbd6bd (49 upstream commits, no fork divergence). Documents API risk audit showing zero breaking changes for Smriti consumers, execution steps, verification gates, and a rollback plan for future syncs.
1 parent ddbd6bd commit da67604

1 file changed

Lines changed: 211 additions & 0 deletions

File tree

‎docs/UPSTREAM_MERGE_PLAN.md‎

Lines changed: 211 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,211 @@
1+
# Upstream Merge Plan — May 2026
2+
3+
**Goal:** Bring `zero8dotdev/qmd` (fork) `main` from `d58fedf` up to `tobi/qmd` `main` at `ddbd6bd` (49 commits behind), then bump the submodule pointer in Smriti.
4+
5+
**Drafted:** 2026-05-18 · **Executed:** 2026-05-18 · **Status:** Complete
6+
7+
**Outcome:** Fork main fast-forwarded from `d58fedf` to `ddbd6bd`. QMD test suite: 868 pass / 0 fail. Stale branches `perf/addmessage-upsert-opt` and `refactor/move-memory-ollama-to-smriti` deleted from origin. Note: QMD process exits with SIGABRT on macOS due to a known Metal device cleanup issue during atexit (not a test regression — all tests pass before exit).
8+
9+
---
10+
11+
## 1. Current State
12+
13+
| Repo | Ref | Commit |
14+
|---|---|---|
15+
| Fork (zero8dotdev/qmd) | `origin/main` | `d58fedf` |
16+
| Upstream (tobi/qmd) | `upstream/main` | `ddbd6bd` |
17+
| Smriti submodule pointer | `qmd` (gitlink) | `d58fedf` |
18+
19+
**Critical finding:** `origin/main` is a **pure ancestor** of `upstream/main` — zero fork-specific commits on main. This is a clean fast-forward at the Git level.
20+
21+
```bash
22+
# Verified:
23+
git log --oneline upstream/main..origin/main # empty
24+
git merge-base origin/main upstream/main # = d58fedf
25+
```
26+
27+
### Other fork branches (stale, will be deleted)
28+
29+
| Branch | Status | Reason |
30+
|---|---|---|
31+
| `origin/perf/addmessage-upsert-opt` | Stale | Adds `memory.ts`/`ollama.ts` to QMD — both modules have since moved to `smriti/src/`. Obsolete. |
32+
| `origin/refactor/move-memory-ollama-to-smriti` | Stale | The move it describes is already complete in Smriti. Obsolete. |
33+
| Local `main` (commit `7ec50b8`) | Stale | Predates the modules-to-smriti move. **Diverged from `origin/main` — must be reset, not merged.** |
34+
| Local `sync-upstream` | Use for this work | Repurpose or recreate from `origin/main`. |
35+
36+
---
37+
38+
## 2. What's Changing (49 Upstream Commits)
39+
40+
### Categories
41+
42+
| Type | Count | Examples |
43+
|---|---|---|
44+
| Search/RRF correctness | 3 | `004714a` RRF weighting by query type, `d045a8b` CJK FTS, `5b9f472` embed collection filter |
45+
| Stability / GPU | 6 | `60c75cb` Metal cleanup abort, `e8229d8` Windows CUDA parallelism, `1f75737` GPU status |
46+
| Embedding | 4 | `910ca07` partial embeddings pending, `b59ba6a` cleanup lifecycle, model resolution |
47+
| CLI / MCP UX | 8 | `c18c74a` serve QMD skill from CLI, `9cecdc8` terse MCP collection summary, `e36ab96` HTTP rerank control |
48+
| Path / docid | 4 | `dff6513` preserve docids across case renames, `2dc8634` qmd:/// aliases, `aa1818e` clamp fromLine |
49+
| Bench / test infra | 6 | `2e0c743` local-index bench, Node+Bun matrix, lifecycle regression |
50+
| Routine merges | 18 | First-parent merges of the above |
51+
52+
### Risk Audit: APIs Smriti Imports from QMD
53+
54+
Smriti consumes QMD via the vendored submodule and imports these symbols directly from `../qmd/src/store.ts`, `../qmd/src/index.ts`, `../qmd/src/db.ts`:
55+
56+
```
57+
createStore, QMDStore — src/db.ts, src/store.ts
58+
initializeMemoryTables — src/qmd.ts (re-export from smriti/src/memory.ts)
59+
hashContent — src/qmd.ts, src/team/share.ts, src/team/sync.ts
60+
chunkDocumentByTokens, reciprocalRankFusion — src/memory.ts
61+
formatQueryForEmbedding, formatDocForEmbedding, RankedResult — src/memory.ts
62+
insertEmbedding — src/memory.ts
63+
Database (type) — src/db.ts, src/memory.ts
64+
```
65+
66+
**Diff scan of these symbols across the 49 commits:**
67+
68+
| Symbol | Change | Smriti impact |
69+
|---|---|---|
70+
| `insertEmbedding` | Added optional 7th param `totalChunks?: number` | None — backward compatible. Smriti's callsites continue to work; new behavior (partial-embedding pending state from `910ca07`) is opt-in. |
71+
| `formatQueryForEmbedding` | Callsite refactor only (passes resolved `embedModel` instead of `llm.embedModelName`). Signature unchanged. | None. |
72+
| `reciprocalRankFusion` | Unchanged. **But `hybridQuery` now weights "original"-type lists at 2x via `getHybridRrfWeights` (commit `004714a`).** | **Improvement.** Smriti calls `reciprocalRankFusion` directly with its own weights — unaffected. QMD callers of `hybridQuery` get better rankings. |
73+
| `createStore`, `QMDStore`, `hashContent`, `chunkDocumentByTokens`, `formatDocForEmbedding`, `RankedResult`, `initializeMemoryTables`, `Database` | No signature or semantic change. | None. |
74+
75+
**Conclusion: zero breaking changes for Smriti.**
76+
77+
### Behavioral changes worth knowing
78+
79+
- **CJK FTS migration (`d045a8b`):** First post-merge call into `documents_fts` rebuilds the FTS table to space-separate CJK characters. **One-time, on-disk migration** in `~/.cache/qmd/index.sqlite`. Smriti's recall path uses its own `memory_fts` table (separate from `documents_fts`), so Smriti recall is unaffected. Direct `qmd` CLI users will see a brief one-time delay on first query.
80+
- **RRF weighting fix (`004714a`):** Affects QMD's `hybridQuery` only. Smriti's `searchMemoryFTS` / `searchMemoryVec` / RRF fusion is a separate code path — not directly improved, but the new `getHybridRrfWeights` helper is a pattern worth porting later (see Future Work).
81+
- **Docid stability (`dff6513`):** Case-only renames now preserve docid. Smriti stores docids inside session messages — pre-merge docids remain valid post-merge.
82+
- **Partial embeddings pending (`910ca07`):** Crash mid-embedding-batch now leaves chunks marked pending instead of orphaning the document. Smriti doesn't use QMD's embed batch (uses its own loop), no impact.
83+
84+
---
85+
86+
## 3. Execution Plan
87+
88+
### Phase A — Fork repo (`zero8dotdev/qmd`)
89+
90+
```bash
91+
cd qmd
92+
93+
# 1. Confirm upstream is current
94+
git fetch upstream main
95+
git fetch origin
96+
97+
# 2. Move to a working branch off origin/main
98+
git checkout -B sync-upstream-2026-05 origin/main
99+
100+
# 3. Fast-forward to upstream/main
101+
git merge --ff-only upstream/main
102+
# ↑ MUST succeed without manual conflict resolution. If it fails,
103+
# re-verify that origin/main has no commits not in upstream/main:
104+
# git log --oneline upstream/main..origin/main
105+
# Should be empty. If non-empty, stop and re-plan.
106+
107+
# 4. Run the QMD test suite locally before pushing
108+
bun install
109+
bun test --preload ./src/test-preload.ts test/
110+
111+
# 5. Push to fork main (fast-forward push, no force needed)
112+
git push origin sync-upstream-2026-05:main
113+
```
114+
115+
### Phase B — Cleanup stale branches
116+
117+
```bash
118+
# Delete obsolete fork branches (after confirming nothing depends on them)
119+
git push origin --delete perf/addmessage-upsert-opt
120+
git push origin --delete refactor/move-memory-ollama-to-smriti
121+
122+
# Local cleanup
123+
git branch -D main # local main is stale (7ec50b8) — recreate
124+
git checkout main # tracks origin/main, now at ddbd6bd
125+
git branch -D sync-upstream # if no longer needed
126+
```
127+
128+
### Phase C — Smriti submodule bump
129+
130+
```bash
131+
cd /Users/zero8/zero8.dev/smriti
132+
133+
# 1. Update submodule pointer
134+
git submodule update --remote qmd
135+
# OR explicit:
136+
# cd qmd && git checkout main && git pull && cd ..
137+
138+
# 2. Reinstall (file: dep resolves the new commit)
139+
bun install
140+
141+
# 3. Run Smriti's full test suite — this is the real gate
142+
bun test
143+
144+
# 4. Smoke-test the CLI surfaces that touch QMD search:
145+
bun src/index.ts recall "test query" --check-conflicts
146+
bun src/index.ts search "test query"
147+
bun src/index.ts status
148+
149+
# 5. Commit the submodule bump
150+
git add qmd bun.lock
151+
git commit -m "chore(qmd): bump submodule to upstream main (ddbd6bd)
152+
153+
Pulls 49 upstream commits including RRF weighting fix (#004714a),
154+
CJK FTS support (#d045a8b), embed collection filter fix (#5b9f472),
155+
and macOS Metal cleanup stability (#60c75cb).
156+
157+
No breaking changes to APIs consumed by Smriti."
158+
```
159+
160+
---
161+
162+
## 4. Verification Gates
163+
164+
A merge is **only accepted** when all of these pass:
165+
166+
1. **QMD's own test suite** — `bun test --preload ./src/test-preload.ts test/` in `qmd/`
167+
2. **Smriti's full test suite** — `bun test` in `smriti/`
168+
3. **Type check** — `bunx tsc --noEmit` in `smriti/` (imports from `../qmd/src/...` must still resolve)
169+
4. **Smoke tests** for the three surfaces that touch QMD:
170+
- `smriti search` (FTS + Vec via `searchMemoryFTS` / `searchMemoryVec`)
171+
- `smriti recall` (RRF via `reciprocalRankFusion`)
172+
- `smriti ingest claude` (writes via `addMessage` / `hashContent` / `insertEmbedding`)
173+
5. **CHANGELOG note** in Smriti's CHANGELOG if it exists
174+
175+
If anything fails: do not push. Revert the submodule bump and triage.
176+
177+
---
178+
179+
## 5. Rollback Plan
180+
181+
The fork main move is a fast-forward push — easy to revert if a regression slips through:
182+
183+
```bash
184+
# Fork rollback (force-push back to d58fedf — only on main if no one else has pulled)
185+
cd qmd
186+
git push origin --force-with-lease d58fedf:main
187+
188+
# Smriti rollback (submodule pointer reverts cleanly)
189+
cd ../
190+
git revert <submodule-bump-commit>
191+
```
192+
193+
`--force-with-lease` is required because we're rewriting a fast-forward; refuses if anyone has pushed since. Coordinate with team before doing this.
194+
195+
---
196+
197+
## 6. Future Work (Out of Scope for This Merge)
198+
199+
These are improvements to consider after the merge lands, not blockers:
200+
201+
- **Port `getHybridRrfWeights` pattern to Smriti recall.** Smriti's RRF fusion uses positional weights similar to the pre-fix QMD logic. Same bug class applies — when query expansion runs first, expansion-derived lists can steal the original-query weight. Worth fixing in `smriti/src/memory.ts` and `src/search/recall.ts`.
202+
- **Reconsider keeping QMD as a submodule vs. upstreaming Smriti's QMD usage as a stable consumer.** The fork has zero divergence — if it stays that way, switch to `"qmd": "github:tobi/qmd#<commit>"` and drop the fork entirely.
203+
- **Wire the `--check-conflicts` recall path through `hybridQuery` instead of separate FTS+Vec+RRF.** Would get the upstream RRF fix for free.
204+
205+
---
206+
207+
## 7. Open Questions
208+
209+
- [ ] Is anything in the codebase still consuming `perf/addmessage-upsert-opt` or `refactor/move-memory-ollama-to-smriti`? (Check before deleting.)
210+
- [ ] Do we want to tag the fork at `v2.1.0-sync-2026-05` for traceability, or just rely on the commit hash in Smriti's submodule pointer?
211+
- [ ] Should the QMD test suite run in CI for this fork, or only at sync time?

0 commit comments

Comments
 (0)