Skip to content

fix(ccswitch-import): recover from malformed journal (readonly) + running cc-switch (db locked) - #132

Merged
weiesky merged 5 commits into
weiesky:mainfrom
souloss:fix/ccswitch-readonly-recovery
Jul 27, 2026
Merged

fix(ccswitch-import): recover from malformed journal (readonly) + running cc-switch (db locked)#132
weiesky merged 5 commits into
weiesky:mainfrom
souloss:fix/ccswitch-readonly-recovery

Conversation

@souloss

@souloss souloss commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes two distinct cc-switch import failures in readCcSwitchProviders (server/lib/ccswitch-import.js):

  1. attempt to write a readonly database (SQLite SQLITE_READONLY) — cc-switch killed mid-write leaves a malformed cc-switch.db-journal; SQLite must discard it (a write) to open the DB, refused under readOnly:true. Surfaced as 导入失败(未检测到 cc-switch 或读取出错): query failed: attempt to write a readonly database.
  2. database is locked (SQLite SQLITE_BUSY) — importing while cc-switch is open. A read-only connection's first query throws SQLITE_BUSY when cc-switch holds a BEGIN EXCLUSIVE write lock (its real contention mode). Surfaced as 导入失败(未检测到 cc-switch 或读取出错): query failed: database is locked.

Commit 6d71f27 — malformed journal → SQLITE_READONLY

  • Root cause (locked deterministically) — when cc-switch is killed mid-write (crash / power loss / task-kill), it can leave a malformed cc-switch.db-journal (truncated or corrupt rollback journal whose bytes are not a valid journal). SQLite must discard a malformed journal to open the DB — a write — which the readOnly:true connection refuses → SQLITE_READONLY. The error fires at the first prepare() (first page access triggers the journal check), not at open time.
  • Fix — keep the default read-only open, but on SQLITE_READONLY escalate once to a read-write open guarded by PRAGMA query_only = ON (lets SQLite recover and discard the corrupt journal, blocks our own writes), then retry. Mirrors what cc-switch itself does on its next normal launch. All other errors still surface their real cause (query failed: …, e.g. "file is not a database") — the existing "real cause must surface" contract is preserved.

Commit 7423438 — running cc-switch → SQLITE_BUSY (follow-up)

  • Root cause (reproduced deterministically against the live, running ~/.cc-switch/cc-switch.db) — the 6d71f27 fix only caught SQLITE_BUSY on its own escalation path; on the main read path BUSY escaped to the catch-all and surfaced as the opaque query failed: database is locked. Reproduced by running readCcSwitchProviders against the real DB while cc-switch (PID observed) was open:
    readOnly OPEN: OK
    readProviders threw: database is locked
    SURFACED: query failed: database is locked
    
  • Key SQLite findingBEGIN IMMEDIATE from a separate process does not block read-only readers (they see the pre-transaction snapshot); only BEGIN EXCLUSIVE does. cc-switch holds EXCLUSIVE, so the contention is real and intermittent (only when cc-switch is mid-write at the instant the import runs). This also corrects the 6d71f27 commit message's claim that "a valid hot journal is skipped cleanly under readOnly" — that holds only when cc-switch is not running; a valid hot journal under a running cc-switch's EXCLUSIVE lock throws SQLITE_BUSY.
  • Fix — detect SQLITE_BUSY on every path (the read-only open and the providers query), retry once after a 200ms backoff (cc-switch's write transactions are short — transient locks usually clear by then), and if still held surface the friendly cc-switch db is locked (cc-switch may be running); retry shortly instead of the raw query failed: database is locked wrapper.

Changes

2 commits, 3 files, +273 / −31.

  • server/lib/ccswitch-import.jsreadCcSwitchProviders gains (a) read-only → read-write+query_only escalation on SQLITE_READONLY (6d71f27), and (b) SQLITE_BUSY detection + single 200ms-backoff retry + friendly locked message on all paths (7423438).
  • test/ccswitch-import.test.js — two new describe blocks: (malformed journal recovery) plants a garbage -journal and asserts recovery + cleanup; (cc-switch running, valid hot journal) spawns a child process holding BEGIN EXCLUSIVE to deterministically reproduce both the held-lock (→ friendly message, no query failed: wrapper) and transient-lock (→ retry recovers providers) paths. Also corrects the misleading comment claiming a valid hot journal is "skipped cleanly".
  • history.md — two Unreleased bullets (one per failure mode); the 6d71f27 bullet's inaccurate "valid hot journal skipped cleanly" wording was corrected.

No touch to sensitive root shims (cli.js, interceptor.js, findcc.js, server.js) — this PR is one server lib + its test only.

How generated

This PR's code and commit history were produced through an iterative session with Claude Code (model: astron-code-latest (GLM5.2 by xfyun)). The session's prompts, in summary:

  • Root-cause investigation via systematic experimentation. Both failure modes were first observed on the real local cc-switch.db. For the READONLY case, several plausible theories (WAL -wal/-shm permission issues, read-only directory/file perms, valid hot rollback journal) were disproven by experiment; the trigger was locked only by planting garbage bytes as the -journal. For the BUSY case, the trigger was locked by running readCcSwitchProviders against the live DB while cc-switch was open, and the EXCLUSIVE vs IMMEDIATE distinction was confirmed via a cross-process child-holder repro (same-process writers do not contend — SQLite's per-process cache — so a real separate process is required to mirror cc-switch itself).
  • TDD. Both regression tests were written first and confirmed red (failing with the user's exact error string), then the fixes were applied to turn them green.
  • Fix design. For READONLY, escalation-to-read-write + query_only was chosen over a plain readOnly:false open (risks SQLITE_BUSY on the common path) or copying the 12MB DB every import (wasteful, can capture a torn page). For BUSY, single-retry-with-backoff + friendly message was chosen over indefinite retry (latency) or silent failure. File content changes are the author's; the model's role was investigation, test design, fix implementation, changelog prose, and PR drafting.

Verification

  • npm run build — ✅ passes
  • test/ccswitch-import.test.js (32 tests, incl. 2 malformed-journal + 2 BUSY cases) + test/api-ccswitch-import.test.js — ✅ 39/39 pass
  • test/server.test.js — ✅ 35/35 (no regression)
  • Against the live, running ~/.cc-switch/cc-switch.db (cc-switch open, 53KB valid hot journal) — 30 consecutive readCcSwitchProviders reads: all succeed on the happy path (cc-switch not mid-write); the held-lock and transient-lock paths are covered deterministically by the child-process tests.
  • npm run test:cli — pre-existing environment flakes only (EACCES / read-only-dir simulation tests that don't reproduce under root, plus im-lock / ask-store / git-diff / brotli-SSE flakes that also affect clean main). Zero new failures introduced by this PR.

⚠️ CI red on this PR = pre-existing brotli/SSE-compression flake (not this PR's regression)

The test check on the head commit may be red on 1-2 brotli / SSE-compression tests (br-negotiated stream decodes byte-identical to the plaintext stream, per-macrotask flush makes frames decodable without ending the stream, wire-compress streaming) — a different subtest flakes each run, the signature of a nondeterministic timing race. These tests live in test/sse-compression.test.js / test/wire-compress.test.js, files this PR does not touch (this PR is server/lib/ccswitch-import.js + its test + history.md only).

Evidence this is pre-existing, not introduced here:

  • All cc-switch tests pass in every CI run (malformed-journal recovery + both BUSY cases).
  • main's own CI is chronically red on the same flake — 5 of the last 6 CI runs on main failed (only the 2026-07-18 run succeeded). The same branch's prior commit 6d71f27 (before the BUSY work) passed CI (6d71f27 → success), so the flake is luck-of-the-draw, not a regression.
  • The flake is a known issue with a prior mitigation attempt (dd2fa26 "deterministic brotli decode in sse-compression broadcast test") that reduced but did not eliminate the race under GitHub Actions' scheduling; it passes reliably locally (5/5 on clean main).

The maintainer may rerun the test check (admin rerun rights) or merge despite this known-main flake. A proper fix for the brotli race belongs in a separate single-responsibility PR (it touches the SSE broadcast layer, out of scope for this cc-switch import fix).

souloss added 2 commits July 20, 2026 09:26
Importing from cc-switch failed with
"query failed: attempt to write a readonly database" when cc-switch had been
killed mid-write, leaving a malformed cc-switch.db-journal (truncated/corrupt
rollback journal). SQLite must discard a malformed journal to open the DB — a
write the read-only connection (readOnly:true) refused → SQLITE_READONLY.

Root cause locked deterministically: a *valid* hot journal is skipped cleanly
under readOnly (node:sqlite handles it); only a malformed one trips recovery.
So the trigger is a torn/partial journal from an unclean crash, not a clean one.

Fix in server/lib/ccswitch-import.js: keep the default read-only open (no
SQLITE_BUSY contention with a running cc-switch), but on SQLITE_READONLY escalate
once to a read-write open guarded by PRAGMA query_only = ON (lets SQLite recover
and discard the corrupt journal, blocks our own writes), then retry — mirroring
what cc-switch itself does on its next normal launch. SQLITE_BUSY on the
escalation path surfaces a clear "db is locked; retry shortly" message; other
errors still surface their real cause (query failed: …, e.g. file is not a database).

Test: new test/ccswitch-import.test.js cases plant a garbage -journal next to a
valid DB and assert recovery reads providers + cleans the journal, with a
follow-up plain read-only read proving the DB is left clean.

Build OK; ccswitch-import 30/30 + api-ccswitch-import 7/7; no new failures vs
clean-main baseline.
…+ friendly message

Importing from cc-switch WHILE IT IS OPEN failed with
  导入失败(未检测到 cc-switch 或读取出错): query failed: database is locked

Root cause (reproduced deterministically against the live, running cc-switch DB):
a read-only connection's first query throws SQLITE_BUSY ("database is locked") when
cc-switch holds a BEGIN EXCLUSIVE write lock — its real contention mode. The prior
malformed-journal fix (6d71f27) only caught SQLITE_BUSY on its OWN escalation path; on
the main read path BUSY escaped to the catch-all and surfaced as the opaque
`query failed: database is locked`. (The 6d71f27 commit message's claim that "a valid
hot journal is skipped cleanly under readOnly" was wrong — disproven by the live DB.)

Key SQLite finding: BEGIN IMMEDIATE from a separate process does NOT block read-only
readers (they see the pre-transaction snapshot); only BEGIN EXCLUSIVE does. cc-switch
holds EXCLUSIVE, so the contention is real and intermittent (only when cc-switch is
mid-write at the instant the import runs).

Fix in server/lib/ccswitch-import.js: detect SQLITE_BUSY on EVERY path (the read-only
open AND the providers query), retry once after a 200ms backoff (cc-switch's write
transactions are short — transient locks usually clear by then), and if still held
surface the friendly `cc-switch db is locked (cc-switch may be running); retry shortly`
instead of the raw wrapper.

Test: new test/ccswitch-import.test.js cases spawn a child process holding BEGIN
EXCLUSIVE to deterministically reproduce both paths — held-lock (→ friendly message,
no `query failed:` wrapper) and transient-lock (→ retry recovers providers). Also
corrected the misleading comment in the malformed-journal test (a valid hot journal
held by a running cc-switch is a SEPARATE failure mode, not something node:sqlite
"skips cleanly").

Verified: 32/32 ccswitch-import tests; 39/39 incl api-ccswitch-import; 35/35
server.test.js; 20/20 reads against the live running cc-switch DB (no lock errors on
the happy path). Build OK.
@souloss souloss changed the title fix(ccswitch-import): recover from malformed leftover journal causing readonly import failure fix(ccswitch-import): recover from malformed journal (readonly) + running cc-switch (db locked) Jul 20, 2026
@souloss
souloss force-pushed the fix/ccswitch-readonly-recovery branch from 1e674d6 to 7423438 Compare July 20, 2026 13:10
souloss and others added 3 commits July 20, 2026 22:07
…s now fixed in PR weiesky#135)

This branch's own changes (cc-switch SQLITE_BUSY import fix) were verified locally
and never the cause of CI red — every CI run on this branch failed only on the
pre-existing brotli/SSE-compression/portsBusy timing flake that affects main itself.
PR weiesky#135 (fix/ci-flakes-brotli-rmsync, now CI-green) eliminates that flake, so this
empty commit retriggers CI to confirm this branch is green with the flake gone.
@weiesky
weiesky merged commit a803382 into weiesky:main Jul 27, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants