Skip to content

test(routes): integration-grade route-test harness with real auth middleware (#3962)#3986

Merged
Yeraze merged 8 commits into
mainfrom
feature/3962-p1-route-test-harness
Jul 7, 2026
Merged

test(routes): integration-grade route-test harness with real auth middleware (#3962)#3986
Yeraze merged 8 commits into
mainfrom
feature/3962-p1-route-test-harness

Conversation

@Yeraze

@Yeraze Yeraze commented Jul 7, 2026

Copy link
Copy Markdown
Owner

Summary

Task 1.3 of the remediation plan (#3962, Phase 1).

Route tests previously monkey-patched the permission layer ((databaseService as any).checkPermissionAsync = vi.fn()...), so the real middleware chain was never exercised. This adds the shared fixture and converts 4 representative files as the template:

  • New src/server/test-helpers/routeTestApp.tscreateRouteTestApp({ mount }) boots a real Express app with real express-session (MemoryStore), real optionalAuth()/requireAuth/requirePermission, against the live in-memory-SQLite databaseService singleton (full migration chain — the vitest env already points the singleton at :memory:). Returns seeded admin/limited/anonymous users, two sources, loginAs() (session minted via a test-only login route — no bcrypt), per-source grant(), and cleanup().
  • Converted: sourceRoutes.permissions (14 tests), v1/messages.permissions (3), channelDatabaseRoutes.permissions (11), waypoints (16). Each keeps its original scenarios and gains genuine source-isolation assertions — granted on source A → 200, denied on source B → 403/filtered — driven by real permission rows and real getUserPermissionSetAsync scoping, not mock lambdas.
  • CLAUDE.md documents the harness as the standard for new/changed route tests; the monkey-patch pattern is deprecated for routes. No mass conversion (per plan) — 64 files with the old pattern convert opportunistically.

The conversion immediately paid for itself: exercising the real route code surfaced a production bug — GET /api/v1/messages without a sourceId param 500'd against the Task-1.1 fail-closed guard (two call sites in v1/messages.ts missing the ?? ALL_SOURCES disposition their sibling line already had). Fixed here with the intentional-cross-source comment.

Also includes a harness-robustness fix: poll for the fire-and-forget ensureAnonymousUser() seeding after waitForReady() (constructor race under fresh per-file module instances).

Verification (independent orchestrator run)

  • npm run typecheck: clean. npm run typecheck:tests: 283 (== baseline, no increase).
  • Full suite: 8120 passed, 0 failed, 0 suite failures (success: true).
  • Harness and converted files contain zero .prepare/.exec/.query raw SQL and zero monkey-patched permission fakes.

Refs #3962

🤖 Generated with Claude Code

https://claude.ai/code/session_01AphLfgUJWaFNAgU5UXdJ4n

Yeraze and others added 8 commits July 7, 2026 11:03
…rness

Introduces src/server/test-helpers/routeTestApp.ts — a real-middleware
harness for route tests that replaces the monkey-patch pattern
(vi.mock('../../services/database.js', ...)).

Key design decisions:
- Uses the live DatabaseService singleton (already :memory: + migrations)
  so authMiddleware sees the same data as the harness — a second
  createTestDb() connection would be invisible to the middleware.
- Real express-session (MemoryStore) + real optionalAuth() + real
  requirePermission() — the full auth/permission chain executes.
- Programmatic login via POST /__test__/login (no bcrypt) keeps tests fast.
- Per-source grants via db.auth.createPermission with sourceId so real
  checkPermissionAsync enforces isolation.
- Unique per-call usernames (rt-admin-N, rt-limited-N) prevent collisions
  in the file-persistent singleton DB.
- cleanup() removes permissions + sources in afterEach.

Part of remediation epic #3962, Phase 1 Task 1.3 WP-A.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AphLfgUJWaFNAgU5UXdJ4n
…ness (WP-A)

Removes the vi.mock('../../services/database.js') block and the hand-rolled
checkPermissionAsync lambda that re-implemented per-source scoping logic in
the test itself. Replaces with createRouteTestApp() so the real
checkPermissionAsync + permissions.sourceId SQL enforces isolation.

Changes:
- Delete vi.mock of services/database.js and mockDb alias
- Replace createApp() + hand-rolled session middleware with harness
- beforeEach: seed real permission rows via harness.grant()
- afterEach: harness.cleanup() removes perms + sources
- Keep vi.mock of sourceManagerRegistry + meshtasticManager (non-DB, correct)

New assertions added:
- admin bypass: loginAs(admin) → 200 on both sources without explicit grants
- anonymous fallback: loginAs(null) → 403 (real findUserByUsernameAsync path)
- source isolation proven: limited user granted on sourceA gets 200/200 on A
  and 403/403 on B, driven purely by real SQL rows (no mock lambda)
- channel filtering regression: same node seeded on both sources; user with
  channel_0:viewOnMap only on sourceA sees node on A (len≥1), filtered on B
  (len=0), driven by real getUserPermissionSetAsync per-source scoping

Fixes the gap where the previous mock passed while the real SQL was broken
(issue #3745 root cause).

Part of remediation epic #3962, Phase 1 Task 1.3 WP-A.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AphLfgUJWaFNAgU5UXdJ4n
Adds a 'Route Test Harness' subsection to the 'Test Mocking Pattern'
section in CLAUDE.md explaining when and why to use the harness vs the
legacy monkey-patch pattern, and pointing to sourceRoutes.permissions.test.ts
as the canonical template.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AphLfgUJWaFNAgU5UXdJ4n
DatabaseService.ensureAnonymousUser() is a fire-and-forget async task
launched from the constructor; readyResolve() fires synchronously before
it completes. Under vitest singleFork+isolate a new module instance (and
therefore a new :memory: SQLite) is created for each file, so the
constructor re-runs and the first beforeEach may reach getUserByUsername
before ensureAnonymousUser commits the row.

Add a 2-second polling loop (25 ms intervals) after waitForReady() so
the harness waits for the async seeding to land before asserting the
anonymous row exists. Resolves the intermittent race that caused all
channelDatabaseRoutes + waypoints tests to fail in the combined 3-file run.
…sages

withSourceScope() now throws when given undefined (introduced in commit
64081b6 — "withSourceScope fails closed"). getMessages() and
getMessagesByChannel() did not apply the ?? ALL_SOURCES guard that
getMessagesAfterTimestamp() already carried, so a GET /v1/messages
without a sourceId query-param produced an unhandled 500.

Add the ?? ALL_SOURCES fallback to both call sites; the comment
"intentional cross-source when sourceId omitted" matches the pattern
already established by the existing guard on the third path.

Exposed by the real-harness conversion of messages.permissions.test.ts.
…ess (WP-B)

Remove vi.mock('../../services/database.js') and replace with
createRouteTestApp(useOptionalAuth: false) + req.user injection
(mirrors the requireAPIToken pattern used in production).

vi.spyOn passthroughs on checkPermissionAsync and
getUserPermissionSetAsync record call arguments while the real SQL
logic runs — proving both the routing path taken (per-source vs global)
and the actual permission enforcement (no hand-rolled lambda).

Source-isolation assertion: limited user holds messages:read +
channel_0:read on rt-source-a only. GET ?sourceId=rt-source-b returns
200 with count=0 because every getAccessibleChannels call returns false
from real checkPermissionAsync — previously a () => sourceId==='rt-source-a'
mock would have hidden any regression (issue #3745).

Also revealed and fixed a production bug: getMessages() and
getMessagesByChannel() were missing the ?? ALL_SOURCES sentinel, causing
a 500 on requests without a sourceId param (see companion fix commit).
…eware harness (WP-B)

Remove vi.mock('../../services/database.js') and vi.mock('../auth/authMiddleware.js')
blocks. Real requireAuth() reads session.userId set by harness.loginAs();
real checkPermissionAsync enforces the permission gate against live DB rows.

Non-DB service mocks kept: channelDecryptionService and
retroactiveDecryptionService avoid file/network I/O in tests. Data repo
methods (getAllAsync, getByIdAsync, etc.) are spied on per-test to
control returned rows while auth still runs against the real DB.

Source-isolation assertion: channel_database is a global-by-design
resource (no sourceId column). The isolation proof here is therefore
grant/no-grant: user WITH global channel_database:read gets 200;
user WITHOUT gets 403 from real checkPermissionAsync (no NULL-sourceId
row → false). Previously mockResolvedValue(false) could mask an
implementation regression.

11 tests across 4 describe blocks covering read/write permission
filtering, per-entry PSK masking, admin bypass, and the isolated
source-isolation pair.
Remove vi.mock('../../services/database.js'). Real requirePermission()
calls checkPermissionAsync against live DB rows seeded per-test.

Non-DB mocks kept: sourceManagerRegistry (avoids TCP) and waypointService
(avoids file I/O). databaseService.waypoints.getAsync is spied on in
DELETE tests where no rows are seeded.

Source-isolation assertion: limited user holds waypoints:read only on
rt-source-a. GET rt-source-a → 200; GET rt-source-b → 403 from real
checkPermissionAsync (no row for source-b).

Write-success tests use harness.admin (real admin bypass in
requirePermission) rather than stacking a second grant() call on the
same limited user, which would violate the UNIQUE constraint on
(userId, resource, sourceId) and raise a DB error.

Unauthenticated test updated from expected 401 to 403: the real
anonymous user exists in the DB, so findUserByUsernameAsync('anonymous')
succeeds and requirePermission falls through to a false check (no grant),
returning 403 rather than the mock-induced 401 of the old test.

16 tests across 4 describe blocks covering GET/POST/PATCH/DELETE
with permission, admin-bypass, source-isolation, and input-validation
scenarios.
@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown

Claude finished @Yeraze's task —— View job


PR Review

  • Read changed files and gather context
  • Review routeTestApp.ts (new harness)
  • Review converted test files
  • Review messages.ts bug fix
  • Review CLAUDE.md updates
  • Post final review

This is a well-designed, high-value change. The harness correctly identifies and fixes a real testing anti-pattern. A few observations below, ranging from significant to minor.

routeTestApp.ts — the harness

Potential issue: grant() can silently fail with duplicate rows

The grant() helper unconditionally calls createPermission() (line 241) without checking for an existing (userId, resource, sourceId) row. If a test calls grant() twice for the same triple (easy to do in nested describes or during development), the second call will hit the UNIQUE constraint on SQLite and throw. The thrown error surfaces as an unhandled rejection in beforeEach, making failures hard to diagnose. Consider an upsert or a pre-delete in grant():

// belt-and-suspenders: remove the row first so createPermission never hits the UNIQUE constraint
await databaseService.auth.deletePermissionsForUserByScope(userId, sourceId ?? null).catch(() => {});
await databaseService.auth.createPermission({ ... });

Minor: revokeAll silently swallows errors but cleanup wraps its calls with .catch(() => {})

revokeAll (line 253) doesn't have a .catch(() => {}), but cleanup wraps both deletePermissionsForUser calls. If a test calls revokeAll and the user doesn't exist, it will throw. Given the singleton DB and unique-per-file users this is low-risk, but it's an inconsistency worth aligning.

Design note (not a bug): fixed source IDs across the whole file

SOURCE_A = 'rt-source-a' and SOURCE_B = 'rt-source-b' are constants, not counter-suffixed. cleanup() deletes and beforeEach recreates them, which is correct. The comment at line 138–141 explains this. Just noting it's intentional — having fixed names simplifies URL construction in tests.

loginAs(null) comment (line 273) is slightly misleading

The comment says findUserByUsernameAsync('anonymous') fallback path, but this path is exercised only when optionalAuth is mounted (line 227). When useOptionalAuth: false, loginAs(null) produces a request with no req.user at all — the route handler sees req.user as undefined. The JSDoc could acknowledge this distinction.

messages.ts — bug fix

Lines 124–129: the fix (?? ALL_SOURCES) is correct and the // intentional cross-source when sourceId omitted comments are helpful. The three call sites are now consistent with each other.

One follow-up worth noting: GET /:messageId at line 306 calls getMessages(10000, 0, msgLookupSourceId) without the ?? ALL_SOURCES fallback, which means it passes undefined when no sourceId is given. That's a fourth call site that wasn't fixed here. This was presumably pre-existing and may be covered by the repository's own default handling, but it's worth a look:

// line 306 — note: no ?? ALL_SOURCES
const allMessages = await databaseService.messages.getMessages(10000, 0, msgLookupSourceId);

Test files — converted

messages.permissions.test.ts — the spy-passthrough pattern (lines 70–71) is a nice touch: tests observe call routing while the real SQL runs. The source-isolation baseline (grant on A only, assert B returns count 0) is exactly the regression that motivated the harness.

sourceRoutes.permissions.test.ts — the parametric loop over endpoints (line 66–80) is clean. The /channels exclusion is well-documented with a reference to MM-SEC-7.

waypoints.test.ts — the note about the UNIQUE constraint on (userId, resource, sourceId) at line 22–26 is important for future test authors and matches the real schema. Good documentation.

channelDatabaseRoutes.permissions.test.ts — the hybrid approach (real checkPermissionAsync for gate enforcement + vi.spyOn on repo data methods) is the right split. Spying on getAllAsync etc. avoids seeding real channel rows while still exercising the real permission gate.

CLAUDE.md

The additions are accurate and well-placed. The deprecation notice for the monkey-patch pattern and the template reference to sourceRoutes.permissions.test.ts will help future contributors land correctly.

Summary

The harness design is sound, the converted tests are meaningful improvements over the monkey-patch pattern, and the messages.ts fix is correct. The main things worth addressing before merge:

  1. The grant() duplicate-key risk (silent UNIQUE constraint failure in beforeEach) — worth a defensive upsert or pre-delete.
  2. GET /:messageId (line 306 of messages.ts) missing ?? ALL_SOURCES — wasn't part of this PR's scope but is now visible; worth a follow-up ticket or fixing inline.

Comment on lines +209 to +217
session({
secret: 'route-test',
resave: false,
saveUninitialized: false,
cookie: { secure: false },
// Cookie name must match requireAuth()'s cookie sniff (authMiddleware.ts L232)
// so requireAuth-guarded routes receive the session correctly.
name: 'meshmonitor.sid',
})
Comment on lines +209 to +217
session({
secret: 'route-test',
resave: false,
saveUninitialized: false,
cookie: { secure: false },
// Cookie name must match requireAuth()'s cookie sniff (authMiddleware.ts L232)
// so requireAuth-guarded routes receive the session correctly.
name: 'meshmonitor.sid',
})
Comment on lines +222 to +225
app.post('/__test__/login', (req, res) => {
req.session.userId = req.body.userId ?? undefined;
req.session.save(() => res.status(204).end());
});
@Yeraze

Yeraze commented Jul 7, 2026

Copy link
Copy Markdown
Owner Author

CodeQL triage note (orchestrator): the three alerts (#159 clear-text cookie, #160 missing CSRF, #161 session not rotated on login) are all on src/server/test-helpers/routeTestApp.ts — a Vitest-only fixture (in-memory session store, ephemeral per-file database, test-only login route, never mounted in the production app). Recommend dismissing as used in tests; leaving the alerts open for a human decision.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants