Skip to content

feat(pam): real-time session log sync via incremental batch uploads#5965

Merged
bernie-g merged 18 commits intomainfrom
feat/pam-session-real-time-log-sync
Apr 9, 2026
Merged

feat(pam): real-time session log sync via incremental batch uploads#5965
bernie-g merged 18 commits intomainfrom
feat/pam-session-real-time-log-sync

Conversation

@bernie-g
Copy link
Copy Markdown
Contributor

@bernie-g bernie-g commented Apr 7, 2026

Context

Instead of uploading session logs in a single bulk request at the end of a session, the gateway now uploads events incrementally every 10 seconds. This unblocks live session monitoring and intervention in the UI, which previously had to wait until the session ended to show any activity.

On the backend, a new pam_session_event_batches table stores encrypted event batches keyed by session ID and byte offset. The session detail page polls every 5 seconds while a session is active. Re-uploads of the same byte range are idempotent via an upsert on (sessionId, startOffset). The existing encryptedLogsBlob read path is preserved for legacy sessions.

The corresponding gateway changes are in a separate CLI repo PR and must be merged after this one.

Screenshots

Steps to verify the change

  1. Start a PAM session
  2. Query SELECT * FROM pam_session_event_batches WHERE "sessionId" = '<id>' ORDER BY "startOffset" — rows should appear every ~10 seconds
  3. Open the session detail page — logs should refresh every 5 seconds while the session is active
  4. End the session and verify all events are present with no gaps
  5. Verify an old session with encryptedLogsBlob still returns logs correctly (legacy read path)

Type

  • Fix
  • Feature
  • Improvement
  • Breaking
  • Docs
  • Chore

Checklist

  • Title follows the conventional commit format: type(scope): short description (scope is optional, e.g., fix: prevent crash on sync or fix(api): handle null response).
  • Tested locally
  • Updated docs (if needed)
  • Updated CLAUDE.md files (if needed)
  • Read the contributing guide

Replaces end-of-session bulk upload with incremental 10-second batch
uploads from the gateway to enable live monitoring and session intervention.

- Add pam_session_event_batches table with (sessionId, startOffset) unique
  constraint for idempotent upserts
- Add pamSessionEventBatchDAL with findBySessionId and upsertBatch methods
- Add POST /v1/pam/sessions/:sessionId/event-batches route (identity-only)
- Update getById to read logs from batches first, falling back to legacy
  encryptedLogsBlob for older sessions
- Add PAM_SESSION_EVENT_BATCH_UPLOAD audit log event type
- Poll session page every 5s while session is active/starting
@maidul98
Copy link
Copy Markdown
Collaborator

maidul98 commented Apr 7, 2026

Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
Open Source Security 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

@bernie-g bernie-g marked this pull request as ready for review April 7, 2026 20:48
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Apr 7, 2026

Greptile Summary

This PR introduces incremental batch uploads of PAM session events: the gateway uploads event blobs every 10 seconds, the backend stores them in a new pam_session_event_batches table (idempotent upsert on (sessionId, startOffset)), and the session detail page polls every 5 seconds to display near-real-time logs. The legacy encryptedLogsBlob read path is preserved.

  • uploadEventBatch skips the org-level gateway permission check (OrgPermissionGatewayActions.CreateGateways) that updateLogsById and endSessionById both enforce; any identity token can pollute session logs when gatewayIdentityId is null.

Confidence Score: 3/5

Not safe to merge — the new batch-upload endpoint lacks org-level authorization, allowing any identity token to pollute session logs.

One P1 security finding: uploadEventBatch omits the org/gateway permission check present in all other gateway-only write operations. Two additional P2s (unhandled JSON.parse in decryptBatches, audit log flooding on polled endpoint) lower confidence further.

backend/src/ee/services/pam-session/pam-session-service.ts — requires the org-level permission check in uploadEventBatch before merging.

Important Files Changed

Filename Overview
backend/src/ee/services/pam-session/pam-session-service.ts Adds uploadEventBatch; missing org-level gateway permission check allows any identity to upload event batches to any session
backend/src/db/migrations/20260407000001_pam-session-event-batches.ts New migration for pam_session_event_batches table; down() lacks idempotency guard around trigger deletion
backend/src/ee/services/pam-session/pam-session-event-batch-dal.ts New DAL with upsert-on-conflict for idempotent batch uploads; straightforward and correct
backend/src/ee/services/pam-session/pam-session-fns.ts Adds decryptBatches helper; JSON.parse lacks try/catch, could surface 500 on corrupted batch data
backend/src/ee/routes/v1/pam-session-router.ts New event-batch upload route added correctly; GET /:sessionId emits audit log on every 5-second poll
backend/src/db/schemas/pam-session-event-batches.ts Auto-generated Zod schema for new table; correct
backend/src/ee/services/pam-session/pam-session-types.ts Adds TUploadEventBatchDTO type; clean
frontend/src/pages/pam/PamSessionsByIDPage/PamSessionByIDPage.tsx Adds 5-second polling for active sessions; correct use of refetchInterval with status check
backend/src/ee/services/audit-log/audit-log-types.ts Adds PamSessionEventBatchUploadEvent audit log type; correct

Comments Outside Diff (1)

  1. backend/src/ee/routes/v1/pam-session-router.ts, line 271-288 (link)

    P2 Audit log emitted on every 5-second poll

    The GET /:sessionId handler creates an audit log entry on every call. With the frontend polling every 5 seconds for active sessions, this generates 720 pam-session-get rows per hour per active session. No other comparable read/polling endpoint in the EE routes emits an audit log unconditionally on every GET. Consider removing the audit log from this endpoint or limiting it to first-access only.

Reviews (1): Last reviewed commit: "docs(pam): update session recording docs..." | Re-trigger Greptile

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 705acde1ff

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@bernie-g
Copy link
Copy Markdown
Contributor Author

bernie-g commented Apr 8, 2026

Note: The "Check API for Breaking Changes" check is throwing a false positive

bernie-g added 4 commits April 8, 2026 16:53
Replaces the all-at-once batch fetch in getById with a dedicated
GET /:sessionId/logs?offset&limit endpoint (max 100 batches/page).
Legacy blob-based sessions are unaffected and returned in full on
the first page. The frontend uses useInfiniteQuery with a load-more
button and polls every 5s for active sessions.
…GET audit log

Polling the session endpoint every 5s would decrypt all loaded batches
on each interval — expensive under load. Removing live refresh from
both the session query and the logs infinite query. Also restores the
PAM_SESSION_GET audit log entry that was dropped by d890840.
Parses the raw buffer as JSON and validates against the event union
schema so malformed batches are rejected at the API boundary rather
than failing silently at read time.
Uses Postgres xmax to detect whether the upsert was a fresh insert
or a re-upload of an existing offset. Only creates a PAM_SESSION_EVENT_BATCH_UPLOAD
audit log entry on first insert, not on retry overwrites.
@bernie-g bernie-g force-pushed the feat/pam-session-real-time-log-sync branch 2 times, most recently from 635f34e to 8c15154 Compare April 8, 2026 22:32
@bernie-g bernie-g force-pushed the feat/pam-session-real-time-log-sync branch from 97282ed to 05b16f3 Compare April 9, 2026 13:28
@bernie-g
Copy link
Copy Markdown
Contributor Author

bernie-g commented Apr 9, 2026

@claude review this

Copy link
Copy Markdown

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Code review skipped — your organization's overage spend limit has been reached.

Code review is billed via overage credits. To resume reviews, an organization admin can raise the monthly limit at claude.ai/admin-settings/claude-code.

Once credits are available, comment @claude review on this pull request to trigger a review.

bernie-g added 6 commits April 9, 2026 11:59
Initial load auto-paginates all existing batches. Active sessions poll
every 5s using a batch cursor so only new batches are fetched — no
re-decryption of existing logs. Load more button removed.
Active sessions poll every 5s advancing a batch cursor — bounded per
tick, catches up then tracks new batches. Completed sessions get a
single initial fetch with a manual load more button for history.
@bernie-g bernie-g merged commit d23bc63 into main Apr 9, 2026
14 checks passed
bernie-g added a commit that referenced this pull request Apr 9, 2026
…g-sync

feat(pam): real-time session log sync via incremental batch uploads
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants