Add Cursor token usage sync from state.vscdb to runtime log by jqdsouza · Pull Request #294 · Asymptote-Labs/agent-beacon

jqdsouza · 2026-07-12T21:01:44Z

Summary

Adds beacon token-usage sync-cursor, a new command that extracts runtime-recorded token usage from Cursor's local state store (state.vscdb) and appends it to the Beacon runtime log as canonical token.usage events. This enables token usage reporting for Cursor sessions, which currently carry no token counts in hook payloads.

Key Changes

New cursorusage package (cli/beacon/internal/cursorusage/):
- db.go: Platform-specific Cursor state database path resolution and snapshot handling (copies DB + WAL to temp directory for safe, offline reading)
- extract.go: Extracts generations from the cursorDiskKV table with schema-drift tolerance (handles multiple token container and field name spellings across Cursor versions)
- map.go: Converts extracted generations to canonical token.usage endpoint events with cursor harness attribution
- sync.go: Orchestrates the sync sweep with dedup state tracking and idempotent re-runs
- state.go: Manages per-composer dedup state (JSON file) and log-based state recovery
New CLI command (cli/beacon/cmd/token_usage_sync_cursor.go):
- beacon token-usage sync-cursor with flags for DB path, state file, log path, dry-run (--print), timestamp filtering (--since), and state rebuild
- Resolves Cursor's per-OS state database location by default
- Integrates with existing endpoint lifecycle and writer infrastructure
Hook payload usage extraction (cli/beacon-hooks/cmd/genai_usage.go):
- New extractGenAIUsage() function that normalizes token usage from hook payloads across multiple container and field name spellings
- applyGenAIUsageFields() merges extracted usage into gen_ai.usage for Cursor hooks (afterAgentThought, beforeSubmitPrompt, recordLocalEdit)
- Handles numeric strings, negative counts, and nested container precedence
Comprehensive test coverage:
- cursorusage_test.go: Tests extraction with schema variations, zero-count skipping, parse error handling, WAL reading, event serialization, and idempotent syncing
- genai_usage_test.go: Tests usage extraction from various payload shapes and hook integration
- token_usage_sync_cursor_test.go: Tests command registration and integration with token usage reporting
Documentation (docs/cli/token-usage-sync-cursor.mdx):
- Command overview, how it works, examples, and flag reference
- Updated token-usage.mdx and cursor.mdx to reference the new sync capability

Implementation Details

Best-effort extraction: Unparseable rows and generations without recorded token counts are skipped and counted in the summary; Beacon never estimates token counts
Idempotent syncing: Synced generations are tracked in a local state file (~/.beacon/cursor/usage-sync-state.json) and skipped on re-runs; state is rebuilt from the runtime log if lost
No double counting: Synced events carry raw.metric_name=cursor.db.token.usage so they are suppressed if Cursor ever ships usage in hook payloads (hook events take precedence in rollups)
Read-only and offline: Snapshots the state database before reading, never touches live Cursor files, writes only to the local runtime JSONL log
Dedup integration: Token usage aggregation now recognizes and dedupes cursor sync events against hook-reported usage in the same (harness, session) scope

https://claude.ai/code/session_01EHe58DhhybqUq5Yxw4yA3J

Note

Medium Risk
Touches local SQLite reads and appends to the runtime JSONL log with new dedup semantics; mistakes could mis-attribute or double-count tokens, though the design is read-only against Cursor and prefers hook usage over sync.

Overview
Adds offline Cursor token coverage via beacon token-usage sync-cursor, which snapshots Cursor's local state.vscdb (including WAL), best-effort reads per-generation counts from cursorDiskKV, and appends canonical token.usage events with gen_ai.usage, harness.name=cursor, and session.id matching hook conversation ids. Zero-count rows are skipped; dedup state (~/.beacon/cursor/usage-sync-state.json) plus log rebuild keep re-runs idempotent; --print, --since, and --rebuild-state are supported.

Forward-compatible hook path: new extractGenAIUsage / applyGenAIUsageFields normalize assorted payload spellings into gen_ai.usage on common hook emitters (including file edits via recordLocalEdit); model_id is accepted for model attribution.

Rollup contract: synced events use raw.metric_name=cursor.db.token.usage so beacon token-usage / dashboard dedupe suppresses them when hook-reported usage exists for the same harness/session. Docs and CLAUDE.md describe the Cursor gap and sync workflow; modernc.org/sqlite backs read-only DB access.

^{Reviewed by Cursor Bugbot for commit c35a975. Bugbot is set up for automated code reviews on this repo. Configure here.}

…sync Cursor's hook payloads carry no token counts, so token views had no Cursor coverage. This adds the two local-only mechanisms: - Hook passthrough: beacon-hooks now normalizes token-usage fields from any hook payload into canonical gen_ai.usage (OTel GenAI semconv names) via a tolerant alias extractor wired into emitHookEvent and recordLocalEdit. Inert today; activates automatically if Cursor ships usage in hooks. - beacon token-usage sync-cursor: an explicit, read-only, offline command backed by internal/cursorusage that snapshots Cursor's state.vscdb, extracts runtime-recorded per-generation token counts from bubble rows, and appends token.usage events (harness=cursor, session=conversation id) through the endpoint writer. Idempotent via a per-bubble state file with rebuild-from-log recovery; zero-count generations are skipped, never estimated; cost is never derived locally. Synced events carry raw.metric_name=cursor.db.token.usage so the existing channel dedupe in internal/tokens prefers future hook-reported usage per (harness, session); a regression test locks the contract in. Adds modernc.org/sqlite (pure Go, CGO_ENABLED=0-compatible) to cli/beacon only. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EHe58DhhybqUq5Yxw4yA3J

mintlify · 2026-07-12T21:01:47Z

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated (UTC)
asymptote-labs	🟢 Ready	View Preview	Jul 12, 2026, 9:02 PM

cursor

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{Bugbot Autofix is ON. A cloud agent has been kicked off to fix the reported issues. You can view the agent here.}

^{Reviewed by Cursor Bugbot for commit c35a975. Configure here.}

cursor · 2026-07-12T21:03:55Z

+		if err := state.RebuildFromLog(opts.LogPath); err != nil {
+			return sum, fmt.Errorf("rebuild state from log: %w", err)
+		}
+	}


Partial rebuild saves broken state

High Severity

When RebuildFromLog fails after marking some bubbles, the deferred Save still persists that partial state. The next run sees a non-empty Composers map, skips rebuild, and re-emits already-logged generations that were never marked, double-counting tokens in rollups.

Additional Locations (1)

cli/beacon/internal/cursorusage/state.go#L86-L123

^{Reviewed by Cursor Bugbot for commit c35a975. Configure here.}

cursor · 2026-07-12T21:03:55Z

+		}
+		if g.Timestamp.IsZero() {
+			g.Timestamp = now
+		}


Undated generations bypass since filter

Medium Severity

--since only skips generations that already have a non-zero timestamp. Bubbles with TimestampSource=sync always pass the filter, then receive now after sorting, so old undated rows still sync and are written before older dated events despite the chronological append contract.

^{Reviewed by Cursor Bugbot for commit c35a975. Configure here.}

jqdsouza self-assigned this Jul 12, 2026

mintlify Bot deployed to staging - docs July 12, 2026 21:02 View deployment

cursor Bot reviewed Jul 12, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add Cursor token usage sync from state.vscdb to runtime log#294

Add Cursor token usage sync from state.vscdb to runtime log#294
jqdsouza wants to merge 1 commit into
mainfrom
claude/cursor-token-usage-6sui0o

jqdsouza commented Jul 12, 2026 •

edited by cursor Bot

Loading

Uh oh!

mintlify Bot commented Jul 12, 2026 •

edited

Loading

Uh oh!

cursor Bot left a comment •

edited

Loading

Uh oh!

cursor Bot Jul 12, 2026

Uh oh!

cursor Bot Jul 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

jqdsouza commented Jul 12, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Key Changes

Implementation Details

Uh oh!

mintlify Bot commented Jul 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cursor Bot Jul 12, 2026

Choose a reason for hiding this comment

Partial rebuild saves broken state

Uh oh!

cursor Bot Jul 12, 2026

Choose a reason for hiding this comment

Undated generations bypass since filter

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jqdsouza commented Jul 12, 2026 •

edited by cursor Bot

Loading

mintlify Bot commented Jul 12, 2026 •

edited

Loading

cursor Bot left a comment •

edited

Loading