feat: report funnel metrics on channel mcp - #52
Draft
Starllordz wants to merge 2 commits into
Draft
Conversation
## Changed
- Tool calls now emit a call_success or call_error, reported from inside
CallTool's own try/catch so the original SDK error is still available to
classify before it is rewritten
- Shutdown flushes the queued events before exiting, capped at 2s
- The cast through the SDK's protected `client` moved to a single guarded
accessor, reused by the existing X-Lara-Client headers
## New
- src/metrics.ts: batched, fire-and-forget event reporting to the
integrations monitoring backend. Buys a token from /auth/issue-token,
posts to /metrics/ingest-events every 2s, retries once on 401, requeues
on 429/5xx, drops on any other rejection, queue capped at 10 000. A
backend that is down never delays or fails a translation
- Events: install (first run only), auth_success (deduplicated per account
per process, since the HTTP transport builds a Translator per request),
call_success, call_error, and auth_fail beside a call_error Lara answered
with 401/403. Both transports report
- accountId is the Lara account behind the credentials, read from the id
claim of the token the SDK holds. No token means no event: a made-up id
would be a fake account in every dashboard
- Installation id persisted at ${LARA_HOME:-~/.lara}/installation-id, with
a per-process fallback when the home is not writable
- Opt-out via DO_NOT_TRACK: nothing is measured, queued, written to disk or
sent. METRICS_URL / METRICS_API_KEY override the compiled-in defaults,
which are empty until the backend is deployed
- 60 tests covering the client in isolation and the server wiring
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
There are a few correctness/documentation gaps (envelope stamping safety, queue cap enforcement on requeue, and README wording vs. actual “no token → no events” behavior) that should be addressed before approval.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds funnel-style usage metrics reporting for the MCP channel, integrating a self-contained telemetry client into both STDIO and HTTP transports without impacting translation behavior when the backend is unavailable or telemetry is disabled.
Changes:
- Introduces
src/metrics.tsto queue, batch, and deliver funnel events (install,auth_success,call_success/call_error,auth_fail) with opt-out and safe-failure semantics. - Wires metrics emission into MCP tool execution (
CallTool) and session setup, plus flush-on-shutdown handling. - Adds env support, documentation, and comprehensive Vitest coverage for client behavior and end-to-end MCP wiring.
File summaries
| File | Description |
|---|---|
| src/metrics.ts | New telemetry client: installation id, batching, token issuance, error classification, and per-call reporting. |
| src/mcp/tools.ts | Emits call success/error metrics from inside CallTool while preserving existing error rewriting. |
| src/mcp/server.ts | Creates per-session metrics context and centralizes SDK client access via getLaraClient(). |
| src/lara-client.ts | Adds a guarded accessor for the SDK’s internal client to centralize the protected cast. |
| src/index.ts | Flushes queued metrics during shutdown with a 2s cap. |
| src/env.ts | Adds metrics-related environment variables and opt-out configuration. |
| src/tests/server/mcp.metrics.test.ts | End-to-end wiring tests for MCP server metrics emission. |
| src/tests/metrics.test.ts | Extensive unit tests for metrics client behavior (queueing, batching, retries, opt-out, etc.). |
| README.md | Documents usage metrics, payload contents, and opt-out behavior. |
| CLAUDE.md | Updates repository guidance to reflect the new metrics module and tests. |
| .env.example | Documents new env vars for metrics configuration and opt-out. |
Review details
- Files reviewed: 11/11 changed files
- Comments generated: 3
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
## Changed - The event envelope is stamped last, so channel and sessionId always win over whatever a caller passes - A batch handed back after a failed delivery is trimmed to the queue cap immediately; a server going idle while the backend is down used to sit above the bound until the next event arrived - README no longer implies rejected credentials produce an auth_fail: no account is known in that case, so no event is sent at all ## New - Two tests, each verified to fail without its fix: the envelope resists a caller overriding it, and a requeued batch stays within the cap
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reports funnel metrics to the integrations monitoring backend on channel
mcp, so the gap between installs and actual usage can be measured. Both transports report — unlike the private server, which is HTTP-only because its installation id lives in Redis.Follows the shared event reporting standard (
src/modules/metrics/frontend.mdinlara-integrations-monitoring).Events
installaccountIdauth_successauth_failcall_errorLara answered with 401/403call_success/call_errorlatencyMsandcharsTranslatedauth_successis deduplicated per account per process: the HTTP transport is stateless and builds aTranslatorper request, which would otherwise report one on every tool call.What is sent
accountIdis the Lara account (acc_...) behind the credentials, read from theidclaim of the token the SDK holds. An access key is not an account — it belongs to one and is shared by its users — so reporting it would split the same customer across channels. No token means no account and no event: a made-up id is a fake account in every dashboard, which is why a rejected key produces noauth_fail.metadatacarriesfeature(text/language_detection/resource_management),toolName,transport, and the language tags ontranslate. Never the translated text, never a credential.Delivery
Buys a token from
/auth/issue-tokenwith the channel API key, posts to/metrics/ingest-events. Batched every 2s behind anunref'd timer, one retry on 401 only, requeued on 429/5xx, dropped on any other rejection, queue capped at 10 000, flushed once on shutdown with a 2s cap.A backend that is down never delays, alters or fails a translation.
Opt-out
DO_NOT_TRACKset to1/true/yesand nothing is measured, queued, written to disk or sent. Documented in the README.The installation id is a random UUID at
${LARA_HOME:-~/.lara}/installation-id, tied to nothing; it exists so repeated runs of the same install count as one. An unwritable home falls back to a per-process id rather than dropping telemetry.Before this can ship
DEFAULT_METRICS_URLandDEFAULT_METRICS_API_KEYinsrc/metrics.tsare empty, so telemetry is off for everyone as it stands. They need the deployed backend hostname and a production key issued for themcpchannel — the monitoring backend has no deployment yet, and only ships a development key.METRICS_URL/METRICS_API_KEYoverride them meanwhile.Note the key is compiled into a public npm package and will be readable by anyone who installs it. It is write-only, but worth a deliberate decision before filling it in.
Notes
Events are emitted inside
CallTool's own try/catch rather than from a wrapper around it:CallToolrewritesLaraApiErrorintoInvalidInputErrorbefore it escapes, so only there is the original error still available to classify. A wrapper would have seen every failure asvalidation_errorandauth_failwould never have fired.The cast through the SDK's
protected clientmoved intosrc/lara-client.ts, reused by the existingX-Lara-Clientheaders, so an SDK shape change surfaces in one place.Testing
60 new tests (226 total, all green): the client in isolation — queue, batching, token cooldown, 401 retry, installation id races, opt-out, payload never carrying credentials or text — and the server wiring end to end.
Verified against a local monitoring backend with real Lara credentials, both transports, reading the rows back out of ClickHouse:
install→auth_success→ threecall_successwith the right account,charsTranslated: 19on a translate whosetranslatable: falseblock was correctly excludedauth_successacross two stateless requests, thenauth_fail+call_erroron a revoked keyDO_NOT_TRACK=1, unconfigured defaults, malformedMETRICS_URL, unreachable backend → server behaves exactly as before,~/.laranever created