Merge pull request #1787 from kirill-markin/claude/agent-followups-p20b

feat(analytics): attribute AI chat reviews to the device that started the run

Author: kirill-markin

apps/admin/README.md

@@ -92,7 +92,7 @@ The old dashboard showed nothing for them, but that was not its replica join: th
 
 The email that the `%@example.com` exclusion needs is not in the events table, so it is joined from `actor_id` to `org.user_settings`. `actor_id` is a UUID and renders as canonical lowercase hex, while `org.user_settings.user_id` is an unconstrained `TEXT` primary key, so that join folds the stored side with `pg_catalog.lower`. Comparing as stored would silently miss an uppercase-hex row, and a test account with no matched email is counted rather than excluded.
 
-Platform is read off the event row and never derived. The buckets are `web`, `android`, `ios`, `agent`, and `unattributed`, and they are always split, never summed: an agent-API client merged into `web` would read as a person using the site. `agent` is an upper bound on human agent use rather than a count of people, because a scheduled or polling machine client files activity on a timer, and `db/migrations/0121_backfill_synthetic_app_opened_days.sql` states this in full. A `review_answered` row carries the platform the backend resolved from the replica that recorded the review, and migration `0122` filled the same value on the reconstructed history, `0123` on the live rows the producer wrote before it could resolve one, so both platform charts colour real device activity. A device value appears only for a `client_installation` replica on `ios`, `android`, or `web`, and a machine-API replica resolves to `agent`; an AI-chat and a seed/reset replica both leave the column NULL, as does a review whose replica row is gone. `unattributed` therefore means the row carries no resolved device fact - either the actor behind it is not a device, or no device could be resolved for it - and it stays its own bucket rather than being guessed at or summed into a device.
+Platform is read off the event row and never derived. The buckets are `web`, `android`, `ios`, `agent`, and `unattributed`, and they are always split, never summed: an agent-API client merged into `web` would read as a person using the site. `agent` is an upper bound on human agent use rather than a count of people, because a scheduled or polling machine client files activity on a timer, and `db/migrations/0121_backfill_synthetic_app_opened_days.sql` states this in full. A `review_answered` row carries the platform the backend resolved from the replica that recorded the review, and migration `0122` filled the same value on the reconstructed history, `0123` on the live rows the producer wrote before it could resolve one, so both platform charts colour real device activity. A device value appears for a `client_installation` replica on `ios`, `android`, or `web`, and for an AI-chat review whose chat run was started by a request that named its device; a machine-API replica resolves to `agent`, while any other AI-chat review and a seed/reset replica leave the column NULL, as does a review whose replica row is gone. `unattributed` therefore means the row carries no resolved device fact - either the actor behind it is not a device, or no device could be resolved for it - and it stays its own bucket rather than being guessed at or summed into a device.
 
 One thing to read the two platform charts with: a bulk review-history import ensures a replica from the importing request, and every imported review event stores that replica, so one import files its whole batch under the device that performed the import rather than under the device that originally answered. Those rows already land on the import day, because `occurred_at` falls back to the server anchor outside the 30-day window. A large import therefore shows as a single-platform spike on a single day, and that is the import showing through rather than a defect.
 

apps/admin/src/reports/reviewEventsByDate/query.ts

@@ -517,19 +517,22 @@ export function buildReviewEventsByDateAvailableRangeSql(): string {
 // Keeping the anonymized history is intended - the reviews really happened - and `identity_state` on
 // `analytics.product_events_resolved` is the handle if they ever need filtering out.
 //
-// PLATFORM IS READ OFF THE ROW AND NEVER DERIVED. The producer derives it once per drain from the
-// replica that recorded the review (`apps/backend/src/productAnalytics/serverFacts/reviewAnswers.ts`), and
-// migration `0122` filled the same value on the history `0120` reconstructed, `0123` on the live
-// rows the producer wrote before it could resolve one. That derivation reads
+// PLATFORM IS READ OFF THE ROW AND NEVER DERIVED. The producer derives it once per drain
+// (`apps/backend/src/productAnalytics/serverFacts/reviewAnswers.ts`), from the replica that recorded
+// the review for every review except an AI-chat one, and migrations `0122` and `0123` filled the
+// same replica-derived value on the history `0120` reconstructed and on the live rows the producer
+// wrote before it could resolve one. That derivation reads
 // `sync.workspace_replicas.platform` only together with `actor_kind` on the same row, so a value
 // read off that column appears only for a `client_installation` replica on 'ios', 'android' or
 // 'web': an `agent_connection` replica stores 'web' for the machine API, an `ai_chat` replica
 // stores a hard-coded 'web' that describes no device, and seed/reset replicas store 'system'. An
 // `agent_connection` replica resolves to `agent` from its actor kind instead of from that column.
-// An `ai_chat` or seed/reset replica, and a review whose replica row is gone or whose resolution
-// failed, stays NULL and lands in the `unattributed` bucket, which means no resolved device fact -
-// either the actor behind the row is not a device or no device could be resolved for it - rather
-// than either case alone.
+// An AI-chat review takes the device platform stored on its chat run instead, so a device value
+// also appears for an AI-chat review whose run was started by a request that named its device, and
+// any other AI-chat review stays NULL. A seed/reset replica, and a review whose replica row is gone
+// or whose resolution failed, stays NULL too. Every NULL lands in the `unattributed` bucket, which
+// means no resolved device fact - either the actor behind the row is not a device or no device
+// could be resolved for it - rather than either case alone.
 export function buildReviewEventsByDateSql(from: string, to: string): string {
   assertValidDateRange({ from, to }, "Review events report");
 

apps/backend/scripts/postgresIntegrations/boundaries.mjs

@@ -70,6 +70,14 @@ export const createdRolesByMigration = new Map([
   ["0044_reporting_readonly_role.sql", Object.freeze(["reporting_readonly"])],
 ]);
 export const boundaryDefinitions = Object.freeze([
+  Object.freeze({
+    migrationFileName: "0136_ai_chat_run_client_platform.sql",
+    expectedMigrationCount: 138,
+    testFiles: Object.freeze([
+      "src/chat/cardImages/operation.postgres.integration.ts",
+      "src/chat/runs/generatedImageAttemptBudget.postgres.integration.ts",
+    ]),
+  }),
   Object.freeze({
     migrationFileName: "0135_generated_media_promotion_job_created_at_select.sql",
     expectedMigrationCount: 137,
@@ -127,14 +135,6 @@ export const boundaryDefinitions = Object.freeze([
       "src/mediaAssets/blobLifecycle/cleanup/sharedProvenance.postgres.integration.ts",
     ]),
   }),
-  Object.freeze({
-    migrationFileName: "0127_ai_chat_run_live_attach_ownership.sql",
-    expectedMigrationCount: 129,
-    testFiles: Object.freeze([
-      "src/chat/cardImages/operation.postgres.integration.ts",
-      "src/chat/runs/generatedImageAttemptBudget.postgres.integration.ts",
-    ]),
-  }),
   Object.freeze({
     migrationFileName: "0123_backfill_live_review_answered_platform.sql",
     expectedMigrationCount: 125,

apps/backend/src/agent/reviews.ts

@@ -8,6 +8,7 @@ import {
   type DatabaseExecutor,
 } from "../database";
 import { getDeck } from "../decks";
+import type { ProductAnalyticsClientReportablePlatform } from "../productAnalytics/catalog";
 import { createPostCommitAnalyticsBudget } from "../productAnalytics/serverFacts/postCommitBudget";
 import { runTransactionReportingReviewAnswers } from "../productAnalytics/serverFacts/reviewAnswers";
 import type { FsrsCardState, ReviewRating } from "../scheduling";
@@ -357,11 +358,13 @@ export async function resolveAgentConnectionReviewReplica(
  * content.review_events: a repeated reviewId dedupes there and never advances the schedule, and
  * a reviewId the caller reused on another card is refused outright rather than read as that retry.
  * The replica half of that key comes from resolveReplicaId, so each surface dedupes within its own
- * sync actor. */
+ * sync actor. aiChatClientPlatform is the device the in-app chat run submitting this review was
+ * started from; the surfaces that write through an agent connection pass null. */
 export async function submitAgentReview(
   context: AgentReviewContext,
   request: AgentReviewInput,
   resolveReplicaId: AgentReviewReplicaResolver,
+  aiChatClientPlatform: ProductAnalyticsClientReportablePlatform | null,
 ): Promise<AgentReviewResult> {
   const input = parseReviewRequest(submitReviewSchema, request);
   if (
@@ -378,6 +381,7 @@ export async function submitAgentReview(
   const clientEventId = `agent-review:${input.reviewId}`;
   return runTransactionReportingReviewAnswers(
     createPostCommitAnalyticsBudget(),
+    aiChatClientPlatform,
     (runInTransaction) =>
       transactionWithWorkspaceScope(context, runInTransaction),
     async (executor) => {

apps/backend/src/cards/review/reviews.ts

@@ -272,6 +272,7 @@ export async function submitReview(
   // and microseconds in practice.
   return runTransactionReportingReviewAnswers<ReviewResult>(
     createPostCommitAnalyticsBudget(),
+    null,
     (runInTransaction) => transactionWithWorkspaceScope({ userId, workspaceId }, runInTransaction),
     async (executor) => {
       const hotChangeWriteLock = await lockWorkspaceSyncMetadataForHotChangesInExecutor(executor, workspaceId);

apps/backend/src/chat/cardImages/operation.postgres.integration.ts

@@ -103,7 +103,7 @@ async function createClaimedImageRun(fixture: PostgresIntegrationFixture): Promi
   const prepared = await prepareChatRun(
     fixture.userId, fixture.workspaceId, undefined,
     [{ type: "text", text: "Generate an image for this card." }],
-    randomUUID(), "Europe/Madrid", null, true,
+    randomUUID(), "Europe/Madrid", null, true, null,
   );
   const claimed = await claimChatRun(fixture.userId, fixture.workspaceId, prepared.runId);
   if (claimed === null) {

apps/backend/src/chat/http/handlers.ts

@@ -11,6 +11,7 @@ import {
   getBackendTraceCarrier,
   startBackendSpan,
 } from "../../observability/sentry";
+import { readProductAnalyticsClientPlatform } from "../../productAnalytics/catalog";
 import { parseOptionalWorkspaceIdParam } from "../../server/requestContext";
 import {
   parseJsonBody,
@@ -178,6 +179,7 @@ export function createPostChatHandler(dependencies: ChatRouteDependencies): Hand
         // version is greater than 1.5.0.
         body.uiLocale ?? null,
         requestContext.transport === "bearer" || requestContext.transport === "session",
+        readProductAnalyticsClientPlatform(context.get("clientPlatform") ?? null),
       ));
     } catch (error) {
       return mapStoreError(error);

apps/backend/src/chat/openai/loop/loop.testSupport.ts

@@ -32,6 +32,7 @@ export function createParams(
     sessionId: "session-1",
     generatedImageEligible: false,
     generatedImageOperationDeadlineMs: Date.now() + 600_000,
+    clientPlatform: null,
     modelId: "gpt-5.6-terra",
     reasoningEffort: "xhigh",
     timezone: "Europe/Madrid",

apps/backend/src/chat/openai/loop/loop.ts

@@ -26,6 +26,7 @@ import {
   type ChatRuntimeReasoningEffort,
 } from "../../config";
 import type { ChatRunClaimToken } from "../../runs";
+import type { ProductAnalyticsClientReportablePlatform } from "../../../productAnalytics/catalog";
 import {
   buildOpenAIResponsesRequest,
   buildPromptCacheKey,
@@ -87,6 +88,7 @@ export type StartOpenAILoopParams = Readonly<{
   sessionId: string;
   generatedImageEligible: boolean;
   generatedImageOperationDeadlineMs: number;
+  clientPlatform: ProductAnalyticsClientReportablePlatform | null;
   modelId: ChatRuntimeModelId;
   reasoningEffort: ChatRuntimeReasoningEffort;
   timezone: string;
@@ -111,6 +113,7 @@ async function runOneToolCall(
     workspaceId: string;
     signal: AbortSignal | null;
     generatedImageOperationDeadlineMs: number;
+    clientPlatform: ProductAnalyticsClientReportablePlatform | null;
     rootObservation: LangfuseObservation | null;
   }>,
 ): Promise<ExecutedChatToolCall> {
@@ -340,6 +343,7 @@ async function runLoopWithDeps(
       workspaceId: params.workspaceId,
       signal: params.signal,
       generatedImageOperationDeadlineMs: params.generatedImageOperationDeadlineMs,
+      clientPlatform: params.clientPlatform,
       rootObservation: params.rootObservation,
       onExecutionPhaseChanged: params.onExecutionPhaseChanged,
       shouldStopBeforeNextStep: params.shouldStopBeforeNextStep,

apps/backend/src/chat/openai/loop/modelCall.ts

@@ -16,6 +16,7 @@ import {
   type ChatRuntimeReasoningEffort,
 } from "../../config";
 import type { ChatRunClaimToken } from "../../runs";
+import type { ProductAnalyticsClientReportablePlatform } from "../../../productAnalytics/catalog";
 import { createGeneratedImageOperationKey } from "../../generatedImageOperationIdentity";
 import { GENERATED_IMAGE_TOOL_NAME } from "../tools/generatedImageToolContract";
 import {
@@ -77,6 +78,7 @@ export type RunOneToolCall = (params: Readonly<{
   workspaceId: string;
   signal: AbortSignal | null;
   generatedImageOperationDeadlineMs: number;
+  clientPlatform: ProductAnalyticsClientReportablePlatform | null;
   rootObservation: LangfuseObservation | null;
 }>) => Promise<ExecutedChatToolCall>;
 
@@ -93,6 +95,7 @@ type ExecuteToolCallsParams = Readonly<{
   workspaceId: string;
   signal: AbortSignal | undefined;
   generatedImageOperationDeadlineMs: number;
+  clientPlatform: ProductAnalyticsClientReportablePlatform | null;
   rootObservation: LangfuseObservation | null;
   onExecutionPhaseChanged: ((phase: "idle" | "model" | "tool") => void) | undefined;
   shouldStopBeforeNextStep: (() => boolean) | undefined;
@@ -282,6 +285,7 @@ export async function executeToolCalls(
         workspaceId: params.workspaceId,
         signal: params.signal ?? null,
         generatedImageOperationDeadlineMs: params.generatedImageOperationDeadlineMs,
+        clientPlatform: params.clientPlatform,
         rootObservation: params.rootObservation,
       });
       if (output.stopReason !== null) {