Skip to content

Commit 534020a

Browse files
UDtorreyclaude
andauthored
feat(registry-dash): persist AI tool-call entries in chat scrollback (SRE-1963) (#141)
* feat(registry-dash): persist AI tool-call entries in chat scrollback (SRE-1963) Tool-call indicators previously vanished the moment a tool returned, leaving the user with no record of which tools the assistant called or how each one resolved. Make each tool call a persistent pill in the chat timeline so the chat surface mirrors how Claude Code's CLI renders tool calls. Frontend-only — the SSE stream from RegistryDashAiAction already carries tool name, args, structured status (SRE-1958), and diagnostic. No backend change required. Data model - New ToolMessage variant + ChatTimelineEntry union in ai-analysis.models.ts. ConversationMessage stays the wire shape; toWireHistory() strips tool entries before sending follow-up turns to the backend. - Drop now-unused ToolInFlight / ToolCompleted types. Service lifecycle (ai-analysis.service.ts) - conversationHistory becomes the single source of truth for the timeline, typed ChatTimelineEntry[]. - On tool_use: flush any pending streamed text to an assistant entry first, then push an IN_FLIGHT tool entry. Preserves chronological order across text -> tool -> text streams. - On tool_result: update the latest matching IN_FLIGHT entry in place. - analyze() smart-merges incoming wire history (prefix-match append, falling back to wholesale replace) so prior turns' tool entries survive across follow-ups instead of being clobbered by the next request body. - On cancel/abort/network drop: terminateInFlight() flips any leftover IN_FLIGHT entries to INTERNAL_ERROR with diagnostic "Cancelled" so pills stop pulsing. - Drop toolsInFlight / toolsCompleted / toolsUsed signals — derivable from conversationHistory and no external consumers. Modal (ai-analysis-modal.component.{ts,html,scss}) - New tool branch in the persistent chat loop renders a single .tool-pill. Variants: --in-flight (italic + pulsing dots), --ok (presence == success, no chip suffix), --warn / --error (chip suffix carries the SRE-1958 status text + tooltip with the diagnostic). - Streaming block trimmed to streamed text + progress bar; tool rendering moves into the persistent timeline. - chipFor() updated for ToolMessage; new ariaLabelFor() composes the screen-reader description. - runQueuedPrompt() filters via toWireHistory() before constructing the request body so tool entries never round-trip to the backend. Tests - ai-analysis.service.spec.ts: rewrites the two existing tool tests to assert on conversationHistory; adds 4 new SRE-1963 cases (multi-tool ordering, text-tool-text interleave, IN_FLIGHT-resolved-on-cancel, cross-turn preservation + wire-shape stripping). - ai-analysis-modal.component.spec.ts: drops toolsInFlight from the stub (no longer on the service); adds a persistent tool pills describe block covering IN_FLIGHT/OK/non-OK rendering, multi-pill ordering, and the chipFor matrix. - All 115 ai-analysis specs pass. Pre-existing ExploreComponent / RevenueBilling failures are unrelated (verified via stash). Test plan (.claude/plugins/.../test-plan.md) - Test 18 rewritten for persistence (pill stays visible through and past the response). - Test 65 updated: OK = pill present, no chip suffix; non-OK = pill + chip suffix. - New Test 68 covers multi-tool persistence, cross-turn survival, wire- payload stripping, and cancel cleanup. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(registry-dash): FIFO match + accurate diagnostic for AI tool pills Address two correctness issues in ai-analysis.service.ts surfaced during review of #141 (SRE-1963): 1. updateLatestInFlight() iterated LIFO, so when the same tool was invoked twice in a single turn the first tool_result (which arrives first per the backend's FIFO ordering) would land on the second pill, swapping diagnostics. Iterate from the start instead so oldest IN_FLIGHT pill gets the first matching result. 2. terminateInFlight() unconditionally tagged leftover IN_FLIGHT entries with diagnostic "Cancelled". This misrepresents network drops / 5xx / unexpected stream ends as user-cancelled. Pass the diagnostic in from the caller — "Cancelled" only when controller.signal.aborted, else "Interrupted". New tests: - same-tool-twice → results match pills FIFO - stream-end-without-result on a non-aborted controller → diagnostic "Interrupted", not "Cancelled" 117/117 ai-analysis specs pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
1 parent 96a71b6 commit 534020a

8 files changed

Lines changed: 638 additions & 166 deletions

File tree

.claude/plugins/ud-registry-dash/skills/test-registry-dash/test-plan.md

Lines changed: 52 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -418,24 +418,29 @@
418418

419419
---
420420

421-
## Test 18: Tier 3 Tool Use — Indicator UX
421+
## Test 18: Tier 3 Tool Use — Persistent Tool Pill (SRE-1963)
422422

423-
**Goal:** Verify the analysis modal shows transient tool indicators when Claude calls a tool.
423+
**Goal:** Verify the analysis modal shows a tool pill when Claude calls a tool, and the pill **stays visible in the chat scrollback after execution** so the user can see what tools ran.
424424
**Tier:** full
425425

426426
### Steps:
427427
1. Navigate to **Domain Activity** (or any page with sparkle).
428428
2. Click sparkle → open the analysis modal.
429429
3. Wait for the initial analysis to finish.
430430
4. In the follow-up box, type: `What specific domains transferred in the last 30 days for the example tld?` (substitute a TLD with seeded transfer data).
431-
5. Watch for an inline indicator below the streaming text: `🔍 Searching transfers…` (italic, slight pulse animation).
432-
6. The indicator should appear when the tool is in flight and disappear once the tool result arrives.
433-
7. The final assistant text should reference specific domain names that came from the tool.
431+
5. Watch for an inline tool pill: `🔍 Searching transfers…` (italic with pulsing dots while in flight).
432+
6. When the tool completes, the pill **must remain visible** — the dots and italic styling go away, but the pill itself stays in the timeline at the chronological position the tool fired.
433+
7. The final assistant text streams in **below** the pill (not above, not replacing it).
434+
8. After the response finishes, scroll up — the pill is still there.
435+
9. Send a follow-up question that triggers a different tool (e.g. `Look up registrar TheRegistrar`). The new pill appears below the prior turn; the old pill from turn 1 is still visible.
434436

435437
### Expected:
436-
- Indicator appears mid-stream and is replaced by the next text chunk.
437-
- Final answer contains specific domain names, not just chart-level summaries.
438-
- Modal does not crash or hang.
438+
- A tool pill appears for every `tool_use` event and persists through and past the response.
439+
- IN_FLIGHT pill: italic, pulsing `` indicator.
440+
- OK terminal: solid pill, no chip suffix (the pill's mere presence confirms success).
441+
- Non-OK terminal: solid pill with a colored chip suffix (see Test 65 for status-specific coverage).
442+
- Final answer contains specific domain names from the tool result.
443+
- Modal does not crash or hang; no pills disappear or become orphaned.
439444

440445
---
441446

@@ -1429,25 +1434,26 @@ Steps: open modal on Financials > Forecasting, ask "How many domains in tld exam
14291434

14301435
---
14311436

1432-
## Test 65: Tier 3 Tool Use - Result-status chips (SRE-1958)
1437+
## Test 65: Tier 3 Tool Use - Result-status chips (SRE-1958, updated for SRE-1963)
14331438

1434-
**Goal:** Verify the modal renders disambiguated status chips next to tool-call indicators when a tool returns a non-OK status, with the diagnostic visible on hover.
1439+
**Goal:** Verify the persistent tool pill (SRE-1963) carries a disambiguated status-chip suffix when a tool returns a non-OK status, with the diagnostic visible on hover.
14351440

14361441
### Steps:
14371442
1. Navigate to **Domain Activity** and open the analysis modal (any prompt).
14381443
2. Wait for the initial analysis to finish.
14391444
3. Trigger an `EMPTY_FOR_RANGE` case: ask "Show transfers for tld example between 2025-01-01 and 2025-01-02." (adjust dates to a known-empty window in the target env).
1440-
4. Watch the tool indicator - when it resolves, a yellow/muted chip should appear with text "No data". Hover the chip; the tooltip must contain a diagnostic naming the active filter and data extent.
1441-
5. Trigger an `INVALID_ARGS` case: via DevTools issue an analyze request with `tld=""` or omit a required arg. The chip should be red, text "Invalid args", tooltip naming the offending arg.
1442-
6. Trigger a `PERMISSION_DENIED` case: while logged in as a non-FTE user with limited scope, ask about a TLD outside the scope. The chip should be red, text "No access".
1443-
1444-
### Expected:
1445-
- For `OK`: no chip, current silent green-checkmark behavior preserved.
1446-
- For `EMPTY_FOR_RANGE`: yellow/muted chip "No data" + diagnostic in `matTooltip`.
1447-
- For `OUT_OF_RANGE`: yellow chip "Out of range" + diagnostic.
1448-
- For `INVALID_ARGS`: red chip "Invalid args" + diagnostic.
1449-
- For `PERMISSION_DENIED`: red chip "No access" + diagnostic.
1450-
- For `INTERNAL_ERROR`: red chip "Tool error" + sanitized diagnostic.
1445+
4. When the tool resolves, the pill stays in the scrollback with a yellow/muted **chip suffix** "No data". Hover the pill; the tooltip must contain a diagnostic naming the active filter and data extent.
1446+
5. Trigger an `INVALID_ARGS` case: via DevTools issue an analyze request with `tld=""` or omit a required arg. The chip suffix should be red, text "Invalid args", tooltip naming the offending arg.
1447+
6. Trigger a `PERMISSION_DENIED` case: while logged in as a non-FTE user with limited scope, ask about a TLD outside the scope. The chip suffix should be red, text "No access".
1448+
1449+
### Expected:
1450+
- For `OK`: pill stays in the scrollback with **no chip suffix** — its presence alone confirms the tool ran (SRE-1963).
1451+
- For `EMPTY_FOR_RANGE`: pill with yellow/muted "No data" chip suffix + diagnostic in `matTooltip`.
1452+
- For `OUT_OF_RANGE`: pill with yellow "Out of range" chip suffix + diagnostic.
1453+
- For `INVALID_ARGS`: pill with red "Invalid args" chip suffix + diagnostic.
1454+
- For `PERMISSION_DENIED`: pill with red "No access" chip suffix + diagnostic.
1455+
- For `INTERNAL_ERROR`: pill with red "Tool error" chip suffix + sanitized diagnostic.
1456+
- All pills persist in scrollback after the response completes.
14511457
- The streamed `tool_result` frame in the network tab carries `{type: "tool_result", tool, status, diagnostic?, ok}`.
14521458

14531459
---
@@ -1591,3 +1597,28 @@ Steps: open modal on Financials > Forecasting, ask "How many domains in tld exam
15911597
### Expected:
15921598
- The `description` text contains both `dates=required` (for sources like REVENUE, DOMAIN_ACTIVITY, RENEWAL_RATES, EXPIRATION_CURVE, TRANSACTIONS) and `dates=n/a` (for DOMAIN_COUNTS and PRICING_RULES).
15931599
- The `input_schema.required` array does NOT contain `start_date` or `end_date`.
1600+
1601+
---
1602+
1603+
## Test 73: Tier 3 Tool Use - Multi-tool persistence + cross-turn survival (SRE-1963)
1604+
1605+
**Goal:** Confirm every tool call gets its own persistent pill in chronological order, pills survive across follow-up turns, and tool entries are NOT replayed back to the backend on subsequent turns.
1606+
1607+
### Steps:
1608+
1. Navigate to **Domain Activity** and open the analysis modal.
1609+
2. Wait for the initial analysis to finish.
1610+
3. Ask a multi-tool prompt: `Do a quick test of each tool — check transfers, pricing rules, and registrar activity for tld example.`
1611+
4. Watch the timeline: a separate pill should appear for **each** tool that fires, in the order it fires.
1612+
5. After the response completes, scroll up — every pill is still there.
1613+
6. Send a follow-up: `Now look up registrar TheRegistrar.` Watch the new pill appear in the new turn.
1614+
7. Open DevTools → Network → click the second `/console-api/registry-dash/ai/analyze` request → Payload tab.
1615+
8. Inspect the `conversationHistory` array in the request body.
1616+
9. (Optional) Click Stop mid-response on a long multi-tool prompt. Verify any IN_FLIGHT pill resolves to a terminal state (no spinner stuck pulsing forever).
1617+
1618+
### Expected:
1619+
- Step 3-4: every `tool_use` event produces a distinct pill; multiple sequential pills render in chronological order interleaved correctly with assistant text chunks.
1620+
- Step 5: prior turn's pills are still visible after the response completes.
1621+
- Step 6: the new turn's pill appears below the prior turn's pills; the prior turn's pills survive.
1622+
- Step 8: `conversationHistory` in the wire payload contains ONLY `{role: "user"|"assistant", content: ...}` entries — **no `role: "tool"` entries** are sent back to the backend.
1623+
- Step 9: cancelled IN_FLIGHT pills resolve to a non-spinning terminal state (e.g., red "Tool error" chip with diagnostic "Cancelled" or "Interrupted" depending on whether the abort was user-initiated).
1624+
- No regression on auto-scroll, queue, resize (PR #134) or stale-state-on-reopen (PR #127).

console-webapp/src/app/registry-dash/ai/ai-analysis-modal.component.html

Lines changed: 25 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -51,29 +51,37 @@ <h2 mat-dialog-title>{{ data.title }}</h2>
5151
<mat-icon class="message-icon">auto_awesome</mat-icon>
5252
<div class="message-content markdown-body" [innerHTML]="msg.content | markdown"></div>
5353
</div>
54+
<!-- Persistent tool pill (SRE-1963). Shows pulsing dots while
55+
IN_FLIGHT, a status chip suffix on non-OK terminal statuses,
56+
and just the label for OK — its mere presence confirms the
57+
tool ran. Diagnostic, when present, is the matTooltip. -->
58+
<div *ngIf="msg.role === 'tool'"
59+
class="tool-pill"
60+
[class.tool-pill--in-flight]="msg.status === 'IN_FLIGHT'"
61+
[class.tool-pill--ok]="msg.status === 'OK'"
62+
[class.tool-pill--warn]="chipFor(msg)?.tone === 'warn'"
63+
[class.tool-pill--error]="chipFor(msg)?.tone === 'error'"
64+
[matTooltip]="msg.diagnostic || ''"
65+
[attr.data-tool]="msg.tool"
66+
[attr.data-status]="msg.status"
67+
tabindex="0"
68+
role="status"
69+
[attr.aria-label]="ariaLabelFor(msg)">
70+
<span class="tool-pill-label">{{ msg.label }}</span>
71+
<span *ngIf="msg.status === 'IN_FLIGHT'" class="tool-pill-dots"></span>
72+
<span *ngIf="chipFor(msg) as chip" class="tool-pill-chip"
73+
[class.tool-pill-chip--warn]="chip.tone === 'warn'"
74+
[class.tool-pill-chip--error]="chip.tone === 'error'">{{ chip.text }}</span>
75+
</div>
5476
</ng-container>
5577

56-
<!-- Streaming response -->
78+
<!-- Streaming response. Tool pills now render in the persistent
79+
timeline above (SRE-1963), so this block carries only the
80+
streamed text and progress bar. -->
5781
<div *ngIf="streaming()" class="message assistant-message streaming">
5882
<mat-icon class="message-icon">auto_awesome</mat-icon>
5983
<div class="message-content markdown-body">
6084
<div [innerHTML]="streamedText() | markdown"></div>
61-
<ng-container *ngFor="let t of toolsCompleted()">
62-
<div *ngIf="chipFor(t) as chip" class="tool-status-chip"
63-
[class.tool-status-chip--warn]="chip.tone === 'warn'"
64-
[class.tool-status-chip--error]="chip.tone === 'error'"
65-
[matTooltip]="t.diagnostic || ''"
66-
[attr.data-tool]="t.tool"
67-
[attr.data-status]="t.status"
68-
tabindex="0"
69-
role="status"
70-
[attr.aria-label]="t.label + ' — ' + chip.text + (t.diagnostic ? ': ' + t.diagnostic : '')">
71-
{{ t.label }} — {{ chip.text }}
72-
</div>
73-
</ng-container>
74-
<div *ngFor="let t of toolsInFlight()" class="tool-indicator">
75-
{{ t.label }}<span class="tool-indicator-dots"></span>
76-
</div>
7785
<mat-progress-bar mode="indeterminate" class="stream-progress"></mat-progress-bar>
7886
</div>
7987
</div>

console-webapp/src/app/registry-dash/ai/ai-analysis-modal.component.scss

Lines changed: 33 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -249,27 +249,52 @@ mat-dialog-actions {
249249
}
250250
}
251251

252-
.tool-indicator {
252+
// Persistent tool pill (SRE-1963). Renders inline in the chat scrollback
253+
// for every tool that ran. Stays visible after completion so the user
254+
// can see what was called. State variants:
255+
// - --in-flight: italic, pulsing dots while running
256+
// - --ok: solid muted pill (presence == success)
257+
// - --warn / --error: chip suffix carries status text + tooltip
258+
.tool-pill {
259+
display: inline-flex;
260+
align-items: center;
261+
gap: 6px;
253262
margin: 8px 0;
254263
padding: 6px 10px;
255264
background: var(--ud-surface-2, #f5f5f5);
256265
border-left: 3px solid var(--ud-accent);
257266
border-radius: 4px;
258267
font-size: 13px;
259268
color: var(--ud-text-muted, #666);
260-
font-style: italic;
269+
cursor: help;
270+
271+
&--in-flight {
272+
font-style: italic;
273+
}
274+
275+
&--warn {
276+
border-left-color: var(--ud-warn-border, #ffd54f);
277+
}
278+
279+
&--error {
280+
border-left-color: var(--ud-error-border, #f5b3ad);
281+
}
261282
}
262283

263-
.tool-status-chip {
284+
.tool-pill-dots {
285+
display: inline-block;
286+
animation: tool-pulse 1.2s ease-in-out infinite;
287+
}
288+
289+
.tool-pill-chip {
264290
display: inline-flex;
265291
align-items: center;
266-
margin: 4px 8px 4px 0;
267-
padding: 4px 10px;
268-
border-radius: 12px;
269-
font-size: 12px;
292+
padding: 2px 8px;
293+
border-radius: 10px;
294+
font-size: 11px;
270295
line-height: 1.4;
271296
font-weight: 500;
272-
cursor: help;
297+
font-style: normal;
273298

274299
&--warn {
275300
background: var(--ud-warn-bg, #fff8e1);
@@ -284,12 +309,6 @@ mat-dialog-actions {
284309
}
285310
}
286311

287-
.tool-indicator-dots {
288-
display: inline-block;
289-
margin-left: 2px;
290-
animation: tool-pulse 1.2s ease-in-out infinite;
291-
}
292-
293312
@keyframes tool-pulse {
294313
0%, 100% { opacity: 0.3; }
295314
50% { opacity: 1; }

0 commit comments

Comments
 (0)