test(observability): validate Flow Activity and Trace Details via API + UI

[daniellicnerski1]

Summary

• Replace the failing /monitor/transactions assertion (HTTP 422 — the endpoint requires flow_id, which the previous test omitted) with three new tests in a shared describe block that creates a real flow once and validates traces both through the API and the UI.
• Delete the pagination test for /monitor/transactions: the endpoint has no React consumer in the Langflow frontend (verified via grep useGetTransactionsQuery — defined but never imported), so it would be a regression test for a code path no user can reach.
• New fixture tests/assets/flows/basic-prompting-trace-fixture.json (a Basic Prompting flow snapshot) is loaded in beforeAll. The run intentionally fails with "A model selection is required" (no provider configured) — that failure still emits the trace, so the tests work without OPENAI_API_KEY and consume 0 tokens.

What each test validates

• API — GET /api/v1/monitor/traces?flow_id=<id> returns totalLatencyMs (number, ≥ 0), totalTokens (number, ≥ 0), the correct flowId, a valid status (success / error / running), and an ISO startTime.
• UI — Flow Activity table — column totalLatencyMs cell text matches /^\d+\s*ms$/, column totalTokens cell text matches /^\d+$/.
• UI — Trace Details modal — clicking the Run cell opens trace-detail-view; the modal contains span-tree + span-detail; exactly 4 span-node-* (root + Prompt Template + Chat Input + Language Model); span-detail shows the Latency label with <N> ms; the span tree lists all three component names.

Validation pipeline

• npm run typecheck ✅
• npx eslint ✅ (only pre-existing warnings on tests 4–5 of the file that this PR did not touch)
• 3 consecutive runs with --retries=0 --trace=on — 3/3 green each (~10.6s / 11.0s / 12.5s)
• Force-fail confirmed: swapping .toHaveCount(4) for 999 and flowId for a sentinel both produce real failures
• 🚨 Backend Error audit — 0 occurrences

Out of scope

The two remaining tests in the file (/monitor/messages shape, /logs page accessibility) are untouched. They have soft-pass patterns (if (empty) return;) that should be revisited but are not the focus of this PR.

Test plan

• PR review confirms the fixture is appropriate (Basic Prompting snapshot from the current Langflow)
• CI passes typecheck + lint
• Manual run on a clean Langflow instance shows the 3 tests green

[Copilot]

Pull request overview

This PR updates the observability monitoring regression coverage by replacing the previous /monitor/transactions assertions with a serial, end-to-end flow-based suite that generates a real trace once and validates latency/tokens via both API and UI.

Changes:

• Added a serial describe block that creates an API key + imports a fixture flow, runs it once (intentionally failing to avoid provider setup), and validates /api/v1/monitor/traces.
• Added UI validations for Flow Activity (latency/tokens columns) and Trace Details (span tree + latency labels).
• Added a new flow snapshot fixture (basic-prompting-trace-fixture.json) used to deterministically generate trace data without consuming tokens.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
tests/tests-automations/regression/core-functionality/observability-monitoring/traces-latency-tokens.spec.ts	Replaces transactions-based checks with a serial API+UI trace validation suite using a real flow run.
tests/assets/flows/basic-prompting-trace-fixture.json	Adds a Basic Prompting flow snapshot used by the new trace tests to generate trace entries without provider configuration.

Comments suppressed due to low confidence (1)

tests/tests-automations/regression/core-functionality/observability-monitoring/traces-latency-tokens.spec.ts:136

• Same as above for tokens: toHaveText() here relies on default timeouts and can flake if the grid cell appears before values are filled. Add an explicit timeout (or wait for the grid’s data row readiness) to make the test robust on CI.

      const tokensCell = page
        .locator('.ag-cell[col-id="totalTokens"]')
        .first();
      await expect(tokensCell).toBeVisible();
      await expect(tokensCell).toHaveText(/^\d+$/);

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[daniellicnerski1]

Fixed in a8255bb. Captured the response and assert expect([200, 500]).toContain(runRes.status()) — Langflow returns 200 with an error payload for component-level failures, anything outside that range (401/422/etc.) now fails fast. Also added expect.poll against /monitor/traces (30s timeout, 500/1000/2000ms intervals) in beforeAll so downstream tests no longer race the async trace writer.

[daniellicnerski1]

Fixed in a8255bb. Added explicit { timeout: 15000 } to both toHaveText() calls (latency cell + tokens cell), matching the toBeVisible timeout above. The default 5s expect timeout was shorter than the visibility wait, which is exactly the race condition you flagged.

[daniellicnerski1]

Copilot review feedback — addressed

#	Issue	Status
1	Flow run response ignored (race with async trace writer)	✅ Fixed
2	toHaveText() default timeout shorter than visibility wait	✅ Fixed

Changes (a8255bb):

• beforeAll now captures the POST /api/v1/run/${flowId} response and asserts expect([200, 500]).toContain(status) — failures outside that range (401, 422, etc.) fail fast.
• Added expect.poll against /api/v1/monitor/traces?flow_id=${flowId} (30s timeout, 500/1000/2000ms intervals) so downstream tests no longer race the async trace writer.
• Added explicit { timeout: 15000 } to both toHaveText() calls on the Flow Activity grid, matching the toBeVisible timeout above.

Validation pipeline run locally (against live Langflow):

1. typecheck            ✅ PASS
2. lint (filtered)      ✅ PASS (0 errors)
3. static checklist     ✅ PASS
4. run --retries=0      ✅ 5/5 in 10.0s
5. force-fail           ✅ both new assertions caught regressions
6. --trace=on           ✅ 5/5 in 8.6s
7. backend errors       ✅ zero occurrences