[Daily Failure] @stable tests failed on 2026-08-06 (langflowai/langflow-nightly:latest)

[github-actions]

Daily @stable E2E Failure

• Date: 2026-08-06
• Langflow version: langflowai/langflow-nightly:latest
• Run: 31093877484

🔌 Backend liveness (in-run measurement)

No mid-run outage measured (no run of ≥2 consecutive failed probes) on 4 of 4 measured shard(s).
⚠️ But 19 single-probe failure(s) were discarded as blips — the backend did not answer
everything. Treat the wedge as unruled-out and read the liveness-* artifacts (#1030).

@stable auto-removal

🔻 Auto-removed @stable from 3 hard-failing test(s):

• tests/tests-automations/regression/core-functionality/model-provider/openai-provider.spec.ts — configured OpenAI selects a GPT model in the Agent and executes the flow
• tests/tests-automations/regression/mcp/server/mcp-server.spec.ts — Streamable HTTP MCP server with server-everything should load tools correctly
• tests/tests-automations/regression/core-functionality/model-provider/openai-compatible-provider-setup.spec.ts — a discovered model runs a flow through the OpenAI Compatible provider

These were committed to main automatically. Restoring @stable is manual: once the test or Langflow is fixed, re-add the tag via PR.

/cc @Victor-w-Madeira @daniellicnerski1 @rafaelgiln

[claude]

Triage proposal — run 31093877484 (2026-08-06, langflowai/langflow-nightly:latest)

Panorama: 3 hard failures / 2 actionable flakes (of 4 total) / 9 skips — 444 passed. Guard tripped: no. All 3 hard failures already had @stable auto-removed by the workflow (commit cb3082d, verified on main), so no manual hard-failure quarantine is needed this run.

Dedup checked against all 14 open daily-failure issues (gh issue list --label daily-failure --state open) plus keyword search (openai-provider, openai-compatible, mcp-server-tab, auto-save-off, server-everything) — no existing issue matches any cluster below; all 5 actionable rows are create. #1266 was checked specifically against the mcp-server.spec.ts hard failure (row 1) and ruled a different test/signature (see notes).

Read from results.json's call logs, not just the normalized error_signature (same method the last executed triage on this repo used, per #1258's closing comment) — this splits the dataset's provider_wide_clusters hint for OpenAI into two independent issues (rows 2 and 3) instead of one merged cluster: the call logs show two unrelated failure shapes sharing only the provider name.

#	Cluster (symptom/area)	Kind	Action	Target
1	mcp-server.spec.ts:1128 ("Streamable HTTP MCP server with server-everything should load tools correctly") — add-mcp-server-button never becomes visible (15s timeout), identical on all 3 retries. Recurrent 2× same-signature (2026-08-05, 2026-08-06)	hard-failure	create	[Daily #1330] mcp-server — "Add MCP Server" button never appears for the server-everything Streamable-HTTP case (recurrent 2×)
2	openai-provider.spec.ts:201 ("configured OpenAI selects a GPT model in the Agent and executes the flow") — retry 0 times out on the Stop button never hiding (120s); retries 1–2 time out on div-chat-message never appearing at all — no AI response ever lands. Recurrent 3× same-signature (2026-07-14, 2026-07-15, 2026-08-06)	hard-failure	create	[Daily #1330] openai-provider — the Agent's flow run never produces a response (Stop button hangs, then no chat message) (recurrent 3×)
3	openai-compatible-provider-setup.spec.ts:620 ("a discovered model runs a flow through the OpenAI Compatible provider") — the persisted node's credential field reads back "" instead of "OPENAI_COMPATIBLE_API_KEY" on both failing retries. Recurrent 2× same-signature (2026-08-05, 2026-08-06)	hard-failure	create	[Daily #1330] openai-compatible-provider-setup — persisted node's credential reference reverts to an empty string (recurrent 2×)
4	mcp-server-tab.spec.ts:6 ("user should be able to manage MCP server tools and configuration") — mcp-server-dropdown isn't clickable within 3s (helpers/mcp/open-add-mcp-server-modal.ts:10). Recurrent 2× same-signature (2026-08-05, 2026-08-06; a 3rd hit on 2026-07-27 carries a different signature)	flake (actionable)	create	[Daily #1330] mcp-server-tab — the MCP-server dropdown is intermittently not clickable within 3s (recurrent 2×)
5	auto-save-off.spec.ts:59 ("user should be able to manually save a flow when the auto_save is off") — reopening the just-created flow's list-card-open-button times out at 45s. Recurrent 2× same-signature (2026-07-22, 2026-08-06)	flake (actionable)	create	[Daily #1330] auto-save-off — reopening the just-created flow's card intermittently times out at 45s (recurrent 2×)

Why rows 2 and 3 are not one "OpenAI provider-wide" issue, even though the dataset flagged them as a provider_wide_cluster: row 2's own retries show a "no response ever lands" shape (Stop button / chat message never resolve — consistent with the run never completing), while row 3's failure is a credential string going from a named variable to "" on a persisted node — a persistence/UI bug with no connection to whether the account has credits. Splitting keeps each issue's assignee looking at the right code path.

Lead, not a verdict, for row 2: two independent automated issues opened today — #1329 and #1328 (migration-test, not daily-failure) — record the OpenAI account as credit-exhausted (insufficient_quota / credit_balance_exhausted) as of 2026-08-06 09:34 UTC, and 8 of this run's 9 skips carry the identical reason (see below). That is a plausible explanation for today's occurrence of row 2. It does not explain the whole cluster: row 2 recurred with the same signature on 2026-07-14 and 2026-07-15 too, weeks before today's billing event, so the issue is filed on the recurrence itself and should record the billing correlation as a lead for today only, not a root cause for all three dates.

Skips — noted, not filed (already tracked / intentional):

• 8 of 9 skips carry the reason "Provider 'openai' inactive — 10 of 41 candidate model(s) failed validation... last error: You have no credits remaining." — same root cause as Migration test blocked: OpenAI account has no credits (no migration signal) #1329/Migration test blocked: OpenAI account has no credits (no migration signal) #1328 above (confirmed same day, same account). Not filed as a new daily-failure issue: the cause is already documented there, and the general fix (key-level fallback so one drained key doesn't blank out a whole day's OpenAI coverage) is already tracked at Provider API-key fallback: resolve a live key before the suite, fail loud when every candidate is dead #976 (open, qa-infra/follow-up).
• "selecting a different-provider model swaps the canvas provider icon" (agent-component-regression.spec.ts) — reason: "Test 4 requires both OpenAI and Anthropic to be pre-configured in the local Langflow instance." Intentional, same pattern noted on prior triages (e.g. [Daily Failure] @stable tests failed on 2026-08-04 (langflowai/langflow-nightly:latest) #1258) — the daily's single-active-provider rotation doesn't satisfy it by design.

Non-actionable flakes — noted, not filed (single-occurrence or mixed signature, retry absorbed it):

• agent-component-regression.spec.ts:75 ("renders on canvas with default fields and handles") — title-Agent not visible within 10s on attempt 1, passed on retry. Only same-signature hit this window (a 2026-07-20 occurrence carries a different signature).
• openai-compatible-provider-setup.spec.ts:558 ("the configured provider discovers exactly the models its endpoint serves") — live model list came back with 1 model instead of the expected 28, passed on both retries. First occurrence in the window. Corroborates rows 2/3's read that OpenAI/OpenAI-Compatible connectivity was unsettled today, but not actionable on its own.

Quarantines this triage requires (manual/local TODOs — NOT performed by this automation)

Spec:line	Reason
tests/tests-automations/regression/mcp/server/mcp-server-tab.spec.ts:6	flake, recurrent 2× same-signature (2026-08-05, 2026-08-06) — row 4
tests/tests-automations/regression/flow-functionality/auto-save-off.spec.ts:59	flake, recurrent 2× same-signature (2026-07-22, 2026-08-06) — row 5

Each requires removing @stable and wrapping the test with test.fixme(<reason + issue #>), together, in its own PR — see CONTRIBUTING.md → Quarantine mechanism. No hard failure needs a manual quarantine this run: the workflow's automatic removal already covered all 3.

---

A triage-team member: reply pode abrir on this issue to create/enrich the dedicated issues. Quarantines listed above are manual/local TODOs — they are NOT performed by the automation.

[Victor-w-Madeira]

Triage executed — 5 dedicated issues created

Worked from the <!-- triage-proposal --> plan above, after re-deriving the dataset locally and reading the run's results.json call logs (not the normalized error_signature alone).

#	Issue	Cluster	Kind
1	#1332	mcp-server.spec.ts:1128 — add-mcp-server-button never renders (element(s) not found, deterministic across all 3 attempts)	hard failure
2	#1333	openai-provider.spec.ts:201 — the Agent's run never produces a response	hard failure
3	#1334	openai-compatible-provider-setup.spec.ts:620 — persisted node's credential reads back ""	hard failure
4	#1335	mcp-server-tab.spec.ts:6 — mcp-server-dropdown not clickable within 3 s	recurrent flake
5	#1336	auto-save-off.spec.ts:59 — reopening the just-created flow's card times out at 45 s	recurrent flake

@stable was already auto-removed from all 3 hard failures by the workflow (commit cb3082d, verified on main).

Three corrections to the proposal above

1. Skips: 6 carry the "no credits" reason, not 8. The other 3 are intentional/by-design: agent-component-regression.spec.ts (needs two providers configured), memory-history-regression.spec.ts:346 (the template's model is pinned via the flows API, openai-only by design), model-provider-model-toggle.spec.ts:195 (no non-selected dropdown option belongs to the test's provider).
2. The openai-compatible-provider-setup.spec.ts:558 flake received an EMPTY list, not "1 model" — attempt 0 read Array [] against 28 expected ids. The + Received + 1 in the diff header is a line count, not a model count.
3. Row 3 is reframed, and it is the substantive change. The proposal read the empty credential as a persistence/UI defect. A named lead with matching dates says otherwise and is recorded on [Daily #1330] openai-compatible — the persisted node's credential reads back empty; the assertion may be the last inline copy of the premise #14311 removed (recurrent 2×) #1334: upstream 646bdd6b (langflow-ai/langflow#14311, "stop automatic provider field binding") deliberately removed the pre-population of a provider field's value with the credential variable name; it was forward-ported into release-1.12.0 on 2026-08-04, and this spec's only two failures are the 08-05 and 08-06 dailies. The #751 credential-settle guard asserts a binding upstream removed on purpose (#14311) — 19 @stable specs fail by design on 1.11.2 #1274 already migrated the shared helper agent-credential-settle.ts off that premise; this spec asserts the same shape inline (:691) and did not receive the migration — grep confirms it is the last such inline assertion in the suite. Recorded as a lead to be tested, not as a verdict: the regression hypothesis stays mandatory on [Daily #1330] openai-compatible — the persisted node's credential reads back empty; the assertion may be the last inline copy of the premise #14311 removed (recurrent 2×) #1334.

One qualification on row 2's recurrence

#1333 is filed on the recurrence, but the recurrence is weaker than the count reads. The signature is the generic expect(locator).toBeVisible() failed; 2026-07-14 (27 hard failures) and 2026-07-15 (15) were both mass-failure days inside the July window when this account's OpenAI quota was exhausted (#772/#775); and the same test's 2026-08-05 failure carries a different signature that #1274 has since explained. Today's drained-key evidence (6 skips + #1329/#1328) is recorded there as a lead for this run, with a funded-key re-run named as the measurement that settles it.

Umbrella stays OPEN — 2 quarantines pending

Per CONTRIBUTING.md → Triage protocol, both recurrent flakes must be quarantined as prevention (remove @stable and add test.fixme, together). Neither is done yet — they are a code change behind their own authorization:

• tests/tests-automations/regression/mcp/server/mcp-server-tab.spec.ts:6 — recurrent 2× (2026-08-05, 2026-08-06) — [Daily #1330] mcp-server-tab — the MCP-server dropdown is intermittently not clickable within 3 s (recurrent 2×) #1335
• tests/tests-automations/regression/flow-functionality/auto-save-off.spec.ts:59 — recurrent 2× (2026-07-22, 2026-08-06) — [Daily #1330] auto-save-off — reopening the just-created flow's card intermittently times out at 45 s (recurrent 2×) #1336

This issue closes once both are merged.

[Victor-w-Madeira]

Triage complete — closing

Both criterion-required quarantines are merged and verified on main:

Test	PR	State on main
tests/tests-automations/regression/mcp/server/mcp-server-tab.spec.ts:6	#1337 (merged)	test.fixme present, @stable absent — verified at 1e44212
tests/tests-automations/regression/flow-functionality/auto-save-off.spec.ts:59	#1338 (merged)	test.fixme present, @stable absent — verified at a2d52db

The 3 hard failures had @stable auto-removed by the workflow earlier (commit cb3082d).

Dispatched: #1332, #1333, #1334 (hard failures) and #1335, #1336 (recurrent flakes). Nothing was enriched — no open daily-failure issue matched any cluster.

Noted, not filed: 6 skips on the drained OpenAI key (already recorded in #1329/#1328; the structural fix is #976), 3 intentional/by-design skips, and 2 non-actionable flakes (agent-component-regression.spec.ts:75 and openai-compatible-provider-setup.spec.ts:558 — both first occurrences of their signature, absorbed by the retry budget).

Operational note, outside this triage's scope. The daily's provider rotation is openai → anthropic → google by weekday, so Monday 2026-08-10 lands on openai again. If the account is still without credits, that daily loses the same ~6 specs to skips and likely reproduces #1333. Recharging the account or landing #976 before then is what avoids it.