test(llm-agents): quarantine the max-iterations stop assertion against #1264

The PR lane went red on `agent-max-iterations` › "agent stops when max iterations
is reached", 3 of 3 attempts, with the agent answering the task normally instead
of returning `Model call limits exceeded: run limit (1/1)`. The bubble renders and
the locator resolves 34 times, so it is a content failure, not a timeout — the
signature #1264 has been open on since the 2026-08-04 daily.

Not this branch's diff: the same test fails identically on `origin/main`. What
changed is the SELECTION — touching this file put a spec the daily never runs
(`@regression`, never `@stable`) into the impacted-specs lane.

The file is serial, so that failure also skipped the `@stable` causal control —
the half that works, and the half this branch's cleanup change needed to exercise.
So a red here bought no coverage and cost the lane its verification.

Quarantine with `test.fixme`, the same treatment mcp-server.spec.ts (#1266) and
openai-compatible-provider-setup.spec.ts already carry, and record the measurement
where a reader will hit it: reproduced on 1.12.0.dev18 LOCALLY, off CI load, on
claude-haiku-4-5, claude-opus-5 and claude-opus-4-5 — which rules out the mid-run
backend wedge #1264's triage left open as a possible cover.

The causal control is deliberately NOT quarantined: on its own it proves only that
a high limit finishes, and it is what will show the pair working again. Lifting the
quarantine is #1264's call.

Checklist bullets move from `[x]` to `[~]` with the reason, in both §6.2 and §7.7.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Author: Rafael

QA-CHECKLIST.md

@@ -351,8 +351,8 @@
 - [-] Composio (tool integration for Agent) → `composio.spec.ts`
 - [x] Playground shows error when LLM run endpoint returns 500 (mocked invalid API key) → `llm-agents/llm-invalid-api-key-ui.spec.ts`
 - [x] Playground input remains usable after API error (mocked) → `llm-agents/llm-invalid-api-key-ui.spec.ts`
-- [x] Agent stops when configured stop condition is reached → `core-functionality/llm-agents/agent-max-iterations.spec.ts` (`max_iterations` is the Agent's only configurable stop mechanism — no dedicated stop-condition field exists; see #824)
-- [x] Agent stops when maximum number of iterations is reached → `core-functionality/llm-agents/agent-max-iterations.spec.ts`
+- [~] Agent stops when configured stop condition is reached → `core-functionality/llm-agents/agent-max-iterations.spec.ts` (`max_iterations` is the Agent's only configurable stop mechanism — no dedicated stop-condition field exists; see #824. **Partial for the same reason as the bullet below:** the stop itself is quarantined against #1264)
+- [~] Agent stops when maximum number of iterations is reached → `core-functionality/llm-agents/agent-max-iterations.spec.ts` (**partial: the stop assertion is quarantined (`test.fixme`, never `@stable`) against a live product failure — #1264.** With `max_iterations=1` on a task that needs several tool-calling iterations, the agent answers the task normally instead of returning `Model call limits exceeded: run limit (1/1)`; the message renders, so this is a content failure, not a timeout. Reproduced on 1.12.0.dev18 locally, off CI load, on `claude-haiku-4-5`, `claude-opus-5` and `claude-opus-4-5`, which rules out the mid-run backend wedge #1264's triage left open as a cover. The `@stable` causal control — a high limit finishes without the limit message — still runs, so what remains proven is that a high cap does not stop the agent, not that a low one does)
 - [x] Agent with multiple configured tools executes correctly → `agent-multi-tool-selection.spec.ts`
 - [ ] Agent with configured timeout respects the limit (no product surface on 1.12.x — the Agent component exposes no timeout field: its inputs are `max_iterations`, `max_tokens`, `n_messages`, `system_prompt`, `context_id`, `stream`, and tools; Langflow's only configurable per-component timeouts live on the A2A Agent (`timeout`) and MCP tools (`tool_execution_timeout`), not the Agent, and the Language Model / chat models expose none either. The client/transport execution timeout that bounds any run is covered by `ui-ux/execution-error-notification.spec.ts`. Not automatable as written; #825, same class as #824)
 - [x] Connecting an external model in Agent drops the prior model selection (connection-mode isolation, prevents stale provider config) → `llm-agents/agent-model-connection-isolation.spec.ts`
@@ -436,7 +436,7 @@
 
 #### 7.7 Model Parameters (Agent)
 - [x] Maximum token count — response truncated as configured → `llm-agents/agent-max-tokens.spec.ts`
-- [x] Maximum agent iterations → `core-functionality/llm-agents/agent-max-iterations.spec.ts`
+- [~] Maximum agent iterations → `core-functionality/llm-agents/agent-max-iterations.spec.ts` (partial — the stop assertion is quarantined against #1264; see §6.2 for the measurement)
 - [x] Use of custom `context_id` for memory isolation → `agent-context-id-isolation.spec.ts`
 - [x] Output formatting (JSON via output_schema, Markdown, plain text) → `agent-structured-output.spec.ts`
 

docs/core-functionality/llm-agents/agent-max-iterations.md

@@ -29,6 +29,21 @@ Two tests establish this **causally**:
 > completes). The bug appears **fixed**, so this is authored as a normal passing
 > `@stable` test, not an expected-fail. Flagged on the issue/PR.
 
+> **Quarantine (2026-08-06, nightly 1.12.0.dev18):** Test 1 is `test.fixme`
+> against **#1264** — the product no longer enforces the cap. With
+> `max_iterations = 1` the agent answers the task normally
+> (`"I'll fetch that URL for you."` and variants) instead of returning the limit
+> message; the bubble renders and the locator resolves 34 times, so this is a
+> **content** failure, not a timeout. Reproduced LOCALLY on 1.12.0.dev18, off CI
+> load, on `claude-haiku-4-5`, `claude-opus-5` and `claude-opus-4-5` — which is
+> what rules out the mid-run backend wedge #1264's triage left open as a possible
+> cover. Quarantined rather than left red because the test is `@regression` and
+> never `@stable`: the daily does not run it, so a red here only failed the PR
+> lane of any diff touching this file — and the file is serial, so it also skipped
+> the `@stable` causal control. Test 2 is deliberately NOT quarantined; on its own
+> it proves only that a high limit finishes. Lifting the quarantine is #1264's
+> call, once the cap is enforced again on `langflowai/langflow-nightly:latest`.
+
 If this fails, the agent no longer honours its iteration cap — a regression in a
 core safety/cost control.
 
@@ -38,7 +53,9 @@ core safety/cost control.
 
 `@stable` `@regression` `@agents` `@playground`
 
-`@stable` added only after multiple clean `--retries=0` runs on the fresh nightly.
+`@stable` added only after multiple clean `--retries=0` runs on the fresh nightly,
+and it is carried by **Test 2 only** — Test 1 has never carried it and is now
+`test.fixme` (see the quarantine note above; #1264).
 `@regression` — guards the max-iterations enforcement from regressing (the bug
 #481 documented); `@agents` — agent execution; `@playground` — the flow is run
 through the Playground.

tests/tests-automations/regression/core-functionality/llm-agents/agent-max-iterations.spec.ts

@@ -161,7 +161,32 @@ for (const { label, options, skipReason } of targets) {
   const provider = options.provider ?? (Object.keys(providerConfigMap)[0] as Provider);
 
   test.describe(`Agent Max Iterations [${label}]`, () => {
-    test(
+    // QUARANTINED — the product no longer enforces the cap (#1264, still open).
+    // The agent answers the task normally instead of stopping: the assertion below
+    // reads a real, rendered message (`34 × locator resolved`), so this is a
+    // content failure, not a timeout. Received on three independent runs of the
+    // three different Anthropic models tried:
+    //
+    //   "I'll fetch that URL for you."
+    //   "I'll fetch that URL for you and retrieve the version value."
+    //   "I'll fetch that URL for you and get the version value."   (daily #1258)
+    //
+    // Reproduced on 1.12.0.dev18 LOCALLY, off CI load, on claude-haiku-4-5,
+    // claude-opus-5 and claude-opus-4-5 — which is what rules out the mid-run
+    // backend wedge #1264's triage left open as a possible cover.
+    //
+    // `test.fixme` rather than leaving it red, for the reason the same quarantine
+    // is used in mcp-server.spec.ts (#1266) and openai-compatible-provider-setup:
+    // this test is `@regression` and never `@stable`, so the daily does not run it
+    // and the only thing a red here does is fail the PR lane of any diff that
+    // touches this file — while the file is SERIAL, so its failure also skipped
+    // the `@stable` causal control below, the half that still works.
+    //
+    // Lifting the quarantine (remove `test.fixme`) is #1264's call, once the cap is
+    // enforced again on `langflowai/langflow-nightly:latest`. The causal control
+    // below is deliberately NOT quarantined: on its own it proves only that a high
+    // limit finishes, and it is what will show the pair working again.
+    test.fixme(
       "agent stops when max iterations is reached",
       { tag: ["@regression", "@agents", "@playground"] },
       async ({ page }) => {