fix(mcp): harden mcp-client-agent against pre-1.10.x Playground prefill race - #228
Conversation
…ll race
The `agent calls echo MCP tool and returns echoed message` test failed
deterministically on Langflow 1.9.2 even though the underlying MCP echo
integration worked when reproduced manually. Investigation showed the
prefill `useEffect` in `flow-page-sliding-container.tsx` has `inputs` and
`nodes` in its dep array; their array references change across renders,
so the effect re-fires and resets `chatValue` back to the Chat Input
node's template `input_value` ("Hello, how are you?") between Playwright's
`.fill()` and the send button click. The send then dispatches the stale
value while the textarea still displays our prompt — the agent never
receives the echo instruction.
Hardening:
- Keep `.clear()` + `.fill()` + `toHaveValue()` (issue #226 acceptance
criteria) for the surface contract on 1.10.x.
- Run the actual send as an atomic DOM operation inside a single
`page.evaluate`: set the textarea value, dispatch a synthetic `input`
event so React's controlled component picks up the change, and call
`.click()` on the send button in the same synchronous tick. React
useEffects only run after the current task completes, so the prefill
cannot race the click.
- Assert the user message that reached the chat history equals the echo
prompt before waiting on the agent response, so a residual race fails
loudly at the input step instead of silently at the final assertion.
Verified green on local Langflow 1.9.2 across 5 consecutive `--retries=0`
runs (~18–22s each, one build stream per run = no retried agent calls).
Resolves #226.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…atch) The previous validation only checked the agent's response text for "hello mcp", which passed coincidentally when the LLM echoed the prompt back from memory without invoking any tool. Strengthen the assertion chain to prove the tool was actually called: Proof #1 — Playground renders a "Called tool" indicator (ContentBlockDisplay returns null when no tool was invoked, so this row only exists after a tool call). Proof #2 — the trigger reads "Called tool ECHO" (formatToolTitle uppercases the tool name); case-insensitive match tolerates rawTitle variants. Proof #3 — the echoed payload "hello mcp" appears in the final response (kept from the previous validation, now last in the chain). Notes for future maintainers, documented inline: - header-icon is OUTSIDE div-chat-message — they are siblings under the untestid'd "flex w-[94%]" wrapper in chat-message.tsx. - Langflow's AccordionTrigger (src/components/ui/accordion.tsx) uses asChild and wraps a <div>, NOT a <button>; selector must be tag-agnostic. - chat-message.tsx defaults hideHeader=false (accordion collapsed); a best-effort DOM click on the row's cursor-pointer chevron expands it. Quote the tool name in single quotes in the prompt — empirically the most reliable phrasing for gpt-4o-mini. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Victor-w-Madeira
left a comment
There was a problem hiding this comment.
LGTM on the code side — validation passed cleanly against project conventions.
Validation summary
--trace=onrun:2 passed (1.1m)— openai/gpt-5.4 (36.1s) + google/gemini-2.5-flash (26.4s), no retries, no🚨 Backend Error- ESLint: 0 errors / 5 pre-existing warnings (none introduced here)
tsc --noEmit: clean@stablepresent; spec imports fromfixtures/fixtures; usesSimpleAgentTemplatePage+models.jsonpermcp/CLAUDE.md- New Proof #1 (
Called tool) and Proof #2 (ECHO) assertions remove the "presence-only" false-positive risk that was flagged in CONTRIBUTING.md
Blocker before merge — docs/mcp/client/mcp-client-agent.md is out of sync with the code
-
Step 7 (line 27) still shows the old prompt
"Use the echo tool to echo: hello mcp". The code now sends"Use the 'echo' tool to echo: hello mcp"— the single quotes aroundechoare a deliberate choice documented in the inline comment (lower refusal rate on gpt-4o-mini). The doc must reflect the actual prompt the test sends. -
Step 7 does not describe the atomic
page.evaluatesend, which is the actual fix for #226 (set the textarea value, dispatch the syntheticinputevent, and click send in one synchronous tick so the prefilluseEffectcannot race the dispatch). The doc still describes the naivefill→clickflow that this PR is replacing — so a reader of the spec doc cannot tell what the PR actually changed. -
Validation criterion (line 35) lists only "Agent response contains
hello mcp". The test now asserts three independent proofs:- Playground renders a
Called toolindicator (agent invoked some tool) - The tool invoked is
echo(case-insensitive match on theECHOformatted title) - The echoed payload
hello mcpappears in the final agent message
Without these in the doc, the rationale of commit
7e78416("assert echo MCP tool was actually invoked (not just text match)") is invisible to readers of the spec. - Playground renders a
Please update the spec doc to reflect the new prompt, the atomic-send technique, and the three-proof validation criterion before merging.
Summary
agent calls echo MCP tool and returns echoed messagetest intests/tests-automations/regression/mcp/client/mcp-client-agent.spec.tsfailed deterministically on Langflow 1.9.2 despite the underlying MCP echo integration working when reproduced manually.flow-page-sliding-container.tsxthe prefilluseEffecthasinputsandnodesin its dep array. Their array references change across renders, so the effect re-fires and resetschatValueback to the Chat Input node's templateinput_value("Hello, how are you?") between Playwright's.fill()and the send-button click —send()then dispatches the stale value while the textarea still displays our prompt. Issue's Option 1 + Option 2 alone do not close the race (verified empirically)..clear()+.fill()+toHaveValue()(issue acceptance contract), then run the actual send as an atomic DOM operation inside a singlepage.evaluate— set the textarea value, dispatch the syntheticinputevent so React's controlled component updates, and call.click()on the send button in the same synchronous tick. ReactuseEffects only run after the current task completes, so the prefill cannot race the click. Added an exact-match assertion on the user message that landed in chat to catch any residual race loudly at the input step.Test plan
npm run typecheck— 0 errorsnpm run linton the touched file — 0 errors, 0 new warnings (5 pre-existing)npx playwright test ... --workers=1 --retries=0runs PASS on local Langflow 1.9.2 withMODEL_TEST_ID=gpt-4o-mini(~18–22s each, one build-events stream per run = no retried agent calls, deterministic)toHaveValueassertion → failed at the input step with a clear diagnostic showing the textarea reverting to"Hello, how are you?"mid-typing; reverted → PASS--trace=onrun produced a coherent trace matching the declaredtest.step()blocks🚨 Backend Erroroccurrences in the test outputlangflowai/langflow-nightly:latestbefore merge (issue acceptance criterion)Spec doc
docs/mcp/client/mcp-client-agent.mdintentionally not touched — the test logic (send the echo prompt, assert the echoed response) is unchanged, andLast validatedis not bumped because end-to-end re-validation against nightly is still pending.🤖 Generated with Claude Code