test(core-components): cover use_double_brackets toggle in prompt template - #218
Conversation
…plate Adds prompt-template-double-brackets-regression with 5 @stable tests exercising the use_double_brackets toggle and the mustache code path (InspectionPanel exposure, default f-string mode, toggle-on switch to mustache, toggle-off revert with re-extraction on next save, and backend persistence via GET /api/v1/flows/{id}). Cross-links from the existing prompt-template spec doc and extends the §3.2 QA checklist.
- Scope display-name assertion with .first() to avoid Playwright strict-mode failure if "Use Double Brackets" ever appears elsewhere on the page. - Add baseline check in test 5: poll the saved flow for use_double_brackets.value === false before toggling, so the test proves the round-trip, not just the final state. - Surface test 4's mode-swap as an explicit test-level expect() on the modal-open testid pair, so the HTML report shows a real assertion in the "swaps back" step rather than only the helper's internal wait. - Hoist readUseDoubleBrackets() to module scope to keep the response-status guard out of the test body (avoids playwright/no-conditional-in-test). - Reword the spec doc's "core contracts" paragraph: drop the unconditional "real-time re-extraction" claim that no longer matches test 4's contract (re-extraction is verified on the next save, not on the toggle alone).
There was a problem hiding this comment.
Pull request overview
This PR adds end-to-end Playwright coverage for the Prompt Template component’s use_double_brackets toggle, ensuring both the default f-string parsing path and the mustache parsing path are exercised and documented within the langflow-e2e regression suite.
Changes:
- Adds a new
@stableregression spec covering toggle visibility, parser switching behavior, handle extraction semantics, and persisteduse_double_bracketsstate viaGET /api/v1/flows/{id}. - Updates QA documentation to cross-link the new coverage and marks the new scenarios as validated in the checklist.
- Adds a dedicated regression doc describing the new spec’s intent, steps, and upstream dependencies.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| tests/tests-automations/regression/core-components/prompt-template-double-brackets-regression.spec.ts | Adds 5 serial @stable Playwright tests covering use_double_brackets behavior end-to-end, including backend persistence polling. |
| QA-CHECKLIST.md | Adds five new §3.2 checklist entries pointing to the new spec. |
| docs/core-components/prompt-template-double-brackets-regression.md | New documentation describing the new regression spec’s scenarios, steps, and dependencies. |
| docs/core-components/prompt-template-component-regression.md | Cross-links the new spec from the existing Prompt Template regression doc. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // `BoolInput(..., display_name="Use Double Brackets", ...)` declaration — | ||
| // asserting it catches an accidental rename at the source. Scoped to the | ||
| // toggle's enclosing field row so an unrelated occurrence of the string | ||
| // elsewhere on the page (a tooltip, a help string) cannot satisfy or | ||
| // break the assertion via Playwright strict mode. The `info` text is | ||
| // intentionally not asserted — the InspectionPanel collapses it into a | ||
| // hover-tooltip icon when the panel is narrow. | ||
| // `.first()` avoids Playwright strict-mode failures if "Use Double | ||
| // Brackets" ever shows up elsewhere (e.g. a future tooltip or help | ||
| // string). The toggle's existence was already asserted in the previous | ||
| // step, so this assertion is solely about the label string. | ||
| await expect( | ||
| page.getByText("Use Double Brackets").first(), | ||
| ).toBeVisible(); |
There was a problem hiding this comment.
Addressed in commit 9a2e05f. The bare .first() was indeed not a real scope — the comment claimed scoping while the implementation didn't deliver it. The fix anchors the assertion on the toggle via an XPath ancestor walk:
const labelOwner = page
.getByTestId("toggle_bool_use_double_brackets")
.locator("xpath=ancestor::*[contains(., 'Use Double Brackets')][1]");
await expect(labelOwner).toBeVisible();If the label is renamed and no ancestor of the toggle has the text in its subtree, the locator resolves to zero elements and toBeVisible() fails — which is the regression we want to catch. Force-fail confirmed (substituted a placeholder string and the assertion failed cleanly with element(s) not found).
…opilot)
Replaces `getByText("Use Double Brackets").first()` with an XPath ancestor
walk from the toggle element. The previous version would have passed if
"Use Double Brackets" appeared anywhere else on the page (a future tooltip,
help text, or stale element kept after the label was removed); the new
version resolves to zero elements and fails when the label-toggle wiring
regresses, which is what the test exists to detect.
Verified by force-fail: the toggle has no ancestor whose subtree contains
a placeholder string, so the locator returns nothing and toBeVisible fails
cleanly.
Summary
prompt-template-double-brackets-regressionwith 5@stabletests exercising theuse_double_bracketstoggle and the mustache code path on the Prompt Template component.prompt-template-component-regression.mddoc and extends the §3.2 entries inQA-CHECKLIST.md.nightly(23s end-to-end); typecheck + ESLint pass with 0 errors and no new warnings.Scope of coverage
toggle_bool_use_double_bracketsis visible after add; "Use Double Brackets" display name renders{var}and treats{{var}}as literalHello {single} and {{double}}!yields exactly one handle (single){{var}}creates handle and{var}is ignoreddoublehandle{{name}}→ toggle OFF → re-save →namehandle drops → save{var}→varhandle appearsuse_double_bracketsvalue persists in the autosaved flowGET /api/v1/flows/{id}returnstemplate.use_double_brackets.value === trueafter toggleNotes on scope adjustments from issue #213
Two of the issue's original framings were revised during implementation to match how the UI actually behaves:
infotext and the "gated behind advanced options" framing for test 1 do not hold in practice —advanced=Trueonly filters the field from the on-canvas node body viaisCanvasVisible(); the InspectionPanel renders the bool toggle directly. The test instead asserts the display name (BoolInput(display_name=...)) is visible. Theinfostring collapses into a hover-tooltip icon when the panel is narrow and is not asserted.update_build_configruns, but the dynamic handles linger until the next save. The test verifies the contract end-to-end (toggle + save) rather than asserting toggle-alone re-extraction.Both adjustments are documented in
docs/core-components/prompt-template-double-brackets-regression.mdunder Notes.Test plan
npm run typecheck— 0 errorsnpm run lint— 0 errors, no new warnings from the new specnpx playwright test tests/tests-automations/regression/core-components/prompt-template-double-brackets-regression.spec.ts— 5/5 pass againstlangflowai/langflow-nightly:latestCloses #213