test(regression): tag global-variables-crud as @stable + add Credential value hiding test by daniellicnerski1 · Pull Request #245 · oriontech-me/langflow-e2e

daniellicnerski1 · 2026-05-18T18:17:17Z

Summary

Tagged the 2 existing tests in tests/tests-automations/regression/ui-ux/global-variables-crud.spec.ts as @stable after running the full 7-step validation pipeline (typecheck, lint, anti-pattern checklist, run --retries=0, force-fail per test, --trace=on, backend error audit — all green).
Added a 3rd test, "Credential variable value is hidden from the variable list", that creates a Credential-typed variable with a distinctive sentinel value and asserts the value never surfaces as visible text on the page (getByText(sentinel).toHaveCount(0)). Defends against regressions that would leak API keys / secrets into the list, a toast, or any preview.
Updated docs/ui-ux/global-variables-crud.md to document Test 3 and remove "Credential-type variable creation" from the "does not cover" section.
Updated QA-CHECKLIST.md bullets in §4.3: Create global variable, Delete global variable, Create global variable of type "Generic" flipped [-] → [x]; added a new [x] bullet for Credential variable value is hidden from the variable list.

Test plan

npm run typecheck — clean
npx eslint <spec> — 0 errors (warnings are pre-existing patterns already accepted in @stable specs like webhook-component-regression, tool-mode)
npx playwright test <spec> --workers=1 --retries=0 — 3 passed (24.7s) on the validation run, 3 passed (28.3s) on the --trace=on run
Force-fail per test: each test fails at the expected assertion line, then reverted and passes again
Zero 🚨 Backend Error occurrences in the run output

Notes

The Credential test asserts only that the saved value never surfaces as visible text. It does not interact with the eye/show-value toggle nor assert that the input uses type="password" — browser-level masking and toggle UX are out of scope and documented as such in the spec doc.

…al value hiding test

Copilot

Pull request overview

This PR promotes the Global Variables CRUD regression coverage to @stable, adds a credential secrecy regression test, and documents the related UI/UX scenario.

Changes:

Added @stable tags to existing global variable create/delete tests.
Added a new test asserting a credential sentinel value is not visible after save.
Updated QA checklist and added documentation for the Global Variables CRUD scenario.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 6 comments.

File	Description
`tests/tests-automations/regression/ui-ux/global-variables-crud.spec.ts`	Adds stable tags and a new credential value hiding test.
`QA-CHECKLIST.md`	Marks related Global Variables checklist items as validated.
`docs/ui-ux/global-variables-crud.md`	Documents the covered Global Variables CRUD tests and exclusions.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

daniellicnerski1 · 2026-05-19T13:36:59Z

+      // Credential is the default type in the modal — no need to switch tabs
+      await page
+        .getByPlaceholder("Enter a name for the variable...")
+        .fill(varName);
+      await page
+        .getByPlaceholder("Enter a value for the variable...")
+        .fill(sentinelValue);


Caught a real bug — fixed in fc12387. Confirmed via tests/tests-automations/regression/ui-ux/globalVariables.spec.ts:43-66 that the modal opens on the Generic tab by default: line 51 asserts "Generic" type label is visible immediately after Add New, and line 63 explicitly clicks credential-tab before saving the Credential variable.

So the previous test was silently creating a Generic variable and asserting its value wasn't leaked — which passes trivially because Generic variables also do not echo their value back into the list.

Fix: inserted await page.getByTestId("credential-tab").click() between the name fill and value fill, matching the existing flow. After the fix:

--workers=1 --retries=0: 3/3 PASS (26.1 s)

Force-fail on the sentinel toHaveCount(0) → toHaveCount(42): failed exactly on the assertion line, reverted, repassed.

Zero backend errors in the audit run.

daniellicnerski1 · 2026-05-19T13:37:01Z

+
+1. Run setup
+2. Click "Add New Variable" via JS evaluate
+3. Credential is the default type — no need to switch tabs


Valid — fixed in fc12387 in lockstep with the spec change. The Step-by-step section for Test 3 now has an explicit step "Click credential-tab — the modal opens on the Generic tab by default; this switches it to Credential before save." The misleading "Credential is the default type — no need to switch tabs" line is gone.

daniellicnerski1 · 2026-05-19T13:37:06Z

- [-] Create global variable of type "Generic"
+- [x] Delete global variable
+- [x] Create global variable of type "Generic"
+- [x] Credential variable value is hidden from the variable list


Declining this one — the Copilot suggestion contradicts the convention in this repo.

CLAUDE.md §QA-CHECKLIST.md (lines 172-177) explicitly states that the Coverage Summary table and the Phase 0 — Validated block (counts + bulleted list) are auto-generated and must never be edited manually. The update-coverage-summary.yml workflow regenerates both on every push to main that touches QA-CHECKLIST.md, the regeneration scripts, or any .spec.ts. The regenerators are idempotent — running them locally either produces no diff (if main is already in sync) or a transient diff that is immediately overwritten on the next push to main, which adds churn without value.

So flipping the bullets to [x] is the only edit needed in this PR; the table and Phase 0 block will be in sync as soon as this merges.

daniellicnerski1 · 2026-05-19T13:37:08Z

+
+## Step by step *(required)*
+
+**Both tests share the same setup:**


Valid, fixed in fc12387. Heading is now "All three tests share the same setup:" — the file has had 3 tests in scope since Test 3 was added; the "Both" carryover was stale.

daniellicnerski1 · 2026-05-19T13:37:12Z

+      // is the value being echoed back into the list, a toast, or any preview.
+      await expect(
+        page.getByText(sentinelValue, { exact: true }),
+      ).toHaveCount(0, { timeout: 5000 });


Valid — fixed in fc12387. Dropped exact: true so getByText(sentinelValue) now does substring matching. A leak embedded in a longer message ("Saved: SECRET-SENTINEL-..." or any labeled preview) now fails the assertion, which matches the guarantee stated by the test name ("hidden from the variable list").

The inline comment was also expanded to explain the substring behaviour and clarify the deliberate gap (input-value attributes still don't count as visible text, since <input type="password" value="…"> is the expected mechanism for masking the editable input). Force-fail probe at toHaveCount(0) → toHaveCount(42) confirmed the assertion fails on the exact line; reverted, repassed.

daniellicnerski1 · 2026-05-19T13:37:16Z

+4. Fill name `credential-{timestamp}`, fill value with a distinctive sentinel `SECRET-SENTINEL-{timestamp}`
+5. Click "Save Variable"
+6. Sanity: assert variable name is visible in the list
+7. Critical: assert `getByText(sentinelValue, { exact: true })` has `count() === 0` — value must not surface as visible text anywhere on the page


Valid, fixed in fc12387 in lockstep with the spec. The validation criterion now reads:

Credential value (sentinel) has substring-text count 0 anywhere on the page after save (Test 3) — getByText(sentinelValue) without exact: true, so embedded occurrences inside longer messages also fail the test

The Test 3 Step-by-step also calls out the substring behaviour explicitly ("getByText(sentinelValue) (substring match, no exact)") so the doc no longer overstates what the assertion catches.

…ntinel check Addresses Copilot review on PR #245. Two real bugs in the new Test 3: 1. The test never clicked `credential-tab` before saving, so it was silently creating a Generic variable and asserting that the value wasn't leaked. The assertion passed trivially because Generic variables also don't echo their value back into the list. The existing globalVariables.spec.ts:63 confirms Generic is the default tab — Credential requires an explicit click. Fixed by inserting `getByTestId('credential-tab').click()` between name fill and value fill, matching the existing flow. 2. The sentinel assertion used `getByText(sentinelValue, { exact: true })`, which only catches the value when rendered as a standalone exact-match. A leak embedded inside a toast or labeled preview like 'Saved: SECRET-SENTINEL-...' would have passed. Dropped `exact: true` so substring matches also fail the assertion — that matches the guarantee described in the test name ("hidden from the variable list"). Doc updated to reflect both changes (Step-by-step gains the credential-tab click step; validation criterion now describes substring matching). The heading 'Both tests share the same setup' was also stale — fixed to 'All three tests'. 7-step validation pipeline passed: - typecheck clean - lint: 0 errors, 31 pre-existing warnings (no new ones introduced) - 3/3 PASS at --workers=1 --retries=0 (26.1 s) - Force-fail on `toHaveCount(0)` -> `toHaveCount(42)` failed on the expected line, reverted, re-passed - Trace coherent, zero backend errors

test(regression): tag global-variables-crud as @stable + add Credenti…

07d0116

…al value hiding test

Victor-w-Madeira requested a review from Copilot May 18, 2026 21:12

Copilot started reviewing on behalf of Victor-w-Madeira May 18, 2026 21:13 View session

Copilot AI reviewed May 18, 2026

View reviewed changes

daniellicnerski1 merged commit 3c17f5f into main May 19, 2026
2 checks passed

Victor-w-Madeira deleted the test/global-variables-crud-stable branch May 19, 2026 15:57

daniellicnerski1 mentioned this pull request May 23, 2026

chore(ui-ux): consolidate global-variables tests cluster #321

Merged

Navigation Menu

Uh oh!

test(regression): tag global-variables-crud as @stable + add Credential value hiding test - #245

test(regression): tag global-variables-crud as @stable + add Credential value hiding test#245
daniellicnerski1 merged 2 commits into
mainfrom
test/global-variables-crud-stable

daniellicnerski1 commented May 18, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

daniellicnerski1 May 19, 2026

Uh oh!

daniellicnerski1 May 19, 2026

Uh oh!

daniellicnerski1 May 19, 2026

Uh oh!

daniellicnerski1 May 19, 2026

Uh oh!

daniellicnerski1 May 19, 2026

Uh oh!

daniellicnerski1 May 19, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants


		## Step by step (required)

		Both tests share the same setup:

Uh oh!

Conversation

daniellicnerski1 commented May 18, 2026

Summary

Test plan

Notes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

daniellicnerski1 May 19, 2026

Choose a reason for hiding this comment

Uh oh!

daniellicnerski1 May 19, 2026

Choose a reason for hiding this comment

Uh oh!

daniellicnerski1 May 19, 2026

Choose a reason for hiding this comment

Uh oh!

daniellicnerski1 May 19, 2026

Choose a reason for hiding this comment

Uh oh!

daniellicnerski1 May 19, 2026

Choose a reason for hiding this comment

Uh oh!

daniellicnerski1 May 19, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants