chore(flow-functionality): add @stable and harden curlApiGeneration test - #139
Conversation
The previous version asserted only that the clipboard was non-empty, which silently passed even when the wrong (PowerShell) variant was copied. The test now explicitly switches to the macOS/Linux platform sub-tab and validates the curl command structurally: HTTP verb, /api/v1/run/<UUID> URL with stream flag, JSON content-type header, x-api-key header, and the JSON body keys (input_value, session_id, output_type). Adds the spec doc under docs/flow-functionality/, and marks coverage as [x] in QA-CHECKLIST.md and QA-SCENARIOS-GUIDE.md scenario 5.1. Drops the @api tag — this test exercises UI snippet generation, not the REST API endpoint.
There was a problem hiding this comment.
Pull request overview
This PR hardens the Playwright regression test that validates the “API Access → cURL” snippet generation/copy flow, and updates QA documentation to reflect that coverage.
Changes:
- Strengthened
curlApiGeneratione2e assertions from “clipboard non-empty” to structural validation of the generated macOS/Linux cURL command. - Reduced test flakiness by explicitly selecting the macOS/Linux platform variant and improving step-level traceability via
test.step(), plus adding@stable. - Added/updated QA documentation to mark the scenario/checklist as covered and to document the spec for ongoing maintenance.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| tests/tests-automations/regression/flow-functionality/curlApiGeneration.spec.ts | Makes the cURL generation test deterministic (forces macOS/Linux) and validates key command structure instead of only clipboard length. |
| QA-SCENARIOS-GUIDE.md | Marks scenario 5.1 as covered. |
| QA-CHECKLIST.md | Marks “Generate curl for API execution” as covered and links to the spec. |
| docs/flow-functionality/curlApiGeneration.md | Adds a dedicated spec doc describing intent, validation criteria, and upstream dependencies. |
Comments suppressed due to low confidence (1)
QA-SCENARIOS-GUIDE.md:310
- Marking scenario 5.1 as covered is potentially misleading because the scenario’s step-by-step currently calls out verifying a
curl -X POSTmethod, while the actual generator/test expectation inflow-functionality/curlApiGeneration.spec.tsiscurl --request POST. Consider updating the scenario text to assert the HTTP verb is POST (without prescribing-Xvs--request) so the checklist/guide aligns with the test and the UI output format.
### 5.1 Generate curl for execution `[x]`
**Objective:** Verify that Langflow generates a valid `curl` command for flow execution.
**Step by step:**
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Replace prescriptive 'curl -X POST' wording with a method-agnostic description, matching how the test actually validates (POST verb via either '-X POST' or '--request POST'). Resolves Copilot review feedback on PR #139.
Copilot review — addressedThe single low-confidence note pointed out a real discrepancy: Branch was also merged with the latest Full revalidation after the change
|
Summary
clipboardContent.length > 0assertion with structural checks against the actual curl command (HTTP verb,/api/v1/run/<UUID>?stream=false, headers, JSON body keys).getOS()was driving the previous flakiness — Chromium'snavigator.platformwas producing the PowerShell variant).curl_api_generation→ sentence form), wrap each phase intest.step(), drop the misleading@apitag (this test does not exercise the REST API), add@stable.docs/flow-functionality/curlApiGeneration.md.[x]inQA-CHECKLIST.mdandQA-SCENARIOS-GUIDE.mdscenario 5.1.Validation
"input_value": "Hello"with"ThisShouldFail"→ test fails for the right reason.--retries=0, average 3.7s per run.npx eslint tests/tests-automations/regression/flow-functionality/curlApiGeneration.spec.ts— clean.npx tsc --noEmit— clean.waitForTimeout, noif/else, notry/catch; 8 expects per test.Test plan
weekly-stable.ymlpicks up the test on the next scheduled run via the new@stabletag.pr-validation.yml(typecheck + lint) passes.