qa-infra: go-httpbin is wired into daily-stable only — PR/nightly/manual still hit public httpbin.org and red on its outages

[rafaelgiln]

The external-echo decoupling built for #462/#639 exists in one workflow. Every other lane still calls the public host, so a third-party outage reds a PR that changed nothing related.

Surfaced on PR #1122 (run 30537081047).

Evidence

Run impacted E2E specs → 2 failed / 4 flaky / 74 passed. Both hard failures are the same third party answering 503, and 8 occurrences of 503 Service Temporarily Unavailable appear in the job log:

Spec (line)	Failure
core-functionality/llm-agents/agent-multi-tool-selection.spec.ts:439	Expected: "fetch-tool-returned-expected" / Received: "fetch_content output did not contain /Sample Slide Show/i: … 503 Service Temporarily Unavailable …"
core-functionality/llm-agents/agent-tool-inspection.spec.ts:318	Expected: "tool-inspection-captured" / Received: "fetch_content block found but input/output did not match — input={\"urls\":[\"https://httpbin.org/json\"]} output={… 503 Service Temporarily Unavailable"

The PR under test changed tests/helpers/flows/load-template-by-name.ts; neither spec has anything to do with it. It was selected only because the import graph reaches it (#1054), which is correct behaviour — the point is that a correct selection was reded by a third party.

Root cause: the fix was never extended past daily-stable

daily-stable.yml self-hosts ghcr.io/mccutchen/go-httpbin:2.23.1 as a service container and resolves ECHO_BASE_URL to its container IP. The specs read that variable and fall back to the public host when it is unset:

process.env.ECHO_BASE_URL ?? "https://httpbin.org"      // agent-multi-tool-selection, agent-tool-inspection
process.env.ECHO_BASE_URL ?? "https://postman-echo.com" // api-component-regression, api-request-component-regression

Service containers per workflow today:

Workflow	Services	ECHO_BASE_URL
daily-stable.yml	langflow, ollama, go-httpbin	resolved to the container IP
pr-validation.yml	langflow	unset → public fallback
nightly.yml	langflow, ollama	unset → public fallback
manual.yml	langflow, ollama	unset → public fallback

So the four specs above are protected exactly once a day and exposed on every PR, every nightly and every manual run.

Second gap: URLs that no variable can redirect

Even with ECHO_BASE_URL set, these stay pinned to the public host — some of them assert the literal URL string, so redirecting them is not a one-line substitution:

• api/flows/api-request-component-ui.spec.ts:51-52 — fills and asserts https://httpbin.org/get
• api/flows/api-component-regression.spec.ts:39 — CURL_TARGET_URL = "https://httpbin.org/post"
• api/flows/api-component-regression.spec.ts:457,470,718,738 — fills httpbin.org/get / /post and asserts "source": "https://httpbin.org/…" inside the result dialog

api-component-regression.spec.ts:23 already records this as knowingly deferred ("tests below keep their hardcoded httpbin.org URLs. Migrating those, and …").

Constraint for whoever implements this

Langflow refuses to fetch a loopback host — the same run logged:

SSRF Protection: Hostname localhost resolves to blocked IP address(es): ::1, 127.0.0.1.
To allow this hostname, add it to LANGFLOW_SSRF_ALLOWED_HOSTS environment variable.

which is why daily-stable.yml resolves the service to a raw container IP rather than http://go-httpbin:8080 (the API Request component also rejects a single-label host). Any new lane must reuse that resolution step — or set LANGFLOW_SSRF_ALLOWED_HOSTS — not point a spec at localhost.

Deliverables (Done when)

• The go-httpbin service + the Resolve go-httpbin endpoint step are shared, not copy-pasted: extract them (composite action or reusable workflow) and wire them into pr-validation.yml, nightly.yml and manual.yml.
• A lane where the service fails to come up says so and does not silently fall back to the public host on a spec that is gating a PR. daily-stable chose fail-soft with a warning (ECHO_BASE_URL left unset); decide and record whether the PR lane should fail-closed instead, since that is the lane a human is waiting on.
• The hardcoded URLs above are either migrated to the variable (including the dialog assertions that read the literal URL) or explicitly scoped out with the reason recorded next to them.
• Verified by a manual.yml dispatch (or one PR run) where the two agent specs pass with ECHO_BASE_URL pointing at the service, plus a run with the service deliberately absent to prove the chosen failure mode.

Prior art

#462 (postman-echo outage), #639 (the go-httpbin tag that blocked the whole suite), and the recurrences its comment records (#383/#407). All closed — this is the unfinished half of that work, not a new class.