Raised from @rafaelgiln's review of #1052, where every test in a 20-test slice logged
500 /api/v1/store/tags → [Errno -2] Name or service not known (the container could not
resolve the Langflow Store) and no test failed. The question asked there was whether that is
a deliberate allowlist. It is not — there is no allowlist to be deliberate about.
What the code actually does
tests/fixtures/fixtures.ts monitors responses and collects two error kinds, then:
// after use(page)
if (flowErrors.length > 0 && !allowFlowErrors) {
throw new Error(...);
}
Only flow_error can fail a test. An http_error is pushed into the array, counted in the
⚠️ N HTTP error(s) detected line, and then nothing happens to it — on any path, for any
endpoint. So the 🚨 Backend Error: line is purely advisory, and the only thing standing
between a real backend 500 and a green test is a human reading the terminal.
That human step is written down as a validation requirement — CONTRIBUTING.md → Test
validation checklist step 5 ("Check the terminal logs") and CLAUDE.md → step 4 ("Confirm no
backend errors logged"). It is the load-bearing part of the mechanism, and the Store noise
above degrades it: a log that always carries three 500s is a log nobody reads carefully.
Second, narrower finding
The monitor is not the 4xx/5xx filter both docs describe:
status === 400 || status === 404 || status === 422 || status === 500
Four exact codes. 401, 403, 405, 409, 502, 503 and 504 are invisible to it — including the
403 — No authentication credentials provided shape that shows up under contention (seen in
the same #1052 review, and in #773), and the 502/503 a wedged or restarting backend
produces (#1030/#1048). CLAUDE.md:86 claims "automatic backend HTTP error monitoring
(4xx/5xx)"; that sentence is wrong today whichever way this issue is resolved.
The decision to make
Not obvious, which is why this is an issue rather than a PR:
- Make
http_error fail the test — matches what both docs imply, but needs an allowlist
first, or the Store 500 alone turns every spec red. Also needs a page.allowHttpErrors()
escape hatch for specs that assert error responses on purpose (the API specs do), mirroring
allowFlowErrors().
- Keep it advisory and make the log trustworthy — allowlist the known-benign endpoints
(/api/v1/store/* when the Store is unreachable) so a logged error means something, and fix
the two docs to say "logged, never fails".
- Widen the status filter — orthogonal to 1 vs 2, and cheap, but widening it while errors
stay advisory only adds noise to a signal nobody is required to act on.
Whichever way it goes, CLAUDE.md:86 and the two checklist steps have to end up describing
the real behaviour.
Not in scope
The store/tags 500 itself is environmental (no outbound DNS in the container), not a Langflow
defect. It is the example that exposed the gap, not the thing to fix.
Evidence
Raised from @rafaelgiln's review of #1052, where every test in a 20-test slice logged
500 /api/v1/store/tags→[Errno -2] Name or service not known(the container could notresolve the Langflow Store) and no test failed. The question asked there was whether that is
a deliberate allowlist. It is not — there is no allowlist to be deliberate about.
What the code actually does
tests/fixtures/fixtures.tsmonitors responses and collects two error kinds, then:Only
flow_errorcan fail a test. Anhttp_erroris pushed into the array, counted in the⚠️ N HTTP error(s) detectedline, and then nothing happens to it — on any path, for anyendpoint. So the
🚨 Backend Error:line is purely advisory, and the only thing standingbetween a real backend 500 and a green test is a human reading the terminal.
That human step is written down as a validation requirement —
CONTRIBUTING.md→ Testvalidation checklist step 5 ("Check the terminal logs") and
CLAUDE.md→ step 4 ("Confirm nobackend errors logged"). It is the load-bearing part of the mechanism, and the Store noise
above degrades it: a log that always carries three 500s is a log nobody reads carefully.
Second, narrower finding
The monitor is not the
4xx/5xxfilter both docs describe:Four exact codes. 401, 403, 405, 409, 502, 503 and 504 are invisible to it — including the
403 — No authentication credentials providedshape that shows up under contention (seen inthe same #1052 review, and in #773), and the
502/503a wedged or restarting backendproduces (#1030/#1048).
CLAUDE.md:86claims "automatic backend HTTP error monitoring(4xx/5xx)"; that sentence is wrong today whichever way this issue is resolved.
The decision to make
Not obvious, which is why this is an issue rather than a PR:
http_errorfail the test — matches what both docs imply, but needs an allowlistfirst, or the Store 500 alone turns every spec red. Also needs a
page.allowHttpErrors()escape hatch for specs that assert error responses on purpose (the API specs do), mirroring
allowFlowErrors().(
/api/v1/store/*when the Store is unreachable) so a logged error means something, and fixthe two docs to say "logged, never fails".
stay advisory only adds noise to a signal nobody is required to act on.
Whichever way it goes,
CLAUDE.md:86and the two checklist steps have to end up describingthe real behaviour.
Not in scope
The
store/tags500 itself is environmental (no outbound DNS in the container), not a Langflowdefect. It is the example that exposed the gap, not the thing to fix.
Evidence
tests/fixtures/fixtures.ts— the status filter and the throw that only coversflow_errorCONTRIBUTING.md→ Test validation checklist step 5;CLAUDE.md:86and its step 4