[bugzillarest] Skip bug pages that return HTTP 400 instead of aborting#911
Open
aatchison wants to merge 3 commits into
Open
[bugzillarest] Skip bug pages that return HTTP 400 instead of aborting#911aatchison wants to merge 3 commits into
aatchison wants to merge 3 commits into
Conversation
added 2 commits
June 11, 2026 09:34
A single bug can make the Bugzilla REST API return a 400 (e.g. a field whose data is missing from network storage: "Failed to fetch key <n> from network storage: Not Found"). Because the backend requests include_fields=_all and pages by last_change_time, one such bug aborts the whole batch and freezes incremental collection indefinitely. Catch the HTTPError in __fetch_and_parse_bugs: on a 400, log and skip that page (advance the offset) and continue, so one bad bug no longer wedges collection. A MAX_CONSECUTIVE_SKIPS cap (reset on any success) guards against an infinite skip loop on a systemic failure, re-raising so it still surfaces. Adds tests for both behaviours (skip-and-continue, and abort-after-cap). (cherry picked from commit e85b333cab00f1072f0bcdb80a2d9271d8f3c123) Signed-off-by: Arron Atchison <aatchison@thunderbird.net>
Signed-off-by: Arron Atchison <aatchison@thunderbird.net>
aatchison
force-pushed
the
upstream-skip-bugzilla-http-400
branch
from
aa551e5 to
1a416b4
Compare
With include_fields=_all, a large batch can return a response too big to be parsed (truncated JSON), which previously aborted the whole collection. On a JSONDecodeError the backend now halves the batch and retries; if a single bug is still unparseable at a batch size of one, it is skipped. The existing consecutive-skip cap also bounds this path. Broadens the changelog entry to cover both the HTTP 400 skip and the unparseable-response recovery. Signed-off-by: Arron Atchison <aatchison@thunderbird.net>
Author
|
Follow-up: #912 adds transient-5xx recovery (reduce batch + retry), stacked on this PR. Please review/merge this one first. |
jjmerchante
requested changes
Jun 15, 2026
jjmerchante
left a comment
jjmerchante
left a comment
Contributor
There was a problem hiding this comment.
Thank you for your proposal. Please see my comments below.
| raise | ||
| logger.warning("Skipping bugs page at offset %s after HTTP 400: %s", | ||
| offset, e.response.text) | ||
| offset += batch |
Contributor
There was a problem hiding this comment.
On an HTTP 400, offset += batch permanently skips an entire page, so up to ~499 perfectly good bugs in that window are silently dropped and never retried.
What about handling this similarly to JSONDecodeError by halving the batch size until the problematic bug is identified, and then continuing the collection?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A single bug can make the Bugzilla REST API return an HTTP 400 for a bug-list request. We hit this on bugzilla.mozilla.org, where one bug returns:
for
GET /rest/bug?...&include_fields=_all. BecauseBugzillaRESTBackend.__fetch_and_parse_bugs()requestsinclude_fields=_alland pages bylast_change_time, that 400 raises out of the fetch loop and aborts the entire collection. Since the failing bug stays in the incremental window, every subsequent run fails the same way — collection is frozen indefinitely (in our case ~3 months) behind one bad bug.Fix
Catch the
HTTPErrorin__fetch_and_parse_bugs: on a 400, log and skip that page (advance the offset) and continue, so one bad bug no longer wedges collection. AMAX_CONSECUTIVE_SKIPScap (reset on any success) guards against an infinite skip loop on a systemic failure and re-raises so it still surfaces. Other HTTP errors are re-raised unchanged.The collection cursor advances normally once healthy bugs are yielded.
Tests
Added two tests (both written test-first):
test_fetch_skips_bug_page_on_http_400— first page 400s; asserts the remaining bugs are still collected.test_fetch_aborts_after_too_many_consecutive_400s— persistent 400s abort (bounded) rather than loop forever.tests/test_bugzillarest.pypasses (27 tests);flake8clean. Changelog entry added underreleases/unreleased/.