Replace e2e tests with mock HTTP server integration tests - #28
Draft
ofalvai wants to merge 1 commit into
Draft
Conversation
The e2e test suite relied on a decommissioned auth service to obtain BITRISEIO_ARTIFACT_PULL_TOKEN, and on hardcoded fixture builds from March 2026 that no longer exist. The new tests cover the same scenarios using a mock HTTP server, removing both external dependencies. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ofalvai
marked this pull request as ready for review
May 15, 2026 14:15
ofalvai
enabled auto-merge (squash)
May 15, 2026 14:21
ofalvai
marked this pull request as draft
May 19, 2026 11:39
auto-merge was automatically disabled
May 19, 2026 11:39
Pull request was converted to draft
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Checklist
step.ymlandREADME.mdis updated with the changes (if needed)Version
Requires a PATCH version update
Context
The nightly CI was failing because the e2e test suite depended on two things that are no longer available: a decommissioned auth service (
auth.services.bitrise.io) used to obtainBITRISEIO_ARTIFACT_PULL_TOKEN, and hardcoded fixture builds from March 2026 that have since been cleaned up. SinceBITRISEIO_ARTIFACT_PULL_TOKENis only injected by the platform in real pipeline builds — not in standalone e2e worker builds — there is no clean way to restore the old approach.Changes
e2e/bitrise.ymland thee2e/directory entirelystep/run_integration_test.gowith 5 integration tests that callRun()directly against a mock HTTP servertestify/require(was already ingo.modbut not imported by any existing test)Investigation details
Three approaches were considered:
A. Refresh fixture data + use a stored API token — update the hardcoded build/pipeline IDs and swap the auth service call for
BITRISE_API_TOKEN. Quick to implement but perpetuates the same maintenance burden: fixtures go stale on every build retention cycle, and the token format may differ from what the platform injects in production.B. Run e2e tests inside a real Bitrise pipeline — the step only receives
BITRISEIO_ARTIFACT_PULL_TOKENwhen running as part of an actual pipeline, so restructuring the CI to trigger a child pipeline would be the most faithful test environment. This requires significant changes to the centralized CI setup and is disproportionate for the coverage gained.C. Replace with mock HTTP server integration tests — exercise
Run()end-to-end with a localhttptest.Serverhandling both the Bitrise API routes and file downloads. No external dependencies, no fixture maintenance, archives created programmatically in the test.Decisions
Option C was chosen. The e2e tests were primarily validating infrastructure (auth token exchange, stale fixture data) rather than step logic. The unit test suite already covered individual components well; the only genuine gap was a full-stack
Run()test with real archive extraction. The new tests fill that gap without any ongoing maintenance cost.