[STEP-2494] Migrate to Unified CI

[lpusok]

Checklist

• I've read and followed the Contribution Guidelines
• step.yml and README.md is updated with the changes (if needed)

Version

No version update — CI configuration and maintenance tooling only, the Step itself is unchanged apart from lowercased error strings.

Context

Migrates this repo to Unified CI, the way ios#53 did. The legacy project 9ed7723d7d1575df was the last custom CI for this Step.

The Terraform switch (bitrise-github-steps#203) is already merged and applied, so this PR reports the ci/bitrise/48fa8fbee698622c/pr check and the legacy one is gone.

Depends on #118 (merged) for the refreshed device list — without it the new catalog check fails.

Changes

CI structure

• bitrise.yml: the monolithic test workflow is replaced with check / e2e / sample. check is included from steps-check/steps.bitrise.yml instead of the legacy git::steps-check.git Step.
• e2e/bitrise.yml (new): test_flaky_and_quarantined_tests (the migrated legacy test) and test_device_catalog_up_to_date.
• main.go: 10 error strings lowercased for staticcheck ST1005, which the shared check enforces.
• vendor/: drops the vendored go-utils/sliceutil, left behind when its last usage was removed in remove sliceutils #112.

The maintenance workflow now applies the catalog change instead of reporting it

• go test ./... -update in maintenance/ rewrites maintenance/testdata/device_list.txt and the device table in step.yml's test_devices description; the workflow then regenerates the README, leaving a reviewable diff in the working tree. Without -update the test only reports, which is what the e2e case does.
• The expected list moves from a 169-line const in the test to testdata/device_list.txt, so applying an update is a file write rather than a test rewriting its own source.
• step.yml is edited by locating the description with a YAML parse and rewriting only those lines. Re-encoding the document would reformat the whole file and fight yamlfmt.
• The catalog check is tooling rather than part of the Step, so maintenance/ is its own Go module. gopkg.in/yaml.v3 stays out of the Step's go.mod and out of vendor/, and a nested module is invisible to the Step's go test ./..., so the check no longer needs a build tag to stay out of check.

Investigation details

The legacy e2e never built its own APKs. The app-side primary workflow downloaded prebuilt APKs from two app-level uploaded files ($BITRISEIO_APP_URL, $BITRISEIO_TEST_URL). Nothing in the repo described where they came from — the filenames suggest they were built from googlesamples/android-testing years ago — and the shared Unified CI project does not have them. The e2e now builds android-multiple-test-results-sample with android-build-for-ui-testing, reusing the setup from that Step's own e2e (set-java-version → simple-git-clone → install-missing-android-tools → build). That also means the e2e covers the android-build-for-ui-testing → VDT handoff every real workflow has.

The catalog check had never run. The app-side maintenance workflow that would have invoked it has zero builds, here and in the iOS project. It is now an e2e case, and it caught real drift on its first ever execution — which is what #118 fixed.

Verification. Green on all 17 selected stacks. On the latest run linux-docker-android-22.04 ran the full device test (~4.5 min) and the other 16 stacks ~20-45s each.

Decisions

Both e2e cases are gated to one stack. Their results come from Firebase Test Lab, not from the stack, so running them everywhere only spends device minutes and gcloud sign-ins on the same assertions. test_flaky_and_quarantined_tests is a bitrise-run wrapper behind run_if: {{ or (enveq "IS_LATEST_STACK_UBUNTU" "true") (not .IsCI) }}, so it always runs locally. Ubuntu is also the only selected stack where building an Android app is cheap.

The catalog check was initially left ungated on the reasoning that it needs no devices or app build. That was wrong, and a build proved it: it made 17 gcloud sign-ins per build, and one macOS worker failing to reach oauth2.googleapis.com took the whole build down. It is now gated too.

A build tag would not have been enough for the maintenance module. check runs go test ./... without secrets and the catalog test needs gcloud credentials. A separate module keeps it out of that run and out of vendor/. Skipping at runtime on an empty $SERVICE_ACCOUNT_JSON was the other option, and it lets the package report a pass without checking anything.

The quarantine assertion is stricter than the legacy one. It derives the result XML filename from TEST_DEVICES instead of hardcoding MediumPhone.arm-33-en-portrait..., and it additionally asserts that the identically named method in the sibling test class did run — otherwise an exclusion that was too broad reads as a successful quarantine. The greps use -F: the class name's dots would otherwise match any character, and a parameterized test's [0: 4] suffix would be read as a bracket expression, which can make a failed quarantine report success.

Review follow-ups

A review of this PR found that the step.yml rewriter took the block scalar's indentation from the first content line whatever it was. A leading blank line — legal YAML — made it the empty string, which every line has as a prefix, so the rewrite replaced everything after the description with the description. Nothing exercised the path, because CI runs the check without -update, so the first real drift would have been its first execution.

Fixed in 0698049: the indentation comes from the first non-blank line and an empty one is refused, the range stops at the last indented line instead of swallowing the blank lines after the block, and updateStepYML re-reads and re-parses the file afterwards and fails if the description does not hold what was intended — so a bad splice cannot reach a commit even unattended. maintenance/rewrite_test.go covers both functions, needs no credentials, and therefore runs on every execution of the catalog check rather than only when the catalog drifts.

Known follow-up

IS_LATEST_STACK_UBUNTU currently resolves to linux-docker-android-22.04, a frozen stack with a removal date of 2027-04-08. When it is removed, nothing carries the marker and both gated workflows would skip on every stack while still reporting success. Fixed controller-side in tooling-centralized-ci-pipeline-controller#111; no change needed here.

[lpusok]

Verified end to end before review, since the legacy check on this PR cannot pass (the old project's primary workflow runs bitrise run test, and test is gone — that resolves with bitrise-io/bitrise-github-steps#203).

I pushed a throwaway branch combining this PR with #118 and triggered Unified CI manually against it: build 339900 — green on all 17 stacks.

The stack gate behaves as intended:

• linux-docker-android-22.04 (IS_LATEST_STACK_UBUNTU=true) ran the full device test in 4m37s — Set Java version → Simple Git Clone → Install missing Android SDK components → Android Build for UI Testing → Virtual Device Testing for Android (1.7 min) → both output assertions → deploy.
• the other 16 stacks ran only test_device_catalog_up_to_date, 25–50s each.

An earlier run on this branch alone is also worth noting: test_device_catalog_up_to_date failed there and printed a fresh device table byte-identical to the one in #118. The check earned its keep on its first ever execution — it had never run before, because the app-side maintenance workflow that would have invoked it has zero builds.

One finding for a follow-up, not a blocker: the controller assigns IS_LATEST_STACK_UBUNTU=true to linux-docker-android-22.04, which is a frozen stack — not ubuntu-noble-24.04-bitrise-2025-android or ubuntu-latest-stable. It works today, but "latest Ubuntu" pointing at a frozen Docker image is surprising and probably worth fixing in the controller rather than working around here.

Merge order: #118 → this → bitrise-io/bitrise-github-steps#203.

[ofalvai]

Any reason this automation is written as a test and not as a plain Go file with a main function? Not a blocker, but might be a quick and easy refactor.

[ofalvai]

Is that really an issue? We run ymlfmt + the Autofix CI step. I'd rather let it generate an imperfectly formatted YML instead of all this complex code.

[ofalvai]

Do we really need to test this? The maintenance workflow is run manually, and the result can be visually observed before opening a PR. This is also just an automation, it's not the production step code.

[ofalvai]

I think this comment should be next to the run_if expression