Skip to content

Raci/ci e2e setup prewarm#28667

Draft
racitores wants to merge 9 commits intomainfrom
raci/ci-e2e-setup-prewarm
Draft

Raci/ci e2e setup prewarm#28667
racitores wants to merge 9 commits intomainfrom
raci/ci-e2e-setup-prewarm

Conversation

@racitores
Copy link
Copy Markdown
Contributor

Description

Summary

  • Adds a setup-env-android / setup-env-ios job to both regression workflows that runs before all shards start
  • This pre-warm job calls setup-e2e-env (without AVD/simulator or keystore) once, populating the Yarn, Foundry, Bundler, and CocoaPods caches
  • All shard jobs gain a needs: [..., setup-env-{platform}] dependency so they always hit a warm cache instead of each runner independently downloading and installing the same dependencies
  • For iOS, the detox build-framework-cache step is also moved into the pre-warm job so it runs exactly once across the whole workflow instead of being duplicated per shard

Motivation

Each shard in the regression suite was running the full setup-e2e-env action independently — meaning Yarn install, Foundry, Bundler, CocoaPods, and the Detox framework build were all duplicated N times (up to 4×) in parallel. Since GitHub Actions cache is shared across jobs in the same workflow run, a single pre-warm job can fill those caches before shards start, making each shard's setup near-instant on cache restore.

What's not changed

Smoke test workflows are intentionally left untouched. Their per-job if: conditions mean an unconditional pre-warm job would waste a full runner spin-up whenever only a subset of suites is selected.

Test plan

  • Trigger the Android E2E Regression workflow manually and verify setup-env-android completes before shards start
  • Confirm shards show cache hits for Yarn and Foundry in their setup step logs
  • Trigger the iOS E2E Regression workflow manually and verify setup-env-ios completes the Detox framework build once
  • Confirm iOS shards skip the detox build-framework-cache step (it no longer exists in run-e2e-workflow.yml for regression) and hit cache on CocoaPods/Bundler
  • Verify no regression in test results compared to a previous run

Changelog

CHANGELOG entry:

Related issues

Fixes:

Manual testing steps

Feature: my feature name

  Scenario: user [verb for user action]
    Given [describe expected initial app state]

    When user [verb for user action]
    Then [describe expected outcome]

Screenshots/Recordings

Before

After

Pre-merge author checklist

Pre-merge reviewer checklist

  • I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
  • I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

.github/workflows/** and .github/scripts/** were in the 'shared' filter,
causing any CI YAML edit to trigger full Android + iOS builds (~20 min).

CI files don't affect native builds. Moved them fully into 'ignore' so
workflow-only changes trigger the lightweight provide-artifact path (~2-3 min)
instead of a full rebuild.
E2E test specs run *against* the built app — they don't affect the native
binary. Moving tests/** from shared to ignore means adding/editing test
files won't force a full Android/iOS rebuild.

Same for .github/** (workflows, scripts): CI infrastructure changes don't
affect what the app does natively.
…ct names

Three fixes:
1. Filter by status=completed instead of status=success when searching for
   reusable artifacts — a run where tests failed but the build succeeded still
   has valid APK/app artifacts we can reuse.

2. iOS artifact is named main-qa-MetaMask.app (not main-e2e-MetaMask.app)
   because build-ios-e2e.yml defaults metamask_environment to 'qa'.

3. e2e-smoke-tests-ios was incorrectly running when ios-tests-ready was
   skipped — is true even for skipped jobs. Added explicit
   needs.ios-tests-ready.result == 'success' check.
…rt jobs

dorny/test-reporter requires checks:write to post GitHub Check Run results.
Without it the Report step fails even though fail-on-error:false is set,
because the action itself errors on the API call, not just on test failures.
The report job was starting at 12:22:25 and finishing at 12:22:44 (19s),
while ci-sanity-check-android didn't finish until 12:28:29 (6 min).

Since ci-sanity-check was not in the report job's 'needs', the report ran
before the JUnit XML artifact was uploaded — dorny/test-reporter found
zero files and failed.

Fix: add ci-sanity-check-android/ios to the needs list so the report
waits for the test to finish before trying to collect its results.

Also reverts the unnecessary continue-on-error workaround.
@racitores racitores added the no-changelog no-changelog Indicates no external facing user changes, therefore no changelog documentation needed label Apr 10, 2026
@github-actions
Copy link
Copy Markdown
Contributor

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

@metamaskbot metamaskbot added the team-qa QA team label Apr 10, 2026
@racitores racitores added the skip-smart-e2e-selection Skip Smart E2E selection, i.e. select all E2E tests to run label Apr 10, 2026
@racitores racitores force-pushed the raci/ci-e2e-setup-prewarm branch from 2e4769a to bfdb887 Compare April 10, 2026 15:18
@github-actions
Copy link
Copy Markdown
Contributor

🔍 Smart E2E Test Selection

⏭️ Smart E2E selection skipped - skip-smart-e2e-selection label found

All E2E tests pre-selected.

View GitHub Actions results

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-changelog no-changelog Indicates no external facing user changes, therefore no changelog documentation needed size-L skip-smart-e2e-selection Skip Smart E2E selection, i.e. select all E2E tests to run team-qa QA team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants