Skip to content

Cut the longest CI job from about seven minutes to about two - #217

Open
MarkusPaulsen wants to merge 1 commit into
mainfrom
chore/architecture-tests-fork-reuse
Open

Cut the longest CI job from about seven minutes to about two#217
MarkusPaulsen wants to merge 1 commit into
mainfrom
chore/architecture-tests-fork-reuse

Conversation

@MarkusPaulsen

Copy link
Copy Markdown
Collaborator

Summary

The architecture unit tests started a fresh JVM for each of their 19 classes, which cost about 15 seconds per class against 105 seconds of actual testing. They now share one fork, as the other two unit-test profiles already do, under a stated heap ceiling.

Linked issues

No linked issues

1. Problem

unit-architecture-tests was the only test profile that did not override surefire-reuse-forks. It therefore inherited the false that the coverage profile sets, and Surefire started a new JVM for every test class.

Measured from a CI log: the surefire:test execution ran 404 seconds while the 19 classes together report 105 seconds of test time, and the gaps between classes are about 15.5 seconds each. That is the JVM start, the agent attachment and the class loading, paid nineteen times.

Nothing is broken. This is the slowest job of the workflow and the reason the pipeline takes about eleven and a half minutes.

2. Improvement from the user's perspective

No Improvement from the user's perspective

3. Improvement from the maintainer's perspective

The step should drop from about 407 to about 130 seconds, and with it the critical path of the whole workflow from roughly eleven and a half to roughly seven minutes. A maintainer waits less for a pull request, and the shared runner pool is occupied for less time.

The heap becomes a property so that a profile can change it without repeating the agent and bootclasspath arguments of the whole argLine.

4. Testing manual

Prerequisites

  1. A checkout of this branch and a JDK. The measurements below were taken on JDK 25 locally; CI uses JDK 21.

Steps

Not reproducible from an exercise. This changes the build only.

  1. Run mvn test -Punit-architecture-tests,coverage and note the number of classes, the number of tests and the wall-clock time.
  2. Run the same command on main and compare.
  3. Open the "Unit Tests" job of the CI run for this pull request and read the duration of the "Architecture unit tests" step.

Expected result

Step 1 reports 19 classes and 188 tests, all passing, in roughly one minute. Step 2 reports the same 19 classes and 188 tests in roughly four and a half minutes. Step 3 shows a step duration well below the 407 seconds on main.

Eight runs were made while preparing this change, five of them at the new ceiling; all reported 19 classes and 188 tests with no failures, and peak resident size of the reused fork stayed between 1.76 and 2.20 GB.

Negative case (what must still be rejected)

No test may be skipped or silently dropped. If the run reports fewer than 19 classes or fewer than 188 tests, the change has traded coverage for speed and must be reverted rather than accepted.

Sharing one fork is the risk this change takes. The comment in pom.xml says so and names this line as the first thing to revert if the profile ever turns flaky.

Modes exercised

No mode-specific behaviour changed.

  • ArchUnit + AspectJ
  • ArchUnit + instrumentation
  • WALA + AspectJ
  • WALA + instrumentation

5. Test case coverage regarding this PR

No production Java code changed

Breaking changes and migration

No breaking changes or migration.

Checklist

  • CI is green, or every remaining failure is explained above.
  • No secrets, tokens or absolute local paths are contained in the diff.

Review progress

  • Code review
  • Manual test

unit-architecture-tests was the only test profile that did not override
surefire-reuse-forks, so it inherited the false the coverage profile sets and started a
fresh JVM for each of its 19 classes. Measured in CI: 404 s in the surefire plugin
against 105 s of actual test time, the difference being roughly 15.5 s of JVM start per
fork.

That WALA state survives a shared fork is not assumed. The integration matrix has run
the WALA modes with -Dsurefire-reuse-forks=true all along, and this was verified here:
19 classes and 188 tests green, twice.

One long-lived fork needs a stated heap ceiling rather than the 6g default, since that
default exists for the opposite case. At 6g the reused fork peaked at 3.12 and 3.95 GB
across two runs; at 3g, three runs peaked at 1.76, 2.20 and 1.83 GB with no change in
runtime. The heap becomes a property so a profile can set it without duplicating the
agent and bootclasspath arguments of the whole argLine.
@MarkusPaulsen
MarkusPaulsen requested a review from a team August 26, 2026 15:13
@MarkusPaulsen
MarkusPaulsen requested review from a team and krusche as code owners August 26, 2026 15:13
@coderabbitai

coderabbitai Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: ebeb7dcd-fa11-4e6a-b74f-d43c774abd47


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant