chore: split queue and build timeouts, reattach on retry (#8780)

* ci(cloudbuild): split queue and build timeouts, reattach on retry

Unity Cloud Build concurrency settings now leave builds in the queue
for much longer before a builder picks them up. The previous single
GLOBAL_TIMEOUT counted queue + active build time together, so a long
queue could kill an otherwise healthy build, and nick-fields/retry
created a brand-new build that went to the back of the queue.

Changes:
- Replace GLOBAL_TIMEOUT with QUEUE_TIMEOUT (4h) and BUILD_TIMEOUT (3h).
  The build clock resets on the queue -> started transition so a long
  queue never eats into the active build window.
- Exit 99 on either timeout so nick-fields/retry kicks in (the previous
  sys.exit(1) was not retryable).
- On retry, reattach to the persisted build_info.json build if it is
  still in queue/active, instead of POSTing a fresh build.
- Adaptive polling: bump cadence to 120s after the status has been
  unchanged for 10 minutes, to reduce API noise while queued.
- Log queuedReason on each status change and append a phase-breakdown
  table to \$GITHUB_STEP_SUMMARY for visibility on queue pressure.
- Make cancel_build idempotent so the post-failure cancel step does not
  fail if Unity already finalised the build.
- Raise the wrapping nick-fields/retry timeout_minutes to 450 and the
  job timeout-minutes to 510 so the inner Python budgets are always
  the ones that fire first.
- Cancel Unity build also on step failure() so an outer step timeout
  no longer leaves a Unity-side build holding a slot.

* chore: preserve queue position on retry and trim verbose comments

Don't cancel on inner QUEUE_TIMEOUT — exit 99 to yield the runner while
the build keeps its Unity Cloud queue slot, so the next nick-fields/retry
attempt reattaches via try_resume_build instead of POSTing a fresh build
that goes to the back of the queue. BUILD_TIMEOUT still cancels.

Seed build_start from Unity's totalTimeInSeconds on reattach so the 3h
BUILD_TIMEOUT measures real wall-clock rather than restarting per attempt.

Add the missing sys.exit(0) after the --cancel branch so the cancel step
no longer falls through into the polling loop.

Trim per-line env comments and section banners from build.py and
build-unitycloud.yml per the project's "no comments by default" rule.

* chore: clean up build_info.json on terminal paths

Explicitly delete build_info.json when:
- run_poll_loop returns a terminal status (success/failure/canceled/unknown)
- BUILD_TIMEOUT cancels the build before exit 99
- --cancel mode finishes cancelling

Only the queue_timeout exit keeps the file, so the next retry attempt
can reattach to the still-queued build.

Also fix utils.delete_build_info to actually be idempotent: it was
printing the "ignoring delete" message and then still calling os.remove
on the missing file, which raises FileNotFoundError.

Per-attempt QUEUE_TIMEOUT (effective global cap = QUEUE_TIMEOUT *
max_attempts) is left as-is — that's the retry-as-patience-extension
design intent.

Author: aixaCode

.github/workflows/build-unitycloud.yml

@@ -412,7 +412,8 @@ jobs:
     runs-on: ubuntu-latest
     needs: prebuild
     if: needs.prebuild.outputs.should_build == 'true'
-    timeout-minutes: 360
+    # Safety ceiling around the 450m retry budget + surrounding steps.
+    timeout-minutes: 510
     strategy:
       fail-fast: false
       matrix:
@@ -437,21 +438,25 @@ jobs:
       - name: Execute Unity Cloud build
         uses: nick-fields/retry@v3
         with:
-          timeout_minutes: 180  # matches your GLOBAL_TIMEOUT = 10800s
+          # Wraps QUEUE_TIMEOUT (240m) + BUILD_TIMEOUT (180m) + buffer.
+          timeout_minutes: 450
           max_attempts: 2
           retry_on_exit_code: 99
           retry_wait_seconds: 30
           on_retry_command: |
-            echo "::warning::🔁 Unity Cloud Build retry triggered at $(date '+%Y-%m-%d %H:%M:%S')"
+            echo "::warning::🔁 Unity Cloud Build retry triggered at $(date '+%Y-%m-%d %H:%M:%S'). The next attempt will reattach to the persisted build if it is still in flight."
           command: |
             echo "🔧 Starting Unity Cloud Build attempt at $(date '+%Y-%m-%d %H:%M:%S')"
             python -u scripts/cloudbuild/build.py
         env:
           API_KEY: ${{ secrets.UNITY_CLOUD_API_KEY }}
           ORG_ID: ${{ secrets.UNITY_CLOUD_ORG_ID }}
           PROJECT_ID: ${{ secrets.UNITY_CLOUD_PROJECT_ID }}
-          POLL_TIME: 60  # Set the polling time in seconds
-          GLOBAL_TIMEOUT: 10800  # Set the global timeout in seconds (e.g., 3 hours)
+          POLL_TIME: 60
+          QUEUE_POLL_TIME: 120
+          STALE_POLL_THRESHOLD: 600
+          QUEUE_TIMEOUT: 14400
+          BUILD_TIMEOUT: 10800
           TARGET: t_${{ matrix.target }}
           BRANCH_NAME: ${{ github.head_ref || github.ref_name }}
           COMMIT_SHA: ${{ needs.prebuild.outputs.commit_sha }}
@@ -793,12 +798,12 @@ jobs:
           path: shader_compilation_report.log
           if-no-files-found: warn
 
-      # Will run on cancel or timeout only
+      # Also runs on failure() so an outer-step timeout doesn't leave a Unity-side build holding a slot.
       - name: Cancel Unity Cloud build
-        if: ${{ cancelled() }}
+        if: ${{ cancelled() || failure() }}
         env:
           API_KEY: ${{ secrets.UNITY_CLOUD_API_KEY }}
           ORG_ID: ${{ secrets.UNITY_CLOUD_ORG_ID }}
           PROJECT_ID: ${{ secrets.UNITY_CLOUD_PROJECT_ID }}
-        run: python -u scripts/cloudbuild/build.py --cancel
+        run: python -u scripts/cloudbuild/build.py --cancel || true
 # Test change