USWDS-Site: Speed up CircleCI build

[ethangardner]

Summary

Improved CircleCI build performance by implementing true parallel accessibility testing and optimizing dependency caching.

Related issue

Closes #3275

Problem statement

The CircleCI build had two inefficiencies:

1. Duplicate a11y testing: Jobs were configured with parallelism: 2, but both containers ran the exact same full sitemap through pa11y-ci, resulting in both containers doing scans across the entire sitemap instead of dividing the work in half..

2. Suboptimal caching: Dependency caches lacked fallback keys and didn't follow CircleCI best practices, leading to full reinstalls when lockfiles changed and caching the wrong paths for npm (caching node_modules instead of ~/.npm).

Solution

Implemented true parallelism and improved caching:

1. Created sitemap sharding script (config/shard-sitemap.mjs): Fetches the sitemap at test-time, validates and filters URLs, then splits them round-robin across CircleCI containers. This ensures each of the 2 containers tests a distinct half of the site.

2. Updated CircleCI config:

   • Replaced pa11y-ci:sitemap calls with pa11y-ci:shard in desktop/mobile a11y jobs
   • Added --skip-initial-build flag to jekyll serve since assets are pre-built
   • Added curl readiness check before running pa11y to ensure server is ready

3. Improved dependency caching:

   • Added fallback cache keys for both gem and npm caches
   • Changed npm cache from node_modules to ~/.npm (npm's download store)
   • Added bundle clean --force to remove stale gems after partial cache restores
   • Added version prefix (v1-) for easy cache invalidation
   • Removed redundant cache restores in build job (deps come from workspace)

Major changes

• New file: config/shard-sitemap.mjs — sitemap URL sharding script with origin validation
• Package.json: Added pa11y-ci:shard and pa11y-ci:shard-mobile npm scripts
• CircleCI config: Complete rework of caching strategy and a11y test execution

Testing and review

To verify these changes:

1. Parallelism: Check CircleCI build logs — each container should report testing ~half the URLs (e.g., "Shard 1/2: 150/300 URLs" and "Shard 2/2: 150/300 URLs")
2. Caching: Watch for cache restore fallbacks in CI logs when lockfiles change
3. Build time: Compare total CI time before/after — a11y jobs should complete in roughly half the time

The a11y tests should pass with identical coverage to the previous approach, just faster. Build time went from 14-16 minutes down to 8-9 in my testing.

[jonathanbobel]

This is great 🚀