Proposal: Dedicated staging branch to test code

[tihuang02]

Title: Propose moving to a dedicated, single-branch staging deployment model

Description:

Current State

Currently, any branch containing the string staging is automatically deployed to the Google project cal-itp-data-infra-staging.

Problem

This approach has several significant drawbacks:

• Deployment Visibility: It is difficult to determine exactly what is currently deployed in staging. Because multiple "staging" branches exist, identifying the "winning" version requires manually parsing complex GitHub Actions logs.
• Testing Collisions: Developers often branch from main to create new staging tests. This leads to accidental overwrites of existing tests and disrupts ongoing work.
• Lack of Coordination: There is currently no reliable mechanism to communicate what is being tested, who is testing it, or when a test is complete. We rely on manual coordination, which is often neglected or assumptions lead to broken environments.

Proposed Solution

I propose migrating to a single, dedicated staging branch (tentatively named officialStaging) that acts as the sole target for deployments to cal-itp-data-infra-staging.

Benefits

• Enforced Communication: A dedicated branch serves as a central pivot point. Developers can check the branch status to see if other features are currently being tested, identify potential conflicts, and coordinate deployment windows.
• Conflict Detection: Utilizing Pull Requests into the officialStaging branch will allow team members to review changes and double-check for conflicts before deployment.
• Predictability: We can move from a state of accidental overwrites to an explicit workflow where features are merged, tested, and subsequently rolled back or promoted to main.

Notes

This mainly is for airflow DAGs and gtfs-rt archiver. For infra change, we should remain raising PR with changes in iac/cal-ipt-data-infra-staging against main branch because we want to keep the code and the reality in sync.

I would like to discuss this proposal with the team to understand any potential blockers. If we agree on this path, the next step is to raise a PR to change the github actions to deploy only offcialStaging to cal-itp-data-infra-staging.