Upgrade to Beam 2.74.0

[Abacn]

No description provided.

[gemini-code-assist]

Note

The number of changes in this pull request is too large for Gemini Code Assist to generate a summary.

[codecov]

Codecov Report

❌ Patch coverage is 0% with 21 lines in your changes missing coverage. Please review.
✅ Project coverage is 55.51%. Comparing base (1570c07) to head (515ebaf).
⚠️ Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
.../java/org/apache/beam/it/gcp/TemplateTestBase.java	0.00%	21 Missing ⚠️

❌ Your patch check has failed because the patch coverage (0.00%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@              Coverage Diff              @@
##               main    #3850       +/-   ##
=============================================
+ Coverage     34.03%   55.51%   +21.48%     
- Complexity      501     7024     +6523     
=============================================
  Files           215     1103      +888     
  Lines         13057    67590    +54533     
  Branches       1288     7583     +6295     
=============================================
+ Hits           4444    37525    +33081     
- Misses         8257    27648    +19391     
- Partials        356     2417     +2061     

Components	Coverage Δ
spanner-templates	88.31% <ø> (∅)
spanner-import-export	68.52% <ø> (∅)
spanner-live-forward-migration	90.21% <ø> (∅)
spanner-live-reverse-replication	83.96% <ø> (∅)
spanner-bulk-migration	92.61% <ø> (∅)
gcs-spanner-dv	88.99% <ø> (∅)

Files with missing lines	Coverage Δ
.../java/org/apache/beam/it/gcp/TemplateTestBase.java	0.00% <0.00%> (ø)

... and 913 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Abacn]

failed with google pypi mirror missing dep

Looking in indexes: https://us-python.pkg.dev/artifact-foundry-prod/ah-3p-staging-python/simple/
Requirement already satisfied: aiofiles==25.1.0 in /usr/local/lib/python3.11/site-packages (from -r requirements_all.txt (line 30)) (25.1.0)
ERROR: Could not find a version that satisfies the requirement aiohappyeyeballs==2.6.2 (from versions: none)

going to wait for some time and retrigger

Internal record says

Soaktime verification: Dependency expected to be available on 5/27/2026

[Abacn]

v1/BigtableToAvro test failed with

Failed to list all clusters, some zones were unavailable: [asia-southeast2-b, asia-southeast2-a]" type="com.google.cloud.bigtable.admin.v2.models.PartialListClustersException"><![CDATA[com.google.cloud.bigtable.admin.v2.models.PartialListClustersException: Failed to list all clusters, some zones were unavailable: [asia-southeast2-b, asia-southeast2-a]
	...
		at com.google.cloud.bigtable.admin.v2.BigtableInstanceAdminClient.listClusters(BigtableInstanceAdminClient.java:686)
		at org.apache.beam.it.gcp.bigtable.BigtableResourceManager.getClusters(BigtableResourceManager.java:566)
		at org.apache.beam.it.gcp.bigtable.BigtableResourceManager.getClusterNames(BigtableResourceManager.java:556)

testIcebergToPostgres failed with

ValueError: Error applying transform "ReadFromIceberg" at line 5: org.apache.iceberg.exceptions.ForbiddenException: Forbidden: Not authorized to make this request. Permission 'biglake.catalogs.get' denied on project 'cloud-teleport-testing' (or it may not exist).

Both errors weren't present during RC3 validation. Both looks like GCP service change since them

=======

Rerun passed. Seems like service hiccup

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request primarily focuses on upgrading the Apache Beam version to 2.74.0 and synchronizing the associated Python dependencies. Additionally, it improves the robustness of the template staging process in the integration test base by replacing direct runtime execution with a more controlled ProcessBuilder approach, ensuring better error handling and logging.

Highlights

• Apache Beam Upgrade: Upgraded the Apache Beam version from 2.73.0 to 2.74.0 across the project.
• Process Execution Improvement: Refactored template staging logic in TemplateTestBase.java to use ProcessBuilder and a dedicated thread for reading process output, improving reliability and error logging.
• Dependency Updates: Updated various Python dependencies, including aiohttp, google-api-core, google-cloud-aiplatform, google-cloud-bigquery-storage, google-cloud-build, google-cloud-datastore, google-cloud-dlp, google-cloud-monitoring, google-cloud-pubsub, google-cloud-secret-manager, google-cloud-spanner, google-cloud-storage, google-genai, google-resumable-media, idna, and pytest.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[Abacn]

R: @aIbrahiim @jrmccluskey

[gemini-code-assist]

Code Review

This pull request upgrades Apache Beam to version 2.74.0 and updates various Python package dependencies across multiple requirements files. It also refactors the process execution logic in TemplateTestBase.java to use ProcessBuilder. Feedback on this change suggests simplifying the process stream reading by executing it directly in the main thread, which avoids the unnecessary complexity of spawning a background thread and using a synchronized list.

[Abacn]

R: @derrickaw