Skip to content

[py] run affected python tests per-browser on PR, full suite on schedule #17643

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
titusfortner wants to merge 1 commit into
base: trunk
Choose a base branch
from
+35 −14
Open

[py] run affected python tests per-browser on PR, full suite on schedule #17643

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 28 additions & 10 deletions .github/workflows/ci-python.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@ name: CI - Python

on:
workflow_call:
inputs:
targets:
required: true
type: string
workflow_dispatch:

permissions:
Expand All @@ -10,6 +14,7 @@ permissions:
jobs:
build:
name: Build
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'

@qodo-code-review qodo-code-review Bot Jun 6, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

1. Build job skips scheduled runs 📘 Rule violation ☼ Reliability

In .github/workflows/ci-python.yml, both the build and unit-tests jobs are gated on
github.event_name == 'schedule', but the workflow is only invoked via
workflow_call/workflow_dispatch from .github/workflows/ci.yml, so on scheduled CI the called
workflow’s event name is workflow_call and these jobs will be skipped. This can silently eliminate
intended scheduled build and unit-test coverage and reduce CI safety.
Agent Prompt 
## Issue description
In the reusable workflow `.github/workflows/ci-python.yml`, the `build` and `unit-tests` jobs are gated by `if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'`. When this workflow is executed via `workflow_call` from `.github/workflows/ci.yml` (including scheduled runs), `github.event_name` in the called workflow evaluates to `workflow_call`, so these jobs are skipped unintentionally.

## Issue Context
`.github/workflows/ci.yml` is the workflow that triggers on `schedule` and calls `ci-python.yml` via `uses: ./.github/workflows/ci-python.yml`. The called workflow currently has no `schedule` trigger and thus cannot ever have `github.event_name == 'schedule'`; the condition in the called workflow therefore does not reflect the caller’s trigger and prevents the scheduled build and unit-test coverage from running.

## Fix Focus Areas
- .github/workflows/ci-python.yml[15-18]
- .github/workflows/ci-python.yml[48-61]
- .github/workflows/ci.yml[3-12]
- .github/workflows/ci.yml[121-128]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

uses: ./.github/workflows/bazel.yml
with:
name: Build
Expand Down Expand Up @@ -42,7 +47,7 @@ jobs:

unit-tests:
name: Unit Tests
needs: build
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
uses: ./.github/workflows/bazel.yml
strategy:
fail-fast: false
Expand All @@ -57,18 +62,20 @@ jobs:

remote-tests:
name: Remote Tests
needs: build
uses: ./.github/workflows/bazel.yml
with:
name: Integration Tests Remote
needs-display: true
rerun-with-debug: true
run: |
bazel test --local_test_jobs 1 --flaky_test_attempts 3 //py:test-remote
run: >
./scripts/github-actions/bazel-test-if-targets.sh
--keep_going
--flaky_test_attempts 3
--local_test_jobs 1
$(bazel query "tests(//py:test-remote) intersect set(${{ inputs.targets || '//py/...' }})" --output=label 2>/dev/null)
Comment thread
titusfortner marked this conversation as resolved.

browser-tests:
name: Browser Tests
needs: build
uses: ./.github/workflows/bazel.yml
strategy:
fail-fast: false
Expand All @@ -83,12 +90,16 @@ jobs:
needs-display: true
os: ${{ matrix.os }}
rerun-with-debug: true
run: |
bazel test --local_test_jobs 1 --flaky_test_attempts 3 //py:test-${{ matrix.browser }}
run: >
./scripts/github-actions/bazel-test-if-targets.sh
--keep_going
--flaky_test_attempts 3
--local_test_jobs 1
--skip_incompatible_explicit_targets
$(bazel query "tests(//py:test-${{ matrix.browser }}) intersect set(${{ inputs.targets || '//py/...' }})" --output=label 2>/dev/null)

selenium-manager-tests:
name: Selenium Manager Tests
needs: build
uses: ./.github/workflows/bazel.yml
strategy:
fail-fast: false
Expand All @@ -99,5 +110,12 @@ jobs:
needs-display: true
os: ${{ matrix.os }}
rerun-with-debug: true
run: |
bazel test --local_test_jobs 1 --flaky_test_attempts 3 --pin_browsers=false --build_tests_only --test_tag_filters=se-manager //py/...
run: >
./scripts/github-actions/bazel-test-if-targets.sh
--local_test_jobs 1
--flaky_test_attempts 3
--pin_browsers=false
--build_tests_only
--skip_incompatible_explicit_targets
--test_tag_filters=se-manager
${{ inputs.targets || '//py/...' }}
8 changes: 5 additions & 3 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ jobs:
outputs:
java: ${{ steps.read.outputs.java }}
grid: ${{ steps.read.outputs.grid }}
py: ${{ steps.read.outputs.py }}
py_targets: ${{ steps.read.outputs.py_targets }}
rb_targets: ${{ steps.read.outputs.rb_targets }}
dotnet: ${{ steps.read.outputs.dotnet }}
rust: ${{ steps.read.outputs.rust }}
Expand Down Expand Up @@ -96,10 +96,10 @@ jobs:
}
check_binding "//java" "java"
check_binding "openqa/selenium/grid" "grid"
check_binding "//py" "py"
check_binding "//dotnet" "dotnet"
check_binding "//rust" "rust"
process_binding "//rb" "rb"
process_binding "//py" "py"
Comment thread
titusfortner marked this conversation as resolved.
dotnet:
name: .NET
needs: read-targets
Expand All @@ -122,7 +122,9 @@ jobs:
name: Python
needs: read-targets
uses: ./.github/workflows/ci-python.yml
if: needs.read-targets.outputs.py != ''
if: needs.read-targets.outputs.py_targets != ''
with:
targets: ${{ needs.read-targets.outputs.py_targets }}

ruby:
name: Ruby
Expand Down
3 changes: 2 additions & 1 deletion scripts/github-actions/rerun-failures.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,8 @@ fi
if [[ "$RUN_CMD" == *"/ci-build.sh"* ]]; then
base_cmd="bazel test --config=rbe-ci --build_tests_only --keep_going"
else
base_cmd=$(echo "$RUN_CMD" | sed 's| //[^ ]*||g')
# Strip the targets (literal `//...` or a trailing `$(...)` query) to get the base command.
base_cmd=$(echo "$RUN_CMD" | sed -E 's| \$\(.*$||; s| //[^ ]*||g')
fi
targets=$(tr '\n' ' ' < build/failures/_run1.txt)
echo "Rerunning tests: $base_cmd --test_env=SE_DEBUG=true --flaky_test_attempts=1 $targets"
Expand Down