[bpf-ci-bot] sched_ext CI tests fail deterministically due to argument passing mismatch

[kernel-patches-review-bot]

Summary

The sched_ext CI test job fails with Unexpected argument 'sched_ext'. Use -t to filter tests. on every PR where sched_ext testing is enabled. Commit ff1befcb1683 added strict argument validation to the sched_ext runner binary, but the CI script run-scx-selftests.sh still passes positional arguments via $@ which the runner now rejects.

Failure Details

• Test / Component: sched_ext runner invocation in CI (run-scx-selftests.sh)
• Frequency: Every run where sched_ext tests are enabled (currently on aarch64 and x86_64 for PRs targeting branches where sched_ext is uncommented in matrix.py)
• Failure mode: Immediate exit with error code 1 — no tests are executed
• Affected architectures: aarch64, x86_64 (wherever sched_ext job is triggered)
• CI runs observed:
  • https://github.com/kernel-patches/bpf/actions/runs/24616282649 (bpf/tests: Exhaustive test coverage for signed division and modulo)
  • https://github.com/kernel-patches/bpf/actions/runs/24616278963 (s390/bpf: inline smp_processor_id and current_task)
  • https://github.com/kernel-patches/bpf/actions/runs/24591587355 (selftests/bpf: Trace bpf_local_storage_update)
  • https://github.com/kernel-patches/bpf/actions/runs/24587517155 (Obj relationship v3)

Root Cause Analysis

The CI dispatch chain works as follows:

1. run.sh sets TEST_RUNNERS=$(echo ${KERNEL_TEST} | tr -s ',' ' ') which yields sched_ext
2. The vmtest command runs: ${VMTEST_SCRIPT} ${TEST_RUNNERS} → run-scx-selftests.sh sched_ext
3. run-scx-selftests.sh line 13 executes: ./runner "$@" → ./runner sched_ext
4. runner.c:167-171 (added by commit ff1befcb1683) rejects positional arguments:

   if (optind < argc) {
       fprintf(stderr, "Unexpected argument '%s'. Use -t to filter tests.\n",
           argv[optind]);
       return 1;
   }

Before commit ff1befcb1683 (2026-04-08, "selftests/sched_ext: Improve runner error reporting for invalid arguments"), the runner binary silently ignored the extra argument and ran all tests. After the commit, it rejects any positional argument and exits immediately.

The run-bpf-selftests.sh script uses its positional arguments ($@) to select which test binary to run (e.g., test_progs, test_maps). In contrast, run-scx-selftests.sh only has one runner binary, so the argument is meaningless — it should not be forwarded.

Key files:

• github/libbpf/ci/run-vmtest/run-scx-selftests.sh:13 — passes $@ to runner
• tools/testing/selftests/sched_ext/runner.c:167-171 — rejects positional args
• github/libbpf/ci/run-vmtest/run.sh — constructs ${VMTEST_SCRIPT} ${TEST_RUNNERS}

Proposed Fix

Remove "$@" from the ./runner invocation in run-scx-selftests.sh. The test runner name argument is only relevant for run-bpf-selftests.sh which uses it to select between multiple test binaries; sched_ext has only one runner.

See: 0001-ci-run-vmtest-Do-not-pass-arguments-to-sched_ext-run.patch

Impact

When this issue triggers, the entire sched_ext CI job fails without running any tests, producing a red X on the PR. This is the sole failure in many runs (8+ PRs confirmed), causing patch submitters and reviewers to investigate a false negative. Currently mitigated by the fact that most base branches have sched_ext commented out in matrix.py, but any PR that merges with a branch where it's enabled will hit this.

References

• Commit that introduced strict arg validation: ff1befcb1683 ("selftests/sched_ext: Improve runner error reporting for invalid arguments")
• Existing sched_ext issues (separate bugs): [bpf-ci-bot] The sched_ext numa selftest is flaky #453, [bpf-ci-bot] sched_ext rt_stall test still flaky: measurement includes pre-sleep CPU time #458

[kernel-patches-review-bot]

0001-ci-run-vmtest-Do-not-pass-arguments-to-sched_ext-run.patch

From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: BPF CI Bot <bot+bpf-ci@kernel.org>
Date: Sun, 20 Apr 2026 12:00:00 +0000
Subject: [PATCH] ci/run-vmtest: Do not pass arguments to sched_ext runner

The sched_ext runner binary does not accept positional arguments.
Since commit ff1befcb1683 ("selftests/sched_ext: Improve runner error
reporting for invalid arguments"), passing one causes immediate exit
with: "Unexpected argument 'sched_ext'. Use -t to filter tests."

The CI framework passes the test runner name (e.g., "sched_ext") as a
positional argument to all test scripts via ${VMTEST_SCRIPT}
${TEST_RUNNERS}. For run-bpf-selftests.sh this selects which test
binary to execute, but run-scx-selftests.sh has only one binary and
should ignore the argument rather than forwarding it.

Generated-by: BPF CI Bot (claude-opus-4-6) <bot+bpf-ci@kernel.org>
---
 run-vmtest/run-scx-selftests.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/run-vmtest/run-scx-selftests.sh b/run-vmtest/run-scx-selftests.sh
index 1234567..abcdefg 100755
--- a/run-vmtest/run-scx-selftests.sh
+++ b/run-vmtest/run-scx-selftests.sh
@@ -10,7 +10,7 @@ cd "${SELFTESTS_DIR}"
 echo "Executing selftests/sched_ext/runner"
 echo "runner output is being written to runner.log"

-./runner "$@" 2>&1 > runner.log & wait
+./runner 2>&1 > runner.log & wait

 echo "runner finished, check results"
 echo "[...]"
--
2.53.0

[theihor]

Fixed by libbpf/ci#221