Skip to content

[MINOR][cudf_polars] Validation summary skips query failures #23508

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
nirandaperera wants to merge 2 commits into
base: main
Choose a base branch
from
+11 −1
Open

[MINOR][cudf_polars] Validation summary skips query failures #23508

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b3e8519
minor validation
nirandaperera Aug 2, 2026
008da93
Merge branch 'main' of github.com:rapidsai/cudf into fix/validation-s…
nirandaperera Aug 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion python/cudf/cudf/pandas/_benchmarks/utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# SPDX-FileCopyrightText: Copyright (c) 2025-2026, NVIDIA CORPORATION & AFFILIATES.
# SPDX-FileCopyrightText: Copyright (c) 2025-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

"""Utility functions/classes for running the PDS-H and PDS-DS benchmarks."""
Expand Down Expand Up @@ -756,6 +756,11 @@ def run_pandas(
print( # noqa: T201
f"{len(validation_failures)} queries failed validation: {sorted(set(validation_failures))}"
)
elif query_failures:
print( # noqa: T201
"Validation was skipped for queries that failed to run: "
f"{sorted({q_id for q_id, _ in query_failures})}"
)
Comment on lines +759 to +763

@coderabbitai coderabbitai Bot Aug 2, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Report both validation failures and skipped queries.

Both branches use elif, so mixed failures produce an incomplete validation summary. Keep the non-zero exit code, but also list every affected query category.

  • python/cudf/cudf/pandas/_benchmarks/utils.py#L759-L763: print skipped query IDs even when validation_failures is non-empty, and show the success message only when both failure collections are empty.
  • python/cudf_polars/cudf_polars/streaming/benchmarks/utils.py#L1235-L1239: apply the same independent reporting logic.
📍 Affects 2 files
  • python/cudf/cudf/pandas/_benchmarks/utils.py#L759-L763 (this comment)
  • python/cudf_polars/cudf_polars/streaming/benchmarks/utils.py#L1235-L1239
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@python/cudf/cudf/pandas/_benchmarks/utils.py` around lines 759 - 763, The
validation summary currently uses mutually exclusive branches, hiding skipped
queries when validation failures exist. In both
`python/cudf/cudf/pandas/_benchmarks/utils.py` lines 759-763 and
`python/cudf_polars/cudf_polars/streaming/benchmarks/utils.py` lines 1235-1239,
make failure and skipped-query reporting independent, preserve the non-zero exit
code, and emit the success message only when both `validation_failures` and
`query_failures` are empty.

else:
print("All validated queries passed.") # noqa: T201

Expand Down
5 changes: 5 additions & 0 deletions python/cudf_polars/cudf_polars/streaming/benchmarks/utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1232,6 +1232,11 @@ def _finalize_benchmark_run(
f"{len(validation_failures)} queries failed validation: "
f"{sorted(set(validation_failures))}"
)
elif query_failures:
print(
"⚠️ Validation was skipped for queries that failed to run: "
f"{sorted({q_id for q_id, _ in query_failures})}"
)
else:
print("✅ All validated queries passed.")
args.output.write(json.dumps(run_config.serialize(engine=engine)))
Expand Down
Loading