[MINOR][cudf_polars] Validation summary skips query failures - #23508
Open
nirandaperera wants to merge 2 commits into
Open
[MINOR][cudf_polars] Validation summary skips query failures#23508nirandaperera wants to merge 2 commits into
nirandaperera wants to merge 2 commits into
Conversation
Signed-off-by: niranda perera <niranda.perera@gmail.com>
…ummary-skipped-on-query-failure
📝 WalkthroughSummary by CodeRabbit
WalkthroughChangesBenchmark validation reporting
Estimated code review effort: 1 (Trivial) | ~5 minutes Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@python/cudf/cudf/pandas/_benchmarks/utils.py`:
- Around line 759-763: The validation summary currently uses mutually exclusive
branches, hiding skipped queries when validation failures exist. In both
`python/cudf/cudf/pandas/_benchmarks/utils.py` lines 759-763 and
`python/cudf_polars/cudf_polars/streaming/benchmarks/utils.py` lines 1235-1239,
make failure and skipped-query reporting independent, preserve the non-zero exit
code, and emit the success message only when both `validation_failures` and
`query_failures` are empty.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 428166eb-4614-4324-b17d-3f08d0d63ed6
📒 Files selected for processing (2)
python/cudf/cudf/pandas/_benchmarks/utils.pypython/cudf_polars/cudf_polars/streaming/benchmarks/utils.py
Comment on lines
+759
to
+763
| elif query_failures: | ||
| print( # noqa: T201 | ||
| "Validation was skipped for queries that failed to run: " | ||
| f"{sorted({q_id for q_id, _ in query_failures})}" | ||
| ) |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Report both validation failures and skipped queries.
Both branches use elif, so mixed failures produce an incomplete validation summary. Keep the non-zero exit code, but also list every affected query category.
python/cudf/cudf/pandas/_benchmarks/utils.py#L759-L763: print skipped query IDs even whenvalidation_failuresis non-empty, and show the success message only when both failure collections are empty.python/cudf_polars/cudf_polars/streaming/benchmarks/utils.py#L1235-L1239: apply the same independent reporting logic.
📍 Affects 2 files
python/cudf/cudf/pandas/_benchmarks/utils.py#L759-L763(this comment)python/cudf_polars/cudf_polars/streaming/benchmarks/utils.py#L1235-L1239
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@python/cudf/cudf/pandas/_benchmarks/utils.py` around lines 759 - 763, The
validation summary currently uses mutually exclusive branches, hiding skipped
queries when validation failures exist. In both
`python/cudf/cudf/pandas/_benchmarks/utils.py` lines 759-763 and
`python/cudf_polars/cudf_polars/streaming/benchmarks/utils.py` lines 1235-1239,
make failure and skipped-query reporting independent, preserve the non-zero exit
code, and emit the success message only when both `validation_failures` and
`query_failures` are empty.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
When there are query failures, the validation summary shows,
This is technically correct, but masks the fact that some queries failed. Need to inspect the json to find out what really happened. This is misleading.
This PR, fixes the reporting string when there are failed queries.
Checklist