[auto-merge] bot-auto-merge-release/26.08 to main [skip ci] [bot] - #4931
Merged
Conversation
Contributes to NVIDIA/cudf-spark#15449. ### Description GPU ORC timestamp conversion needs to match Apache ORC's `java.util.TimeZone` semantics for writer and reader timezones. This change: - preserves ORC's `TimeZone.hasSameRules` behavior so same-rule writer/reader pairs skip cross-timezone conversion; - preserves fixed-offset and short timezone IDs by resolving `TimeZone` directly from `ZoneId`; - adds an ORC-compatible `convertFromUtc` path for timestamp columns; - exposes the reader timezone's first transition so the plugin can limit historical rule correction to values before that transition; - preserves Java/ORC overflow, negative timestamp, null-mask, sliced-column, CUDA stream, and memory-resource behavior. Without the direct `ZoneId` overload, values such as `+05:30` and short IDs such as `EST` were silently interpreted as GMT when checking `hasSameRules`. That could incorrectly skip conversion and produce a constant timezone offset error. ### Testing - `GpuTimeZoneDBTest`: `8 passed`, including `UTC` in both directions with `+05:30` and `EST`. - `pre-commit` checks passed; no C/C++/CUDA files were changed by the follow-up fixed-offset commit. - CUDA 12 container build: `BUILD SUCCESS`. - Dependent cuDF Spark A/B: - baseline reproduced all 7 fixed-seed failures; - fixed build passed the same 7 cases. --------- Signed-off-by: Chong Gao <chongg@nvidia.com> Co-authored-by: Chong Gao <res_life@163.com>
Signed-off-by: spark-rapids automation <70000568+nvauto@users.noreply.github.com>
Collaborator
Author
|
SUCCESS - auto-merge |
Contributor
Greptile SummaryThe PR updates the main branch from the 26.08 release line, advances the cuDF dependency pins, and extends ORC timezone compatibility.
Confidence Score: 5/5The PR appears safe to merge; no concrete changed-code defect remains after checking the new timezone conversion paths and coordinated dependency pins. The Java, JNI, and CUDA interfaces remain aligned, transition and DST lookup behavior matches the CPU reference semantics for supported timestamps, and the new tests exercise the relevant boundaries and optimized paths. Important Files Changed
Sequence DiagramsequenceDiagram
participant Caller as Java caller
participant Context as OrcTimezoneContext
participant JNI as GpuTimeZoneDBJni
participant GPU as CUDA timezone kernel
Caller->>Context: Build writer/reader timezone metadata
Context->>JNI: Convert timestamps with tables, offsets, and DST rules
alt Writer and reader rules match
JNI->>GPU: Apply ORC base adjustment only
else Rules differ
JNI->>GPU: Apply transition and recurring-DST lookup
end
GPU-->>Caller: TIMESTAMP_MICROSECONDS column
Reviews (1): Last reviewed commit: "Auto-merge use main versions" | Re-trigger Greptile |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
auto-merge triggered by github actions on
bot-auto-merge-release/26.08to create a PR keepingmainup-to-date. If this PR is unable to be merged due to conflicts, it will remain open until manually fix.