Skip to content

Add create table as select support for iceberg. - #13595

Merged
liurenjie1024 merged 12 commits into
NVIDIA:branch-25.12from
liurenjie1024:ray/13521
Oct 16, 2025
Merged

Add create table as select support for iceberg.#13595
liurenjie1024 merged 12 commits into
NVIDIA:branch-25.12from
liurenjie1024:ray/13521

Conversation

@liurenjie1024

Copy link
Copy Markdown
Collaborator

Fixes #13521 .

Description

Add create table as select support for iceberg.

Checklists

  • This PR has added documentation for new or modified features or behaviors.
  • This PR has added new tests or modified existing tests to cover new code paths.
    (Please explain in the PR description how the new code paths are tested, such as names of the new/existing tests that cover them.)
  • Performance testing has been performed and its results are added in the PR description. Or, an issue has been filed with a link in the PR description.

Copilot AI review requested due to automatic review settings October 10, 2025 10:09

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds support for Iceberg CREATE TABLE AS SELECT (CTAS) operations on GPU by extending the existing Iceberg provider infrastructure to handle AtomicCreateTableAsSelectExec operations.

Key changes:

  • Extended IcebergProvider interface with CTAS support methods
  • Added GPU tagging and conversion logic for CTAS operations
  • Implemented comprehensive test coverage for various CTAS scenarios

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated no comments.

Show a summary per file
File Description
ExternalSource.scala Added CTAS support to GPU tagging and conversion logic
IcebergProvider.scala Extended interface with CTAS-specific methods
iceberg_ctas_test.py Comprehensive test suite covering CTAS scenarios and fallback cases
GpuSparkWrite.scala Added CTAS-specific GPU tagging method with validation logic
package.scala Added catalog support utility function
IcebergProviderImpl.scala Implemented CTAS support methods with GPU conversion

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Signed-off-by: liurenjie1024 <liurenjie2008@gmail.com>

@res-life res-life left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@liurenjie1024

Copy link
Copy Markdown
Collaborator Author

build

@liurenjie1024
liurenjie1024 merged commit 6e6fb8f into NVIDIA:branch-25.12 Oct 16, 2025
60 checks passed
@sameerz sameerz added the feature request New feature or request label Oct 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature request New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEA] Support CTAS in for iceberg.

5 participants