[FEA] Add GPU support for NTILE [databricks]

[gerashegalov]

Fixes #8626.

Description

Adds GPU acceleration for the NTILE window function by reusing the existing window replacement machinery. The replacement computes row_number and a full-partition count, then applies Spark-compatible integer bucket arithmetic, including partitions smaller than the requested bucket count.

GpuNTileMeta is a named meta class colocated with the GPU expression, following the convention tracked by #10838.

Testing on Spark 3.3.0 / Scala 2.12:

• mvn -pl sql-plugin -DskipTests -DskipITs compile
• mvn -pl dist,integration_tests -am package -DskipTests -DskipITs
• Six targeted integration cases passed: partitioned and unpartitioned forced-multibatch inputs, plus 1, 2, 4, and 8 buckets across uneven and undersized partitions.

Performance was smoke-tested locally with the freshly built 26.10 jar using local[4]. The benchmark query was:

data = spark.range(10_000_003).select(
    (f.col('id') % 32).alias('p'), 'id')
window = Window.partitionBy('p').orderBy('id')
query = data.select(
    f.ntile(97).over(window).alias('bucket')) \
    .agg(f.sum('bucket').alias('checksum'))

Each timed run collected the checksum after setting spark.rapids.sql.enabled to false or true. AQE was disabled, spark.sql.shuffle.partitions was set to 32, and medians used three alternating CPU/GPU iterations after one warmup. CPU median was 1.076 s and GPU median was 0.538 s, for a 2.00x speedup.

Checklists

Documentation

• Updated for new or modified user-facing features or behaviors
• No user-facing change

Testing

• Added or modified tests to cover new code paths
• Covered by existing tests
• Not required

Performance

• Tests ran and results are added in the PR description
• Issue filed with a link in the PR description
• Not required

[greptile-apps]

Greptile Summary

Adds GPU acceleration for Spark’s NTILE window function.

• Registers NTILE in the GPU expression override registry.
• Rewrites NTILE as a full-partition count, row number, and integer bucket arithmetic.
• Adds CPU/GPU equivalence coverage for multi-batch execution, uneven partitions, and partitions smaller than the bucket count.
• Updates supported-operation documentation and generated compatibility metadata.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuOverrides.scala	Registers Spark NTILE as a window-only GPU expression with a literal integer bucket parameter.
sql-plugin/src/main/scala/com/nvidia/spark/rapids/window/GpuWindowExpression.scala	Adds NTILE metadata and a window replacement based on row number, partition count, and Spark-compatible bucket arithmetic.
integration_tests/src/main/python/window_function_test.py	Adds CPU/GPU equivalence coverage for NTILE across forced multi-batch and partition-size edge cases.
docs/additional-functionality/advanced_configs.md	Documents the generated per-expression configuration entry for NTILE.
docs/supported_ops.md	Lists NTILE as a supported integer window expression.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Spark NTILE expression] --> B[GpuNTileMeta]
  B --> C[GpuNTile replacement]
  C --> D[Full-partition row count]
  C --> E[Row number]
  D --> F[Integer bucket arithmetic]
  E --> F
  F --> G[NTILE bucket result]

Loading

_{Reviews (2): Last reviewed commit: "Update generated support metadata for NT..." | Re-trigger Greptile}

[gerashegalov]

build

[nartal1]

Let's run the premerge CI including databricks just to be sure.

[gerashegalov]

build

[nartal1]

LGTM

[firestarman]

Nit (non-blocking): Could we add @validate_execs_in_gpu_plan("GpuRunningWindowExec", "GpuCachedDoublePassWindowExec") to this forced-multibatch test and the corresponding single-part test below? That would ensure they keep exercising the intended memory-bounded physical plan, rather than only checking CPU/GPU result equivalence.

[jihoonson]

nit: this PR adds support for NTILE on GPU, and thus introduces a user-facing change. It also updates the document for the new NTILE support. The PR checklist about documentation should be fixed.