Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
83 commits
Select commit Hold shift + click to select a range
7b08f31
Use wildcard mark to import all Rapids test suites migrated from Apac…
wjxiz1992 Nov 17, 2025
063e6e8
Bump up version to 26.02 [skip ci] (#13795)
nvauto Nov 17, 2025
4a8826a
[AutoSparkUT]Enable Spark UT DateExpressionsSuite (#13762)
GaryShen2008 Nov 17, 2025
63325b0
[AutoSparkUT] Migrate DataFrameComplexTypeSuite tests to RAPIDS (#13767)
wjxiz1992 Nov 17, 2025
9eb27e9
Update dependency version JNI, private, hybrid to 26.02.0-SNAPSHOT [s…
nvauto Nov 18, 2025
7c52998
[auto-merge] release/25.12 to main [skip ci] [bot] (#13807)
nvauto Nov 18, 2025
56f1b5f
[AutoSparkUT] Migrate DataFrameNaFunctionsSuite tests to RAPIDS (#13777)
wjxiz1992 Nov 18, 2025
54a659d
[auto-merge] release/25.12 to main [skip ci] [bot] (#13815)
nvauto Nov 19, 2025
4784572
[AutoSparkUT] Combine three existing UT migration PRs into one to fac…
wjxiz1992 Nov 19, 2025
549c9c8
[AutoSparkUT]Enable several suites in Spark330's UT (#13802)
GaryShen2008 Nov 19, 2025
f289734
[auto-merge] release/25.12 to main [skip ci] [bot] (#13821)
nvauto Nov 19, 2025
e9fa2e0
[auto-merge] release/25.12 to main [skip ci] [bot] (#13828)
nvauto Nov 20, 2025
f1a45d2
[auto-merge] release/25.12 to main [skip ci] [bot] (#13833)
nvauto Nov 20, 2025
82cefb5
[auto-merge] release/25.12 to main [skip ci] [bot] (#13835)
nvauto Nov 20, 2025
2732cdf
[auto-merge] release/25.12 to main [skip ci] [bot] (#13836)
nvauto Nov 20, 2025
e847e02
[auto-merge] release/25.12 to main [skip ci] [bot] (#13837)
nvauto Nov 20, 2025
e9ef9e2
[auto-merge] release/25.12 to main [skip ci] [bot] (#13838)
nvauto Nov 20, 2025
3fdc715
[auto-merge] release/25.12 to main [skip ci] [bot] (#13839)
nvauto Nov 21, 2025
b078ce7
[auto-merge] release/25.12 to main [skip ci] [bot] (#13841)
nvauto Nov 21, 2025
f60690b
Support left-outer joins with no columns in either build or stream si…
firestarman Nov 21, 2025
fb631cb
Add the missing spark 357 version to GpuWriteFilesUnsupportedVersions…
jihoonson Nov 21, 2025
eb2bf0a
[auto-merge] release/25.12 to main [skip ci] [bot] (#13847)
nvauto Nov 21, 2025
9392e83
[auto-merge] release/25.12 to main [skip ci] [bot] (#13850)
nvauto Nov 22, 2025
1e7d0cf
[auto-merge] release/25.12 to main [skip ci] [bot] (#13858)
nvauto Nov 24, 2025
ae18394
[auto-merge] release/25.12 to main [skip ci] [bot] (#13862)
nvauto Nov 24, 2025
75424da
[auto-merge] release/25.12 to main [skip ci] [bot] (#13865)
nvauto Nov 25, 2025
aee8fab
Use shim to identify whether DataWriting is supported for LoRe (#13857)
thirtiseven Nov 25, 2025
cea6de9
[auto-merge] release/25.12 to main [skip ci] [bot] (#13868)
nvauto Nov 25, 2025
85a57cc
[auto-merge] release/25.12 to main [skip ci] [bot] (#13876)
nvauto Nov 25, 2025
2f95480
Merge release/25.12 into main
pxLi Nov 26, 2025
d5aedfb
Fix auto merge conflict 13877 [skip ci] (#13887)
pxLi Nov 26, 2025
1495282
[auto-merge] release/25.12 to main [skip ci] [bot] (#13888)
nvauto Nov 26, 2025
d1aee7e
Enable several Spark UT suites (#13883)
GaryShen2008 Nov 27, 2025
ffa95d3
Persist the buffer converters of TypedImperativeAggregate into Logica…
sperlingxx Nov 28, 2025
3f5dbf6
[auto-merge] release/25.12 to main [skip ci] [bot] (#13904)
nvauto Nov 28, 2025
a29957e
Add RapidsCsvSuite (#13902)
GaryShen2008 Nov 28, 2025
0577a4b
[auto-merge] release/25.12 to main [skip ci] [bot] (#13913)
nvauto Dec 1, 2025
a8c0a06
fix race condition due to premature disk handle exposure (#13900)
binmahone Dec 1, 2025
d63e86e
[auto-merge] release/25.12 to main [skip ci] [bot] (#13916)
nvauto Dec 1, 2025
3c38435
[AutoSparkUT]Enable RapidsCsvExpressionsSuite & RapidsCSVInferSchemaS…
GaryShen2008 Dec 2, 2025
3c5b020
[auto-merge] release/25.12 to main [skip ci] [bot] (#13920)
nvauto Dec 2, 2025
d69101d
[auto-merge] release/25.12 to main [skip ci] [bot] (#13924)
nvauto Dec 2, 2025
7aa0d86
Add in API definitions for Rapids UDAF (#13870)
firestarman Dec 3, 2025
925ef96
Use strict priority in conda process [skip ci] (#13931)
pxLi Dec 3, 2025
e3d85e0
Refine GpuTaskMetrics over SpillFrameWork (#13905)
sperlingxx Dec 4, 2025
72f8f5e
Add testRapids case to match GPU execution in RapidsDataFrameWindowFu…
GaryShen2008 Dec 4, 2025
da986c7
[auto-merge] release/25.12 to main [skip ci] [bot] (#13943)
nvauto Dec 4, 2025
4eb0957
Merge remote-tracking branch 'NVDA/release/25.12' into fix-auto-merge…
firestarman Dec 4, 2025
0d50fd3
Fix auto merge conflict 13946 [skip ci] (#13947)
pxLi Dec 4, 2025
65296ba
Set WONT_FIX_ISSUE cases in RadpisDataFrameWindowFunctionsSuite (#13945)
GaryShen2008 Dec 4, 2025
eb871b6
[auto-merge] release/25.12 to main [skip ci] [bot] (#13949)
nvauto Dec 4, 2025
acc60b8
[auto-merge] release/25.12 to main [skip ci] [bot] (#13958)
nvauto Dec 5, 2025
c0e139a
[auto-merge] release/25.12 to main [skip ci] [bot] (#13960)
nvauto Dec 5, 2025
fcd2d71
[auto-merge] release/25.12 to main [skip ci] [bot] (#13963)
nvauto Dec 5, 2025
6c71fdf
Fix a special case in limit where it could return an empty batch with…
revans2 Dec 5, 2025
b338039
[DOC] Update RapidsUDF output types with decimal 128 [skip ci] (#13955)
rishic3 Dec 5, 2025
cb31f54
[SparkUT]Check Java version to decide the expected string in one case…
GaryShen2008 Dec 8, 2025
ab07c8d
Add a non strict mode for lore dump (#13845)
thirtiseven Dec 8, 2025
56161e3
[DOC] fix dead link in testing page [skip ci] (#13933)
nvliyuan Dec 8, 2025
fdf2d20
Make the GPU UDF has different name than the CPU one (#13970)
firestarman Dec 8, 2025
11893da
[auto-merge] release/25.12 to main [skip ci] [bot] (#13974)
nvauto Dec 8, 2025
aaca18c
Fix a join where only a single column is used as a condition (#13964)
revans2 Dec 8, 2025
062d248
Support custom parallelism in non-default test modes [skip ci] (#13983)
yinqingh Dec 9, 2025
89ecc92
[auto-merge] release/25.12 to main [skip ci] [bot] (#13985)
nvauto Dec 9, 2025
035890e
Add in support for getting SQL metrics from expressions (#13981)
revans2 Dec 9, 2025
6ff068a
Add layer of indirection when converting expressions to the GPU (#13987)
revans2 Dec 10, 2025
541abdf
Building scala213 plugin for spark350+ (#13992)
pxLi Dec 11, 2025
a6877eb
Add pmattione to blossom: Attempt 2 [skip ci] (#13997)
pmattione-nvidia Dec 11, 2025
9bca06a
[auto-merge] release/25.12 to main [skip ci] [bot] (#14000)
nvauto Dec 12, 2025
7edfd65
[SparkUT]Add try-catch on dataframe.collect in UT framework (#13994)
GaryShen2008 Dec 12, 2025
8edff3a
Add parquet mixed-encodings test (#13982)
pmattione-nvidia Dec 12, 2025
cc02aeb
[auto-merge] release/25.12 to main [skip ci] [bot] (#14005)
nvauto Dec 12, 2025
b8f65b2
[auto-merge] release/25.12 to main [skip ci] [bot] (#14006)
nvauto Dec 13, 2025
1e4335a
Merge release/25.12 into main
pxLi Dec 15, 2025
dcb7aee
Fix auto merge conflict 14007 [skip ci] (#14008)
pxLi Dec 15, 2025
04372cb
[auto-merge] release/25.12 to main [skip ci] [bot] (#14014)
nvauto Dec 15, 2025
c98e800
Make convert in R2C retryable to prevent Host OOM (#13842)
thirtiseven Dec 15, 2025
068ad6e
[auto-merge] release/25.12 to main [skip ci] [bot] (#14016)
nvauto Dec 16, 2025
ffb2589
[auto-merge] release/25.12 to main [skip ci] [bot] (#14017)
nvauto Dec 16, 2025
bdfc109
Add iceberg 1.9.2 support. (#13986)
liurenjie1024 Dec 16, 2025
c80330d
Update actions/setup-java@v5 [skip ci] (#14024)
pxLi Dec 17, 2025
4ce6f1b
Update plugin scala212 to build 330+ only [databricks] (#13993)
pxLi Dec 17, 2025
4c28fb9
update cuda13 related jars
nvliyuan Dec 17, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .github/workflows/blossom-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,8 @@ jobs:
github.actor == 'pxLi' ||
github.actor == 'SurajAralihalli' ||
github.actor == 'jihoonson' ||
github.actor == 'knoguchi22'
github.actor == 'knoguchi22' ||
github.actor == 'pmattione-nvidia'
)
steps:
- name: Check if comment is issued by authorized person
Expand All @@ -99,7 +100,7 @@ jobs:

# repo specific steps
- name: Setup java
uses: actions/setup-java@v4
uses: actions/setup-java@v5
with:
distribution: adopt
java-version: 8
Expand Down
14 changes: 7 additions & 7 deletions .github/workflows/mvn-verify-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ jobs:
sparkJDKVersions: ${{ steps.all212ShimVersionsStep.outputs.jdkVersions }}
steps:
- uses: actions/checkout@v4 # refs/pull/:prNumber/merge
- uses: actions/setup-java@v4
- uses: actions/setup-java@v5
with:
distribution: 'temurin'
java-version: 8
Expand Down Expand Up @@ -117,7 +117,7 @@ jobs:
- uses: actions/checkout@v4 # refs/pull/:prNumber/merge

- name: Setup Java and Maven Env
uses: actions/setup-java@v4
uses: actions/setup-java@v5
with:
distribution: adopt
java-version: 8
Expand Down Expand Up @@ -164,7 +164,7 @@ jobs:
sparkJDK17Versions: ${{ steps.all213ShimVersionsStep.outputs.jdkVersions }}
steps:
- uses: actions/checkout@v4 # refs/pull/:prNumber/merge
- uses: actions/setup-java@v4
- uses: actions/setup-java@v5
with:
distribution: 'temurin'
java-version: 17
Expand Down Expand Up @@ -228,7 +228,7 @@ jobs:
- uses: actions/checkout@v4 # refs/pull/:prNumber/merge

- name: Setup Java and Maven Env
uses: actions/setup-java@v4
uses: actions/setup-java@v5
with:
distribution: adopt
java-version: 17
Expand Down Expand Up @@ -284,7 +284,7 @@ jobs:
- uses: actions/checkout@v4 # refs/pull/:prNumber/merge

- name: Setup Java and Maven Env
uses: actions/setup-java@v4
uses: actions/setup-java@v5
with:
distribution: adopt
java-version: 17
Expand Down Expand Up @@ -339,7 +339,7 @@ jobs:
- uses: actions/checkout@v4 # refs/pull/:prNumber/merge

- name: Setup Java and Maven Env
uses: actions/setup-java@v4
uses: actions/setup-java@v5
with:
distribution: adopt
java-version: ${{ matrix.java-version }}
Expand Down Expand Up @@ -387,7 +387,7 @@ jobs:
- uses: actions/checkout@v4 # refs/pull/:prNumber/merge

- name: Setup Java
uses: actions/setup-java@v4
uses: actions/setup-java@v5
with:
distribution: adopt
java-version: 11
Expand Down
8 changes: 4 additions & 4 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,15 +127,15 @@ mvn -pl dist -PnoSnapshots package -DskipTests
Verify that shim-specific classes are hidden from a conventional classloader.

```bash
$ javap -cp dist/target/rapids-4-spark_2.12-25.12.0-SNAPSHOT-cuda12.jar com.nvidia.spark.rapids.shims.SparkShimImpl
$ javap -cp dist/target/rapids-4-spark_2.12-26.02.0-SNAPSHOT-cuda12.jar com.nvidia.spark.rapids.shims.SparkShimImpl
Error: class not found: com.nvidia.spark.rapids.shims.SparkShimImpl
```

However, its bytecode can be loaded if prefixed with `spark3XY` not contained in the package name

```bash
$ javap -cp dist/target/rapids-4-spark_2.12-25.12.0-SNAPSHOT-cuda12.jar spark320.com.nvidia.spark.rapids.shims.SparkShimImpl | head -2
Warning: File dist/target/rapids-4-spark_2.12-25.12.0-SNAPSHOT-cuda12.jar(/spark320/com/nvidia/spark/rapids/shims/SparkShimImpl.class) does not contain class spark320.com.nvidia.spark.rapids.shims.SparkShimImpl
$ javap -cp dist/target/rapids-4-spark_2.12-26.02.0-SNAPSHOT-cuda12.jar spark320.com.nvidia.spark.rapids.shims.SparkShimImpl | head -2
Warning: File dist/target/rapids-4-spark_2.12-26.02.0-SNAPSHOT-cuda12.jar(/spark320/com/nvidia/spark/rapids/shims/SparkShimImpl.class) does not contain class spark320.com.nvidia.spark.rapids.shims.SparkShimImpl
Compiled from "SparkShims.scala"
public final class com.nvidia.spark.rapids.shims.SparkShimImpl {
```
Expand Down Expand Up @@ -177,7 +177,7 @@ mvn package -pl dist -am -Dbuildver=340 -DallowConventionalDistJar=true
Verify `com.nvidia.spark.rapids.shims.SparkShimImpl` is conventionally loadable:

```bash
$ javap -cp dist/target/rapids-4-spark_2.12-25.12.0-SNAPSHOT-cuda12.jar com.nvidia.spark.rapids.shims.SparkShimImpl | head -2
$ javap -cp dist/target/rapids-4-spark_2.12-26.02.0-SNAPSHOT-cuda12.jar com.nvidia.spark.rapids.shims.SparkShimImpl | head -2
Compiled from "SparkShims.scala"
public final class com.nvidia.spark.rapids.shims.SparkShimImpl {
```
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ as a `provided` dependency.
<dependency>
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark_2.12</artifactId>
<version>25.12.0-SNAPSHOT</version>
<version>26.02.0-SNAPSHOT</version>
<scope>provided</scope>
</dependency>
```
4 changes: 2 additions & 2 deletions aggregator/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,13 +22,13 @@
<parent>
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark-jdk-profiles_2.12</artifactId>
<version>25.12.0-SNAPSHOT</version>
<version>26.02.0-SNAPSHOT</version>
<relativePath>../jdk-profiles/pom.xml</relativePath>
</parent>
<artifactId>rapids-4-spark-aggregator_2.12</artifactId>
<name>RAPIDS Accelerator for Apache Spark Aggregator</name>
<description>Creates an aggregated shaded package of the RAPIDS plugin for Apache Spark</description>
<version>25.12.0-SNAPSHOT</version>
<version>26.02.0-SNAPSHOT</version>

<properties>
<rapids.module>aggregator</rapids.module>
Expand Down
4 changes: 2 additions & 2 deletions api_validation/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,11 @@
<parent>
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark-shim-deps-parent_2.12</artifactId>
<version>25.12.0-SNAPSHOT</version>
<version>26.02.0-SNAPSHOT</version>
<relativePath>../shim-deps/pom.xml</relativePath>
</parent>
<artifactId>rapids-4-spark-api-validation_2.12</artifactId>
<version>25.12.0-SNAPSHOT</version>
<version>26.02.0-SNAPSHOT</version>

<properties>
<rapids.module>api_validation</rapids.module>
Expand Down
10 changes: 5 additions & 5 deletions build/buildall
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,8 @@ function print_usage() {
echo " use this profile for the dist module, default: noSnapshots, also supported: snapshots, minimumFeatureVersionMix,"
echo " snapshotsWithDatabricks, noSnapshotsWithDatabricks, noSnapshotsScala213, snapshotsScala213."
echo " NOTE: the Databricks-related spark3XYdb shims are not built locally, the jars are fetched prebuilt from a"
echo " . remote Maven repo. You can also supply a comma-separated list of build versions. E.g., --profile=320,330 will"
echo " build only the distribution jar only for 3.2.0 and 3.3.0"
echo " . remote Maven repo. You can also supply a comma-separated list of build versions. E.g., --profile=330,331 will"
echo " build only the distribution jar only for 3.3.0 and 3.3.1"
echo " -m=MODULE, --module=MODULE"
echo " after finishing parallel builds, resume from dist and build up to and including module MODULE."
echo " E.g., --module=integration_tests"
Expand Down Expand Up @@ -277,8 +277,8 @@ export -f build_single_shim
# Install all the versions for DIST_PROFILE

# First build the aggregator module for all SPARK_SHIM_VERSIONS in parallel skipping expensive plugins that
# - either deferred to 320 because the check is identical in all shim profiles such as scalastyle
# - or deferred to 320 because we currently don't require it per shim such as scaladoc generation
# - either deferred to 330 because the check is identical in all shim profiles such as scalastyle
# - or deferred to 330 because we currently don't require it per shim such as scaladoc generation
# - or there is a dedicated step to run against a particular shim jar such as unit tests, in
# the near future we will run unit tests against a combined multi-shim jar to catch classloading
# regressions even before pytest-based integration_tests
Expand All @@ -299,7 +299,7 @@ time (
fi
# This used to resume from dist. However, without including aggregator in the build
# the build does not properly initialize spark.version property via buildver profiles
# in the root pom, and we get a missing spark320 dependency even for --profile=320,321
# in the root pom, and we get a missing spark330 dependency even for --profile=330,331
# where the build does not require it. Moving it to aggregator resolves this issue with
# a negligible increase of the build time by ~2 seconds.
joinShimBuildFrom="aggregator"
Expand Down
2 changes: 1 addition & 1 deletion build/coverage-report
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ TMP_CLASS=${TEMP_CLASS_LOC:-"./target/jacoco_classes/"}
HTML_LOC=${HTML_LOCATION:="./target/jacoco-report/"}
XML_LOC=${XML_LOCATION:="${HTML_LOC}"}
DIST_JAR=${RAPIDS_DIST_JAR:-$(ls ./dist/target/rapids-4-spark_2.12-*cuda*.jar | grep -v test | head -1 | xargs readlink -f)}
SPK_VER=${JACOCO_SPARK_VER:-"320"}
SPK_VER=${JACOCO_SPARK_VER:-"330"}
UDF_JAR=${RAPIDS_UDF_JAR:-$(ls ./udf-compiler/target/spark${SPK_VER}/rapids-4-spark-udf_2.12-*-SNAPSHOT-spark${SPK_VER}.jar | grep -v test | head -1 | xargs readlink -f)}
SOURCE_DIRS=${SOURCE_DIRS:-"./sql-plugin/src/main/scala/:./sql-plugin/src/main/java/:./shuffle-plugin/src/main/scala/:./udf-compiler/src/main/scala/"}

Expand Down
6 changes: 3 additions & 3 deletions build/make-scala-version-build-files.sh
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,8 @@ trap "trap_func" EXIT

VALID_VERSIONS=( 2.13 )
declare -A DEFAULT_SPARK
DEFAULT_SPARK[2.12]="spark320"
DEFAULT_SPARK[2.13]="spark330"
DEFAULT_SPARK[2.12]="spark330"
DEFAULT_SPARK[2.13]="spark350"

usage() {
echo "Usage: $(basename $0) [-h|--help] <version>
Expand Down Expand Up @@ -90,7 +90,7 @@ for f in $(git ls-files '**pom.xml'); do
sed_i 's/^\([[:space:]]*\)\(<!-- #endif scala-'$FROM_VERSION' -->\)/\1-->\2/' $tof
done

# Update spark.version to spark330.version for Scala 2.13
# Update spark.version to spark350.version for Scala 2.13
SPARK_VERSION=${DEFAULT_SPARK[$TO_VERSION]}
sed_i '/<java\.major\.version>/,/<spark\.version>\${spark[0-9]\+\.version}</s/<spark\.version>\${spark[0-9]\+\.version}</<spark.version>\${'$SPARK_VERSION'.version}</' \
"$TO_DIR/pom.xml"
Expand Down
6 changes: 3 additions & 3 deletions datagen/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,12 +24,12 @@ Where `$SPARK_VERSION` is a compressed version number, like 330 for Spark 3.3.0.

After this the jar should be at
`target/datagen_2.12-$PLUGIN_VERSION-spark$SPARK_VERSION.jar`
for example a Spark 3.3.0 jar for the 25.12.0 release would be
`target/datagen_2.12-25.12.0-spark330.jar`
for example a Spark 3.3.0 jar for the 26.02.0 release would be
`target/datagen_2.12-26.02.0-spark330.jar`

To get a spark shell with this you can run
```shell
spark-shell --jars target/datagen_2.12-25.12.0-spark330.jar
spark-shell --jars target/datagen_2.12-26.02.0-spark330.jar
```

After that you should be good to go.
Expand Down
2 changes: 1 addition & 1 deletion datagen/ScaleTest.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ $SPARK_HOME/bin/spark-submit \
--conf spark.sql.parquet.datetimeRebaseModeInWrite=CORRECTED \
--class com.nvidia.rapids.tests.scaletest.ScaleTestDataGen \ # the main class
--jars $SPARK_HOME/examples/jars/scopt_2.12-3.7.1.jar \ # one dependency jar just shipped with Spark under $SPARK_HOME
./target/datagen_2.12-25.12.0-SNAPSHOT-spark332.jar \
./target/datagen_2.12-26.02.0-SNAPSHOT-spark332.jar \
1 \
10 \
parquet \
Expand Down
4 changes: 2 additions & 2 deletions datagen/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -21,13 +21,13 @@
<parent>
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark-shim-deps-parent_2.12</artifactId>
<version>25.12.0-SNAPSHOT</version>
<version>26.02.0-SNAPSHOT</version>
<relativePath>../shim-deps/pom.xml</relativePath>
</parent>
<artifactId>datagen_2.12</artifactId>
<name>Data Generator</name>
<description>Tools for generating large amounts of data</description>
<version>25.12.0-SNAPSHOT</version>
<version>26.02.0-SNAPSHOT</version>
<properties>
<rapids.module>datagen</rapids.module>
<target.classifier/>
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* Copyright (c) 2022-2023, NVIDIA CORPORATION.
* Copyright (c) 2022-2025, NVIDIA CORPORATION.
*
* This file was derived from CheckDeltaInvariant.scala in the
* Delta Lake project at https://github.com/delta-io/delta.
Expand All @@ -24,7 +24,7 @@ package com.databricks.sql.transaction.tahoe.rapids
import ai.rapids.cudf.{ColumnVector, Scalar}
import com.databricks.sql.transaction.tahoe.constraints.{CheckDeltaInvariant, Constraint}
import com.databricks.sql.transaction.tahoe.constraints.Constraints.{Check, NotNull}
import com.nvidia.spark.rapids.{DataFromReplacementRule, ExprChecks, GpuBindReferences, GpuColumnVector, GpuExpression, GpuOverrides, RapidsConf, RapidsMeta, TypeSig, UnaryExprMeta}
import com.nvidia.spark.rapids.{DataFromReplacementRule, ExprChecks, GpuBindReferences, GpuColumnVector, GpuExpression, GpuMetric, GpuOverrides, RapidsConf, RapidsMeta, TypeSig, UnaryExprMeta}
import com.nvidia.spark.rapids.Arm.withResource
import com.nvidia.spark.rapids.RapidsPluginImplicits._
import com.nvidia.spark.rapids.delta.shims.InvariantViolationExceptionShim
Expand Down Expand Up @@ -54,9 +54,10 @@ case class GpuCheckDeltaInvariant(
override def foldable: Boolean = false
override def nullable: Boolean = true

def withBoundReferences(input: AttributeSeq): GpuCheckDeltaInvariant = {
def withBoundReferences(input: AttributeSeq,
metrics: Map[String, GpuMetric]): GpuCheckDeltaInvariant = {
GpuCheckDeltaInvariant(
GpuBindReferences.bindReference(child, input),
GpuBindReferences.bindReference(child, input, metrics),
columnExtractors.map {
case (column, extractor) => column -> BindReferences.bindReference(extractor, input)
},
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* Copyright (c) 2022-2023, NVIDIA CORPORATION.
* Copyright (c) 2022-2025, NVIDIA CORPORATION.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
Expand Down Expand Up @@ -45,7 +45,7 @@ case class GpuDeltaInvariantCheckerExec(

override protected def internalDoExecuteColumnar(): RDD[ColumnarBatch] = {
if (checks.isEmpty) return child.executeColumnar()
val boundRefs = checks.map(_.withBoundReferences(child.output))
val boundRefs = checks.map(_.withBoundReferences(child.output, allMetrics))

child.executeColumnar().mapPartitionsInternal { batches =>
batches.map { batch =>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ case class GpuIncrementMetricMeta(
override val conf: RapidsConf,
p: Option[RapidsMeta[_, _, _]],
r: DataFromReplacementRule) extends ExprMeta[IncrementMetric](cpuInc, conf, p, r) {
override def convertToGpu(): GpuExpression = {
override def convertToGpuImpl(): GpuExpression = {
val gpuChild = childExprs.head.convertToGpu()
GpuIncrementMetric(cpuInc, gpuChild)
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,11 +54,12 @@ case class GpuCheckDeltaInvariant(
override def foldable: Boolean = false
override def nullable: Boolean = true

def withBoundReferences(input: AttributeSeq): GpuCheckDeltaInvariant = {
def withBoundReferences(input: AttributeSeq,
metrics: Map[String, GpuMetric]): GpuCheckDeltaInvariant = {
GpuCheckDeltaInvariant(
GpuBindReferences.bindReference(child, input),
columnExtractors.map { case (column, extractor) =>
column -> BindReferences.bindReference(extractor, input)
GpuBindReferences.bindReference(child, input, metrics),
columnExtractors.map {
case (column, extractor) => column -> BindReferences.bindReference(extractor, input)
},
constraint)
}
Expand Down Expand Up @@ -175,7 +176,7 @@ class GpuCheckDeltaInvariantMeta(
}
}

override def convertToGpu(): GpuExpression = {
override def convertToGpuImpl(): GpuExpression = {
val child = childExprs.head.convertToGpu()
// Delta 4.0 provides columnExtractors as Seq[(String, Expression)] while older
// versions provide Map[String, Expression]. Normalize to Map for GPU version.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* Copyright (c) 2022-2023, NVIDIA CORPORATION.
* Copyright (c) 2022-2025, NVIDIA CORPORATION.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
Expand Down Expand Up @@ -45,7 +45,7 @@ case class GpuDeltaInvariantCheckerExec(

override protected def internalDoExecuteColumnar(): RDD[ColumnarBatch] = {
if (checks.isEmpty) return child.executeColumnar()
val boundRefs = checks.map(_.withBoundReferences(child.output))
val boundRefs = checks.map(_.withBoundReferences(child.output, allMetrics))

child.executeColumnar().mapPartitionsInternal { batches =>
batches.map { batch =>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,7 @@ case class GpuRapidsProcessDeltaMergeJoinExec(
}

private def bindForGpu(e: Expression): GpuExpression = {
GpuBindReferences.bindGpuReference(e, child.output)
GpuBindReferences.bindGpuReference(e, child.output, allMetrics)
}

override protected def doExecute(): RDD[InternalRow] = {
Expand Down
4 changes: 2 additions & 2 deletions delta-lake/delta-20x/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,14 @@
<parent>
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark-jdk-profiles_2.12</artifactId>
<version>25.12.0-SNAPSHOT</version>
<version>26.02.0-SNAPSHOT</version>
<relativePath>../../jdk-profiles/pom.xml</relativePath>
</parent>

<artifactId>rapids-4-spark-delta-20x_2.12</artifactId>
<name>RAPIDS Accelerator for Apache Spark Delta Lake 2.0.x Support</name>
<description>Delta Lake 2.0.x support for the RAPIDS Accelerator for Apache Spark</description>
<version>25.12.0-SNAPSHOT</version>
<version>26.02.0-SNAPSHOT</version>

<properties>
<rapids.module>../delta-lake/delta-20x</rapids.module>
Expand Down
4 changes: 2 additions & 2 deletions delta-lake/delta-21x/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,14 @@
<parent>
<groupId>com.nvidia</groupId>
<artifactId>rapids-4-spark-jdk-profiles_2.12</artifactId>
<version>25.12.0-SNAPSHOT</version>
<version>26.02.0-SNAPSHOT</version>
<relativePath>../../jdk-profiles/pom.xml</relativePath>
</parent>

<artifactId>rapids-4-spark-delta-21x_2.12</artifactId>
<name>RAPIDS Accelerator for Apache Spark Delta Lake 2.1.x Support</name>
<description>Delta Lake 2.1.x support for the RAPIDS Accelerator for Apache Spark</description>
<version>25.12.0-SNAPSHOT</version>
<version>26.02.0-SNAPSHOT</version>

<properties>
<rapids.module>../delta-lake/delta-21x</rapids.module>
Expand Down
Loading