[AutoSparkUT] [SPARK-31284, SPARK-31423: rebasing timestamps in write] - GPU Execution Issue

[wjxiz1992]

Describe the bug

The Spark 3.3 ORC timestamp-rebasing write/round-trip contract produces
different timestamp values on GPU and CPU. The issue reproduces through both
V1 and V2 and with row/vectorized ORC reader settings.

Steps/code to reproduce

Build the exact PR head, then use Spark 3.3.0 and the complete RAPIDS UT
configuration:

mvn package -DskipTests -pl dist -am -Dbuildver=330 \
  -Dmaven.repo.local=./.mvn-repo

$SPARK_HOME/bin/spark-shell \
  --master local[2] \
  --conf spark.sql.optimizer.excludedRules=org.apache.spark.sql.catalyst.optimizer.ConvertToLocalRelation,org.apache.spark.sql.catalyst.optimizer.ConstantFolding \
  --conf spark.rapids.sql.enabled=true \
  --conf spark.plugins=com.nvidia.spark.SQLPlugin \
  --conf spark.sql.queryExecutionListeners=org.apache.spark.sql.rapids.ExecutionPlanCaptureCallback \
  --conf spark.rapids.sql.explain=ALL \
  --conf spark.rapids.sql.test.isFoldableNonLitAllowed=true \
  --conf spark.rapids.sql.csv.read.decimal.enabled=true \
  --conf spark.rapids.sql.format.avro.enabled=true \
  --conf spark.rapids.sql.format.avro.read.enabled=true \
  --conf spark.rapids.sql.format.hive.text.write.enabled=true \
  --conf spark.rapids.sql.format.json.enabled=true \
  --conf spark.rapids.sql.format.json.read.enabled=true \
  --conf spark.rapids.sql.incompatibleDateFormats.enabled=true \
  --conf spark.rapids.sql.python.gpu.enabled=true \
  --conf spark.rapids.sql.rowBasedUDF.enabled=true \
  --conf spark.rapids.sql.window.collectList.enabled=true \
  --conf spark.rapids.sql.window.collectSet.enabled=true \
  --conf spark.rapids.sql.window.range.byte.enabled=true \
  --conf spark.rapids.sql.window.range.short.enabled=true \
  --conf spark.rapids.sql.expression.Ascii=true \
  --conf spark.rapids.sql.expression.Conv=true \
  --conf spark.rapids.sql.expression.GetJsonObject=true \
  --conf spark.rapids.sql.expression.JsonToStructs=true \
  --conf spark.rapids.sql.expression.StructsToJson=true \
  --conf spark.rapids.sql.exec.CollectLimitExec=true \
  --conf spark.rapids.sql.exec.FlatMapCoGroupsInPandasExec=true \
  --conf spark.rapids.sql.exec.WindowInPandasExec=true \
  --conf spark.rapids.sql.hasExtendedYearValues=false \
  --conf spark.unsafe.exceptionOnMemoryLeak=true \
  --conf spark.sql.session.timeZone=America/Los_Angeles \
  --jars $RAPIDS_JAR

Run extract-test-configs.scala first. In the same session, use a new
non-existing path for each run; run GPU first and CPU second for V1/V2 and
spark.sql.orc.enableVectorizedReader=false,true:

import java.util.TimeZone
import spark.implicits._
TimeZone.setDefault(TimeZone.getTimeZone("America/Los_Angeles"))
spark.conf.set("spark.sql.orc.impl", "native")

Seq("1001-01-01 01:02:03.123456", "1582-10-10 11:12:13.654321")
  .toDF("tsS")
  .select($"tsS".cast("timestamp").as("ts"))
  .write.orc(outputPath)

val df = spark.read.orc(outputPath)
println(df.collect().toSeq)
println(df.queryExecution.executedPlan)

Actual output

RAPIDS Enabled = YES
Plugins Loaded = YES
SPARK_VERSION = 3.3.0

GPU V1/V2, vectorized=false/true:
  [1001-01-07 01:09:05.123456]
  [1582-10-15 11:19:15.654321]

CPU V1/V2, vectorized=false/true:
  [1001-01-01 01:02:03.123456]
  [1582-10-15 11:12:13.654321]

GPU read plans contain GpuFileGpuScan (V1) or GpuBatchScan/GpuOrcScan (V2).

Expected behavior

The GPU round trip should return the same two timestamps as Spark CPU:
1001-01-01 01:02:03.123456 and 1582-10-15 11:12:13.654321.

Environment

• Local Spark 3.3.0, revision f74867bddfbcdd4d08076db36851e88b15e66556
• cudf-spark f7457eeee116ca033ffa84eadbe84226e01b2ff7, version
  26.10.0-SNAPSHOT, CUDA 12 JAR
• NVIDIA RTX 5880 Ada Generation, driver 580.95.05
• Java 17

Additional context

• Original Spark test: OrcSourceSuite.scala lines 520-537
• Affected migrated suites: RapidsOrcSourceV1Suite, RapidsOrcSourceV2Suite
• Migration tracker: [AutoSparkUT] Migrate ten additional Spark SQL suites #15465
• Migration PR: [AutoSparkUT] Migrate ten Spark SQL suites [fast-ut] [reduced-it] #15466
• The round-trip result proves the test contract is broken. Cross-reading GPU-
  and CPU-written files is still needed to attribute writer versus reader portions.
• Classification: GPU/CPU data-correctness bug; recovery priority P0
• Reproduction rate: 100% in all four V1/V2 and reader-mode combinations