[AutoSparkUT] [Write Spark version into ORC file metadata] - GPU Execution Issue

[wjxiz1992]

Describe the bug

The GPU ORC writer does not add the org.apache.spark.version user-metadata key
required by Spark 3.3.0. The corresponding CPU writer stores 3.3.0. This is
reproduced through both the V1 and V2 ORC data-source paths.

Steps/code to reproduce

Build the exact PR head and launch Spark 3.3.0 with the complete RAPIDS UT
configuration:

mvn package -DskipTests -pl dist -am -Dbuildver=330 \
  -Dmaven.repo.local=./.mvn-repo

$SPARK_HOME/bin/spark-shell \
  --master local[2] \
  --conf spark.sql.optimizer.excludedRules=org.apache.spark.sql.catalyst.optimizer.ConvertToLocalRelation,org.apache.spark.sql.catalyst.optimizer.ConstantFolding \
  --conf spark.rapids.sql.enabled=true \
  --conf spark.plugins=com.nvidia.spark.SQLPlugin \
  --conf spark.sql.queryExecutionListeners=org.apache.spark.sql.rapids.ExecutionPlanCaptureCallback \
  --conf spark.rapids.sql.explain=ALL \
  --conf spark.rapids.sql.test.isFoldableNonLitAllowed=true \
  --conf spark.rapids.sql.csv.read.decimal.enabled=true \
  --conf spark.rapids.sql.format.avro.enabled=true \
  --conf spark.rapids.sql.format.avro.read.enabled=true \
  --conf spark.rapids.sql.format.hive.text.write.enabled=true \
  --conf spark.rapids.sql.format.json.enabled=true \
  --conf spark.rapids.sql.format.json.read.enabled=true \
  --conf spark.rapids.sql.incompatibleDateFormats.enabled=true \
  --conf spark.rapids.sql.python.gpu.enabled=true \
  --conf spark.rapids.sql.rowBasedUDF.enabled=true \
  --conf spark.rapids.sql.window.collectList.enabled=true \
  --conf spark.rapids.sql.window.collectSet.enabled=true \
  --conf spark.rapids.sql.window.range.byte.enabled=true \
  --conf spark.rapids.sql.window.range.short.enabled=true \
  --conf spark.rapids.sql.expression.Ascii=true \
  --conf spark.rapids.sql.expression.Conv=true \
  --conf spark.rapids.sql.expression.GetJsonObject=true \
  --conf spark.rapids.sql.expression.JsonToStructs=true \
  --conf spark.rapids.sql.expression.StructsToJson=true \
  --conf spark.rapids.sql.exec.CollectLimitExec=true \
  --conf spark.rapids.sql.exec.FlatMapCoGroupsInPandasExec=true \
  --conf spark.rapids.sql.exec.WindowInPandasExec=true \
  --conf spark.rapids.sql.hasExtendedYearValues=false \
  --conf spark.unsafe.exceptionOnMemoryLeak=true \
  --conf spark.sql.session.timeZone=America/Los_Angeles \
  --jars $RAPIDS_JAR

Run the project extract-test-configs.scala verification first, then execute
this in the same session for spark.sql.sources.useV1SourceList=orc and "".
Use a fresh non-existing output path for every run; run GPU first, then set
spark.rapids.sql.enabled=false and repeat on CPU.

spark.range(1).repartition(1).write.orc(outputPath)
val file = new java.io.File(outputPath).listFiles
  .filter(f => f.isFile && !f.getName.startsWith(".") && !f.getName.startsWith("_"))
  .head
val reader = org.apache.orc.OrcFile.createReader(
  new org.apache.hadoop.fs.Path(file.getAbsolutePath),
  org.apache.orc.OrcFile.readerOptions(new org.apache.hadoop.conf.Configuration()))
reader.getMetadataValue("org.apache.spark.version")

Actual output

RAPIDS Enabled = YES
Plugins Loaded = YES
SPARK_VERSION = 3.3.0

GPU-V1: java.lang.IllegalArgumentException: Can't find user metadata org.apache.spark.version
GPU-V2: java.lang.IllegalArgumentException: Can't find user metadata org.apache.spark.version
CPU-V1: metadata=3.3.0
CPU-V2: metadata=3.3.0

Expected behavior

GPU-written ORC files should contain org.apache.spark.version=3.3.0, matching
Spark CPU output.

Environment

• Local Spark 3.3.0, revision f74867bddfbcdd4d08076db36851e88b15e66556
• cudf-spark f7457eeee116ca033ffa84eadbe84226e01b2ff7, version
  26.10.0-SNAPSHOT, CUDA 12 JAR
• NVIDIA RTX 5880 Ada Generation, driver 580.95.05
• Java 17

Additional context

• Original Spark test: OrcSourceSuite.scala lines 370-385
• Affected migrated suites: RapidsOrcSourceV1Suite, RapidsOrcSourceV2Suite
• Migration tracker: [AutoSparkUT] Migrate ten additional Spark SQL suites #15465
• Migration PR: [AutoSparkUT] Migrate ten Spark SQL suites [fast-ut] [reduced-it] #15466
• Classification: GPU ORC writer metadata compatibility bug; recovery priority P1
• Reproduction rate: 100% in V1 and V2