[CORE] Unbundle Arrow memory + vector from gluten-velox-bundle

Mark arrow-memory-{core,unsafe,netty} and arrow-vector as scope=provided in
gluten-arrow/pom.xml. They are bundled in Spark's distribution
($SPARK_HOME/jars/ for Spark 3.x; declared in Spark 4.x's pom), so the user's
classpath already has them at runtime — gluten does not need to ship its own
copy.

Effects:

* The gluten-velox bundle no longer ships ANY org.apache.arrow.memory.* or
  org.apache.arrow.vector.* classes. The class-shadowing problem from #12225
  goes away by construction — there is no gluten-shipped copy left to shadow
  the user's vanilla Arrow.
* The org.apache.arrow shade-relocation block in package/pom.xml becomes
  redundant and is removed: arrow-memory/vector are no longer in the bundle
  to relocate, and arrow-c-data / arrow-dataset (still bundled) were already
  excluded from relocation because their JNI binds to the original class
  names.
* arrow-c-data and arrow-dataset remain at scope=compile in gluten-arrow —
  Spark does NOT ship those, so gluten still bundles them. With the relocation
  block gone, their public method signatures naturally bind to the user's
  vanilla org.apache.arrow.memory.BufferAllocator / arrow-vector types,
  exactly matching what every other Arrow C-Data caller on the classpath
  expects.

Compile-classpath touch-ups:

* backends-velox/pom.xml: re-declare arrow-memory-core and arrow-vector at
  scope=provided. The transitive route through gluten-arrow no longer carries
  them after the scope flip, so backends-velox needs its own provided
  declaration to compile.
* gluten-ut/* and backends-clickhouse already declare arrow at provided
  scope locally, so they are unaffected.

Caveats:

* Spark 3.5 and earlier do NOT declare arrow-memory/arrow-vector in their
  Maven POM (they ship them inside the binary distribution only). gluten
  builds against the version pinned in `arrow.version`. Maintainers should
  keep `arrow.version` aligned with the lowest-common-denominator Arrow
  version across supported Spark distros (DBR 16.4 ships Arrow 12.0.1 with
  Spark 3.5; vanilla Spark 3.5.x ships 15.0.0 — the 15.0.0 default here is
  fine for vanilla Spark 3.5 but may need a compat profile for DBR/Cloudera
  flavors).
* dev/check-arrow-c-shading.sh added in #12226 still passes — the bundle
  still contains org/apache/arrow/c/* classes whose method signatures now
  reference unshaded org.apache.arrow.memory.* / org.apache.arrow.vector.*
  types (which are no longer in the bundle, but resolve at runtime from
  Spark's Arrow).

Builds on #12244 (drop the 15.0.0-gluten Arrow version rename). Addresses the
follow-up direction from #12226 discussion: "remove Arrow from the bundled
Gluten Jar and let users rely on Spark's bundled Arrow".

Author: sezruby

backends-velox/pom.xml

@@ -79,6 +79,24 @@
       <version>${project.version}</version>
       <scope>compile</scope>
     </dependency>
+    <!--
+      arrow-memory-core / arrow-vector are scope=provided in gluten-arrow so
+      they are not bundled. Re-declare them at provided scope here so the
+      compile classpath still resolves them. At runtime they come from
+      Spark's distribution.
+    -->
+    <dependency>
+      <groupId>org.apache.arrow</groupId>
+      <artifactId>arrow-memory-core</artifactId>
+      <version>${arrow.version}</version>
+      <scope>provided</scope>
+    </dependency>
+    <dependency>
+      <groupId>org.apache.arrow</groupId>
+      <artifactId>arrow-vector</artifactId>
+      <version>${arrow.version}</version>
+      <scope>provided</scope>
+    </dependency>
     <dependency>
       <groupId>com.github.ben-manes.caffeine</groupId>
       <artifactId>caffeine</artifactId>

gluten-arrow/pom.xml

@@ -85,17 +85,24 @@
       <version>${spark.version}</version>
       <scope>provided</scope>
     </dependency>
+    <!--
+      Arrow memory + vector come from Spark's distribution (declared in Spark
+      4.x's pom; bundled in $SPARK_HOME/jars/ for Spark 3.x). gluten compiles
+      against the same arrow.version it expects at runtime, but the artifacts
+      are scope=provided so they are NOT bundled into gluten-velox-bundle —
+      avoiding the #12225 class-shadowing problem entirely.
+    -->
     <dependency>
       <groupId>org.apache.arrow</groupId>
       <artifactId>${arrow-memory.artifact}</artifactId>
       <version>${arrow.version}</version>
-      <scope>runtime</scope>
+      <scope>provided</scope>
     </dependency>
     <dependency>
       <groupId>org.apache.arrow</groupId>
       <artifactId>arrow-memory-core</artifactId>
       <version>${arrow.version}</version>
-      <scope>compile</scope>
+      <scope>provided</scope>
       <exclusions>
         <exclusion>
           <groupId>io.netty</groupId>
@@ -111,6 +118,7 @@
       <groupId>org.apache.arrow</groupId>
       <artifactId>arrow-vector</artifactId>
       <version>${arrow.version}</version>
+      <scope>provided</scope>
       <exclusions>
         <exclusion>
           <groupId>io.netty</groupId>

package/pom.xml

@@ -118,28 +118,15 @@
                     <include>com.google.gson.**</include>
                   </includes>
                 </relocation>
-                <relocation>
-                  <pattern>org.apache.arrow</pattern>
-                  <shadedPattern>${gluten.shade.packageName}.org.apache.arrow</shadedPattern>
-                  <!--
-                    arrow's C and dataset wrappers refer to the original class
-                    path, so they must not be relocated. Their public APIs also
-                    take and return org.apache.arrow.memory.* and
-                    org.apache.arrow.vector.* types, so those packages must also
-                    stay unshaded — otherwise the bundled (unshaded)
-                    ArrowArrayStream/ArrowSchema get compiled against the
-                    relocated BufferAllocator/VectorSchemaRoot, producing
-                    `NoSuchMethodError` for any caller passing a vanilla
-                    Apache Arrow allocator. See #12225.
-                  -->
-                  <excludes>
-                    <exclude>org.apache.arrow.c.*</exclude>
-                    <exclude>org.apache.arrow.c.jni.*</exclude>
-                    <exclude>org.apache.arrow.memory.**</exclude>
-                    <exclude>org.apache.arrow.vector.**</exclude>
-                    <exclude>org.apache.arrow.dataset.**</exclude>
-                  </excludes>
-                </relocation>
+                <!--
+                  org.apache.arrow.memory.* and org.apache.arrow.vector.* are
+                  now scope=provided in gluten-arrow/pom.xml — they come from
+                  Spark's distribution at runtime, so there is nothing to
+                  relocate. arrow-c-data and arrow-dataset are still bundled
+                  but never relocated (their JNI binds to the original class
+                  names), so no shade-relocation entry is needed for Arrow.
+                  See #12225 / #12226 for the historical context.
+                -->
                 <relocation>
                   <pattern>com.google.flatbuffers</pattern>
                   <shadedPattern>${gluten.shade.packageName}.com.google.flatbuffers</shadedPattern>