Skip to content

Commit 0fb568c

Browse files
committed
[SPARK-58947][CORE] Handle Char in createArray and add tests for SparkCollectionUtils
### What changes were proposed in this pull request? Adds a unit test suite for `SparkCollectionUtils` (`common/utils`, `org.apache.spark.util`), which had no test coverage, and fixes a gap the tests surfaced. Tests cover all four methods: `toMapWithIndex` (empty input, duplicate keys, equivalence to `zipWithIndex.toMap`); `isEmpty` / `isNotEmpty` (null, empty, non-empty `java.util.Map`); and `createArray` for every primitive type plus reference types and `size == 0`. Adding the `createArray` coverage exposed a latent bug: it dispatched on 7 of the 8 primitive `classOf` types (Boolean/Byte/Short/Int/Long/Float/Double) but not `Char`, so a `Char` argument fell through to the `Array[AnyRef]` cast and threw `ClassCastException` (a `char[]` is not an `Object[]`). This adds the missing `Char` case so `createArray[Char]` fills the array like the other primitives. ### Why are the changes needed? `SparkCollectionUtils` is used in production (e.g. `StructType` field indexing via `toMapWithIndex`) but had no tests. The `Char` gap is currently unreachable (no caller passes `Char`), but it is a latent crash for any future `Char` caller; adding the case makes the dispatch complete and symmetric with the other primitives. ### Does this PR introduce _any_ user-facing change? No. `SparkCollectionUtils` is `private[spark]`; this adds tests and a missing internal `Char` case. ### How was this patch tested? New `SparkCollectionUtilsSuite` (extends `AnyFunSuite`, the base used by the sibling `common/utils` suites), covering all four methods including `createArray` for all eight primitive types, reference types, and `size == 0`. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Opus 4.8) Closes #58227 from uros-b/test-sparkcollectionutils. Authored-by: Uros <221401595+uros-b@users.noreply.github.com> Signed-off-by: Uros Bojanic <221401595+uros-b@users.noreply.github.com>
1 parent d01d020 commit 0fb568c

2 files changed

Lines changed: 66 additions & 0 deletions

File tree

common/utils/src/main/scala/org/apache/spark/util/SparkCollectionUtils.scala

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,8 @@ private[spark] trait SparkCollectionUtils {
5252
Arrays.fill(arr.asInstanceOf[Array[Byte]], defaultValue.asInstanceOf[Byte])
5353
case c if c == classOf[Short] =>
5454
Arrays.fill(arr.asInstanceOf[Array[Short]], defaultValue.asInstanceOf[Short])
55+
case c if c == classOf[Char] =>
56+
Arrays.fill(arr.asInstanceOf[Array[Char]], defaultValue.asInstanceOf[Char])
5557
case c if c == classOf[Int] =>
5658
Arrays.fill(arr.asInstanceOf[Array[Int]], defaultValue.asInstanceOf[Int])
5759
case c if c == classOf[Long] =>
Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
/*
2+
* Licensed to the Apache Software Foundation (ASF) under one or more
3+
* contributor license agreements. See the NOTICE file distributed with
4+
* this work for additional information regarding copyright ownership.
5+
* The ASF licenses this file to You under the Apache License, Version 2.0
6+
* (the "License"); you may not use this file except in compliance with
7+
* the License. You may obtain a copy of the License at
8+
*
9+
* http://www.apache.org/licenses/LICENSE-2.0
10+
*
11+
* Unless required by applicable law or agreed to in writing, software
12+
* distributed under the License is distributed on an "AS IS" BASIS,
13+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
* See the License for the specific language governing permissions and
15+
* limitations under the License.
16+
*/
17+
package org.apache.spark.util
18+
19+
import org.scalatest.funsuite.AnyFunSuite // scalastyle:ignore funsuite
20+
21+
class SparkCollectionUtilsSuite extends AnyFunSuite { // scalastyle:ignore funsuite
22+
23+
test("toMapWithIndex maps each key to its position, matching zipWithIndex.toMap") {
24+
assert(SparkCollectionUtils.toMapWithIndex(Seq("a", "b", "c")) ===
25+
Map("a" -> 0, "b" -> 1, "c" -> 2))
26+
assert(SparkCollectionUtils.toMapWithIndex(Seq.empty[String]) === Map.empty[String, Int])
27+
// Duplicate keys keep the last index, matching zipWithIndex.toMap.
28+
assert(SparkCollectionUtils.toMapWithIndex(Seq("a", "a")) === Map("a" -> 1))
29+
val keys = Seq(10, 20, 30, 40)
30+
assert(SparkCollectionUtils.toMapWithIndex(keys) === keys.zipWithIndex.toMap)
31+
}
32+
33+
test("isEmpty and isNotEmpty handle null, empty and non-empty maps") {
34+
val nullMap: java.util.Map[String, Int] = null
35+
assert(SparkCollectionUtils.isEmpty(nullMap))
36+
assert(!SparkCollectionUtils.isNotEmpty(nullMap))
37+
38+
val empty = new java.util.HashMap[String, Int]()
39+
assert(SparkCollectionUtils.isEmpty(empty))
40+
assert(!SparkCollectionUtils.isNotEmpty(empty))
41+
42+
val nonEmpty = new java.util.HashMap[String, Int]()
43+
nonEmpty.put("a", 1)
44+
assert(!SparkCollectionUtils.isEmpty(nonEmpty))
45+
assert(SparkCollectionUtils.isNotEmpty(nonEmpty))
46+
}
47+
48+
test("createArray fills primitive-typed arrays with the default value") {
49+
assert(SparkCollectionUtils.createArray(3, 7) === Array(7, 7, 7))
50+
assert(SparkCollectionUtils.createArray(2, true) === Array(true, true))
51+
assert(SparkCollectionUtils.createArray(2, 1L) === Array(1L, 1L))
52+
assert(SparkCollectionUtils.createArray(2, 2.5d) === Array(2.5d, 2.5d))
53+
assert(SparkCollectionUtils.createArray(2, 1.5f) === Array(1.5f, 1.5f))
54+
assert(SparkCollectionUtils.createArray(2, 3.toByte) === Array(3.toByte, 3.toByte))
55+
assert(SparkCollectionUtils.createArray(2, 4.toShort) === Array(4.toShort, 4.toShort))
56+
assert(SparkCollectionUtils.createArray(2, 'a') === Array('a', 'a'))
57+
assert(SparkCollectionUtils.createArray(0, 7).isEmpty)
58+
}
59+
60+
test("createArray fills reference-typed arrays and returns empty for size 0") {
61+
assert(SparkCollectionUtils.createArray(2, "x") === Array("x", "x"))
62+
assert(SparkCollectionUtils.createArray(0, "x").isEmpty)
63+
}
64+
}

0 commit comments

Comments
 (0)