Commit 0875765
[SPARK-59173][ML] Use isEmpty/nonEmpty instead of size comparisons in MLlib
### What changes were proposed in this pull request?
This replaces size-vs-zero comparisons with the direct emptiness / non-emptiness predicates in the `mllib` module:
- **Emptiness** (`x.size == 0` -> `x.isEmpty`): `TreeEnsembleModel.totalImportances` (an `OpenHashMap`, which `extends Iterable`) and two `Seq[ShuffleDependency]` checks in `ALSSuite`.
- **Non-emptiness** (`x.size > 0` -> `x.nonEmpty`): `DecisionTreeMetadata.featureArity` and `Strategy.categoricalFeaturesInfo`, both Scala `Map[Int, Int]`.
There are no Java-collection cases in this module.
Deliberately left unchanged, because they are **not** collection-emptiness checks -- they are `mllib.linalg.Vector` dimensions or numeric sizes/thresholds, and `Vector` has no `isEmpty`/`nonEmpty`:
- `ChiSqTest` (`expected` / `observed` are `Vector` dimensions, compared with `!= 0` and `== 0.0`);
- `BinaryClassificationPMMLModelExport` and `GeneralizedLinearPMMLModelExport` (`model.weights`, a `Vector`);
- `MultivariateOnlineSummarizer` (`instance`, a `Vector` -- the message reads "Vector should have dimension larger than zero");
- `BlockMatrix` (a matrix row `Vector`);
- `ALS` (`RatingBlockBuilder.size`, a custom builder with no `nonEmpty`);
- `SpearmanCorrelation` (`cachedUids.size >= 10000000`, a size threshold, not an emptiness check).
### Why are the changes needed?
`isEmpty` / `nonEmpty` state the intent directly. Behavior is unchanged: every converted receiver (`OpenHashMap`, `Seq`, Scala `Map`) defines the predicate as exactly equivalent to the original comparison.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Existing tests. This is a behavior-preserving refactor; the `mllib` module compiles cleanly. Counterpart of SPARK-59149 (SQL) and SPARK-59172 (core).
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Opus 4.8)
Closes #58471 from uros-b/mllib-isempty.
Authored-by: Uros <221401595+uros-b@users.noreply.github.com>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>1 parent db997b3 commit 0875765
4 files changed
Lines changed: 5 additions & 5 deletions
File tree
- mllib/src
- main/scala/org/apache/spark
- mllib/tree/configuration
- ml/tree
- impl
- test/scala/org/apache/spark/ml/recommendation
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
66 | 66 | | |
67 | 67 | | |
68 | 68 | | |
69 | | - | |
| 69 | + | |
70 | 70 | | |
71 | 71 | | |
72 | 72 | | |
| |||
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
229 | 229 | | |
230 | 230 | | |
231 | 231 | | |
232 | | - | |
| 232 | + | |
233 | 233 | | |
234 | 234 | | |
235 | 235 | | |
| |||
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
98 | 98 | | |
99 | 99 | | |
100 | 100 | | |
101 | | - | |
| 101 | + | |
102 | 102 | | |
103 | 103 | | |
104 | 104 | | |
| |||
Lines changed: 2 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
995 | 995 | | |
996 | 996 | | |
997 | 997 | | |
998 | | - | |
999 | | - | |
| 998 | + | |
| 999 | + | |
1000 | 1000 | | |
1001 | 1001 | | |
1002 | 1002 | | |
| |||
0 commit comments