Skip to content

Commit dacfdba

Browse files
committed
[SPARK-58996][SQL][FOLLOWUP] Fix the subset join key test for the branch-4.3 backport
Sets `spark.sql.sources.v2.bucketing.requireAllClusterKeysForCoPartition` to false in the "partially clustered subset-key join keeps its join key positions" test, which is what every other subset test in this suite already does on this branch. Without the config `createKeyedShuffleSpec` refuses the subset shape (join key `id` against partition keys `(extra, id)`), both sides are shuffled, and the no-shuffle assertion fails. On master the test plans without the config because SPARK-58558 relaxed the check from exact key match to key coverage; it is not on this branch. No user-facing change. Assisted-by: Claude Fable 5
1 parent 4f26228 commit dacfdba

1 file changed

Lines changed: 4 additions & 0 deletions

File tree

sql/core/src/test/scala/org/apache/spark/sql/connector/KeyGroupedPartitioningSuite.scala

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3669,6 +3669,10 @@ class KeyGroupedPartitioningSuite extends DistributionAndOrderingSuiteBase with
36693669
sql("INSERT INTO testcat.ns.np2 VALUES (7, 'y')")
36703670

36713671
withSQLConf(
3672+
// SPARK-58558 (4.4.0) relaxed `createKeyedShuffleSpec` from exact key match to key
3673+
// coverage; it is not in this branch, so the subset shape needs the config off, as every
3674+
// other subset test here does.
3675+
SQLConf.REQUIRE_ALL_CLUSTER_KEYS_FOR_CO_PARTITION.key -> "false",
36723676
SQLConf.V2_BUCKETING_PUSH_PART_VALUES_ENABLED.key -> "true",
36733677
SQLConf.V2_BUCKETING_PARTIALLY_CLUSTERED_DISTRIBUTION_ENABLED.key -> "true",
36743678
SQLConf.V2_BUCKETING_ALLOW_KEYS_SUBSET_OF_PARTITION_KEYS.key -> "true",

0 commit comments

Comments
 (0)