Skip to content

Commit 1fba3cc

Browse files
committed
[SPARK-58413][SQL][TEST] Rename sequential fetch label to pipelined fetch (1 client) in NettyTransportBenchmark
### What changes were proposed in this pull request? This PR renames the benchmark case label `sequential fetch` to `pipelined fetch (1 client)` in the `File-Backed Shuffle Block Fetch` suite of `NettyTransportBenchmark`. ### Why are the changes needed? The label is misleading. The case fires all 100 `fetchChunk` requests at once on a single connection and then waits for all of them to complete via a semaphore (`fetchChunksSync`), so the client never waits for one chunk before requesting the next. This is a pipelined fetch over one connection, not a sequential request-response loop. The mislabel skews the interpretation of the results. For example: https://github.com/apache/spark/blob/12785d56624a8b412e78559dd920b816d629704b/core/benchmarks/NettyTransportBenchmark-results.txt#L132-L133 Read as "sequential vs parallel", the 1.8X looks like poor parallelization efficiency. The actual comparison is "pipelined over 1 socket vs pipelined over 4 sockets", where the single pipelined connection already runs at ~4.2 GB/s. The new label makes this clear and contrasts naturally with `parallel fetch (4 clients)`. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Label-only change; manually reviewed. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Fable 5 Closes #57616 from dongjoon-hyun/SPARK-58413. Authored-by: Dongjoon Hyun <dongjoon@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
1 parent 6b793ce commit 1fba3cc

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

core/src/test/scala/org/apache/spark/network/NettyTransportBenchmark.scala

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -599,7 +599,7 @@ object NettyTransportBenchmark extends BenchmarkBase {
599599
output = output)
600600

601601
Seq("NIO", "AUTO").foreach { mode =>
602-
benchmark.addTimerCase(s"$mode, sequential fetch", numIters = 3) { timer =>
602+
benchmark.addTimerCase(s"$mode, pipelined fetch (1 client)", numIters = 3) { timer =>
603603
val conf = createConf(mode)
604604
val streamManager = createFileStreamManager(conf, files)
605605
val rpcHandler = createStreamRpcHandler(streamManager)

0 commit comments

Comments
 (0)