Skip to content

Swap back to a blocking udp socket, test shutdown(2) - #1806

Merged
nbrownus merged 2 commits into
masterfrom
udp-blocking-shutdown
Jul 17, 2026
Merged

Swap back to a blocking udp socket, test shutdown(2)#1806
nbrownus merged 2 commits into
masterfrom
udp-blocking-shutdown

Conversation

@nbrownus

@nbrownus nbrownus commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

Before #1654 the Linux UDP reader used a raw blocking socket with a direct recvmmsg loop. #1654 switched to a net.UDPConn + syscall.RawConn netpoller path to get a clean shutdown. On many-core hosts, netpoller constantly parks and wakes the reader goroutines, and the resulting scheduler churn (findRunnable/lock2/futex) roughly halves UDP throughput. A 16-routine test on a 64 vCPU host fell from ~28.5 Gbps to ~13 Gbps, scaling routines beyond 8 made each perf test progressively worse.

This restores the blocking read, which parks the worker thread in the kernel and brings throughput back to ~28.5 Gbps with the churn gone entirely. Close() sets a closed flag and calls shutdown(2), and the read helpers translate the resulting wake into net.ErrClosed, matching the darwin/generic/windows paths.

The subtlety that sank an earlier blocking-plus-shutdown attempt (#1591, part of the now-closed #1375) is that once a socket has received a packet, shutdown(2) wakes recvmmsg with n>=1/Len==0 rather than n==0, so keying teardown off the message count spins forever; keying it off the closed flag handles every wake shape.

Also added some tests to make sure we correctly shutdown on linux in all situations.

I left runtime.LockOSThread() out because it increased scheduler pressure and made per packet cpu utilization go up in my testing.

@nbrownus nbrownus added this to the v1.11.0 milestone Jul 17, 2026
@nbrownus nbrownus added the smoke-test-extra Run extra smoke tests (freebsd, 32-bit, ipv4 only, etc) label Jul 17, 2026
Comment thread udp/udp_linux.go Outdated
JackDoan
JackDoan previously approved these changes Jul 17, 2026
Co-authored-by: Jack Doan <me@jackdoan.com>
@nbrownus
nbrownus merged commit 147c202 into master Jul 17, 2026
23 checks passed
@nbrownus
nbrownus deleted the udp-blocking-shutdown branch July 17, 2026 20:16
@nbrownus nbrownus mentioned this pull request Jul 23, 2026
56 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla:signed smoke-test-extra Run extra smoke tests (freebsd, 32-bit, ipv4 only, etc)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants