Skip to content

stream: serverStream has 2 scattered bools wasting 8B and causing false sharing with mu #9349

Description

@gidotencate

Summary

`serverStream` (in `stream.go`) has two `bool` fields placed between aligned fields. Each bool forces alignment padding before the next field, wasting 8 bytes total. Additionally, `serverHeaderBinlogged` at offset 240 shares a 64-byte cache line with `mu` at offset 244, causing false-sharing overhead on every `mu.Lock()` under concurrent load.

This is the same pattern fixed for `clientStream` in #9280 / #9281.

Layout (DWARF- and runtime-verified)

Current size: 256B → 256B allocator size class

Field Offset Padding after Guarded by mu?
`recvFirstMsg` 168 7 B (before 8B-aligned `int`) No — "set after the first message is received"
`serverHeaderBinlogged` 240 3 B (before `sync.Mutex`, align 4) No — "doesn't need to be synchronized"

`mu` at offset 244 "protects trInfo.tr after the service handler runs." `serverHeaderBinlogged` is written in `SendHeader()` and `Send()` on every binlogged stream — frequently, and without holding `mu`.

`serverHeaderBinlogged` (offset 240) and `mu` (offset 244) share the same 64-byte cache line (bytes 192–255).

After grouping bools at tail + moving mu to cache line 2

Metric Current Proposed
Struct size 256 B 248 B
Allocator size class 256 B 256 B (no change)
Savings per `serverStream` 8 B
`mu` cache line CL3 (192–255) CL2 (128–191)
`serverHeaderBinlogged` cache line CL3 CL3 (tail, separated from `mu`)

Cache-line false-sharing benchmark

One goroutine spins `mu.Lock()`/`mu.Unlock()` while N stressor goroutines spin-write `serverHeaderBinlogged`:

goos: linux / goarch: amd64 / cpu: AMD Ryzen 7 7800X3D

Original layout (serverHeaderBinlogged @ 240, mu @ 244 — both cache line 3):
BenchmarkLayoutMuLockServerStream/stressors=0-16     3.94 ns/op   (baseline)
BenchmarkLayoutMuLockServerStream/stressors=1-16    20.61 ns/op   5.2× slower
BenchmarkLayoutMuLockServerStream/stressors=4-16    81.89 ns/op  20.8× slower

After tail-group only (serverHeaderBinlogged @ 241, mu @ 232 — still both cache line 3):
BenchmarkLayoutMuLockServerStream/stressors=1-16    20.11 ns/op   5.1× (unchanged)

After moving mu to cache line 2 (mu @ 184 CL2, serverHeaderBinlogged @ 241 CL3):
BenchmarkLayoutMuLockServerStream/stressors=0-16     3.7 ns/op
BenchmarkLayoutMuLockServerStream/stressors=1-16     3.8 ns/op   ~1×
BenchmarkLayoutMuLockServerStream/stressors=4-16     3.7 ns/op   ~1×

Proposed fix

Group both bools at the tail of `serverStream`, adding a `// Not guarded by mu` section with per-field comments. Move `mu` to immediately before `trInfo` (which it guards) so it lands on cache line 2 (offset 184), separate from the tail bools on cache line 3 (offset 240+).

/cc @easwars @mbissa

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type: PerformancePerformance improvements (CPU, network, memory, etc)

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions