otelcol/grpclog: suppress benign gRPC client-disconnect warnings

[kruthiwusirika5]

Fixes #5169

Problem

When a gRPC client disconnects normally (e.g., a .NET app shutting
down its TracerProvider), the collector logs spurious WARN messages
from the gRPC transport layer:

These are expected during graceful client shutdown but appear
alarming to operators and pollute production logs.

Root cause

grpc-go is doing the right thing: messages like this are gated
behind LoggerV2.V(2) in internal/transport/http2_server.go,
i.e. they are explicitly classified as supplemental verbosity, not
severity-level warnings. They should never reach a WARN sink.

The bug is in zapgrpc. zapgrpc.Logger.V is implemented as
levelEnabler.Enabled(zapcore.Level(level - 1)), which conflates
grpclog's verbosity with zap's severity. With WARN enabled,
V(2) returns true, so grpc-go thinks verbosity level 2 is
permitted and emits the message — which then gets formatted by
zapgrpc as a WARN entry. See uber-go/zap#1544.

Solution

Wrap the installed grpclog.LoggerV2 with a fixedVerbosityLogger
whose V() compares the requested level against a fixed threshold
(default 0) instead of the underlying zap severity enabler. This
restores grpclog's intended verbosity gating, so verbose messages
stop at grpc-go before formatting rather than being filtered out
of the log stream after the fact.

The default threshold of 0 matches grpclog when
GRPC_GO_LOG_VERBOSITY_LEVEL is unset.

This approach:

• Addresses the root cause (broken V() mapping) rather than
  string-matching specific message contents.
• Stops chatty messages at the source, before grpc-go formats them.
• Is generic — kills all V(2)+ noise, including future grpc-go
  additions, not just two cherry-picked patterns.
• Leaves severity-level emission (Info / Warning / Error)
  completely unchanged.
• Tiny: ~10 lines of wrapper plus comment.

Testing

• TestFixedVerbosity — asserts V(0) is enabled and V(1)/V(2)
  are not at the default threshold.
• TestSeverityLevelsUnaffected — direct Warning("…") still
  fires at WARN, confirming the wrapper does not gate severity.
• Existing TestGRPCLogger cases continue to pass unchanged.

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: kruthiwusirika5 / name: Sai Kruthi Wusirika (008f6cd, 0570f24, 6b4fa6d, b37adb8)

[kruthiwusirika5]

recheck

[kruthiwusirika5]

/easycla

[kruthiwusirika5]

/easycla

[axw]

This feels like a bit of a heavy-handed solution.

grpc-go already has a verbosity check for these kinds of errors, e.g. see https://github.com/grpc/grpc-go/blob/d574bad188f25ba03d41a506e6f2ef93837ad10b/internal/transport/http2_server.go#L642

The zapgrpc implementation of the V method appears to be incorrect: https://github.com/uber-go/zap/blob/0ab0d5aae5986395e2ca497385d977ccd7cdfc5e/zapgrpc/zapgrpc.go#L234-L237

e.g. verbosity level 2 is being mapped to error level, which is nonsense for two reasons: error level logs should be less verbose than info, not more; grpc-go's verbosity is meant to be independent of the primary logger level.

So either we fix that upstream, or wrap the zapgrpc logger.

[axw]

uber-go/zap#1544

[kruthiwusirika5]

@axw thanks for the pointer to uber-go/zap#1544 — that crystallized it. Pushed a redesign that wraps the grpclog.LoggerV2 with a corrected V() instead of filtering log output. The chatty V(2)-gated transport messages now stop at grpc-go before formatting, rather than being string-matched out of the formatted stream. Severity-level emission is unaffected. PTAL when you have a moment.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.13%. Comparing base (efde8a2) to head (0570f24).
⚠️ Report is 13 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main   #15024   +/-   ##
=======================================
  Coverage   91.13%   91.13%           
=======================================
  Files         701      701           
  Lines       45745    45783   +38     
=======================================
+ Hits        41689    41724   +35     
- Misses       2843     2845    +2     
- Partials     1213     1214    +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[axw]

@kruthiwusirika5 code looks good, thanks. Please create a changelog entry (make chlog-new).

[kruthiwusirika5]

@axw chloggen entry added in 008f6cd1f — bug_fix against pkg/otelcol, links #5169 and explains the underlying uber-go/zap#1544 for context. make chlog-validate passes locally.

Workflows on the new commit are sitting in action_required (first-time contributor gate) — could you trigger them? The previously-failing changelog check should flip green once it re-runs against this SHA.

@open-telemetry/collector-approvers — this PR is now ready for a final pass. Code redesign was acked by @axw two days ago; just the changelog entry was outstanding.

[kruthiwusirika5]

@axw spell-check fix pushed in 6b4fa6d34 — added grpclog, grpclog's, and zapgrpc to .github/workflows/utils/cspell.json (the three identifiers from the chloggen entry that tripped the strict-mode check). Followed the existing convention of including both bare and possessive forms (cf. kafkaexporter / kafkaexporter's).

The two pre-existing lifecycles warnings in CHANGELOG.md / CHANGELOG-API.md are not addressed here — they currently fail on main itself (last 3 Spell Check runs on main are failing for the same reason) so they're out of scope for this PR.

Workflows on the new commit are sitting in action_required again — could you give them the first-time-contributor approval so the spell-check job can re-run? Should be the only outstanding blocker.

[codspeed-hq]

Merging this PR will not alter performance

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

✅ 7 untouched benchmarks
⏩ 76 skipped benchmarks¹

---

_{Comparing kruthiwusirika5:fix/zapgrpc-client-close-warning (0570f24) with main (c64a66b)}

Footnotes

1. 76 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[kruthiwusirika5]

@axw the spell-check rerun on 6b4fa6d34 revealed two remaining lifecycles warnings in CHANGELOG.md / CHANGELOG-API.md — both pre-existing and currently failing on main itself. Pushed 0570f24c which adds lifecycles to cspell.json to unblock this PR; commit message explains the scope. Workflows on the new commit need first-time-contributor approval again — sorry for the back-and-forth.

[otelbot]

Thank you for your contribution @kruthiwusirika5! 🎉 We would like to hear from you about your experience contributing to OpenTelemetry by taking a few minutes to fill out this survey.