fix(licenses): stdlib ignore-list prefix collision drops 31 modules from THIRD_PARTY_NOTICES

[lockwobr]

Summary

scripts/generate-notices.py builds the go-licenses --ignore list from the first path segment of every stdlib package. That yields the bare token go (from go/ast, go/build, and friends), and --ignore matches by prefix, so go also matches every vendored module whose path starts with those two characters.

31 of the operator's 106 vendored modules are silently dropped from operator/THIRD_PARTY_NOTICES.md (and therefore from the root rollup). They are redistributed in the released operator image, so their licenses have to be disclosed.

No error is raised. The tool is asked about the packages, told to ignore them, and reports nothing.

Root cause

scripts/generate-notices.py:65:

stdlib_ignore = ",".join(sorted({line.split("/")[0] for line in stdlib.splitlines() if line}))

line.split("/")[0] reduces go/ast to go. Passing full stdlib package paths cannot collide that way, because no stdlib package path is a prefix of a third-party module path.

Reproduction

$ cd operator
$ go list std | cut -d/ -f1 | sort -u > /tmp/tokens
$ grep '^# ' vendor/modules.txt | awk '{print $2}' | sort -u > /tmp/mods
$ awk 'NR==FNR{t[NR]=$0;n=NR;next}{for(i=1;i<=n;i++) if(index($0,t[i])==1){print t[i]" -> "$0; break}}' /tmp/tokens /tmp/mods

All 31 hits come from the single token go:

go -> go.opentelemetry.io/auto/sdk
go -> go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp
go -> go.opentelemetry.io/otel
go -> go.opentelemetry.io/otel/exporters/otlp/otlptrace
go -> go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc
go -> go.opentelemetry.io/otel/metric
go -> go.opentelemetry.io/otel/sdk
go -> go.opentelemetry.io/otel/trace
go -> go.opentelemetry.io/proto/otlp
go -> go.uber.org/multierr
go -> go.uber.org/zap
go -> go.yaml.in/yaml/v2
go -> go.yaml.in/yaml/v3
go -> golang.org/x/exp
go -> golang.org/x/mod
go -> golang.org/x/net
go -> golang.org/x/oauth2
go -> golang.org/x/sync
go -> golang.org/x/sys
go -> golang.org/x/term
go -> golang.org/x/text
go -> golang.org/x/time
go -> golang.org/x/tools
go -> gomodules.xyz/jsonpatch/v2
go -> google.golang.org/genproto/googleapis/api
go -> google.golang.org/genproto/googleapis/rpc
go -> google.golang.org/grpc
go -> google.golang.org/protobuf
go -> gopkg.in/evanphx/json-patch.v4
go -> gopkg.in/inf.v0
go -> gopkg.in/yaml.v3

Confirming all 31 are absent from the committed notices file:

$ while read -r m; do grep -q "\`$m\`" operator/THIRD_PARTY_NOTICES.md || echo "MISSING $m"; done < <(...)
# 31 / 31 missing

Spot checks against the current file:

module group	vendored packages	mentions in operator/THIRD_PARTY_NOTICES.md
go.opentelemetry.io	59	0
google.golang.org	see above	0
golang.org/x	see above	0
go.uber.org	12	0
go.yaml.in	4	0

Why it stayed hidden

Two things would each have caught this:

1. No completeness gate. Nothing asserts that every package linked into the released binaries is covered by an entry. The pipeline trusts go-licenses to report on what it was asked about, and here it correctly reported nothing, because it was told to ignore them.
2. No freshness gate. There is no notices-check equivalent in CI, so a stale or incomplete file never fails a build. Adding one is currently blocked by a second problem: the generator stamps Generated: {now_utc()} into all four outputs (scripts/generate-notices.py:87,120,150,208), so regenerate-and-diff can never be byte-stable.

Proposed fix

1. Pass full stdlib paths. Drop the .split("/")[0]; join the raw go list std output. One-line change, recovers all 31.
2. Add a completeness gate. After collection, compute go list -deps ./... and assert every third-party package is covered by some index entry, matched by prefix (the index is keyed by module, the dependency list by package). Fail the build naming the uncovered packages. Two details worth copying: skip index keys with no slash, because go-licenses emits a bare github.com row when it cannot resolve a package to a module and that single row would match everything under the host and make the check vacuous; and verify the gate in the negative direction, not just the happy path.
3. Drop the Generated: timestamp, then add a notices-check target that regenerates and git diff --exit-codes, wired into the merge gate on dependency changes.
4. Consider handling unclassifiable licenses explicitly. go-licenses save exits non-zero on a license it cannot classify, which is a separate way for disclosure to silently go missing if anyone works around it by widening --ignore.

Prior art

NVIDIA/aicr had the identical bug in tools/generate-notices and is fixing it in NVIDIA/aicr#2384 (31 modules recovered there too, coincidentally the same count). That PR carries the ignore-list fix, the completeness gate including the bare-github.qkg1.top subtlety, and the negative test. AICR's generator already emits deterministic output and has a notices-check merge gate, so items 1 through 3 above can largely be lifted from it.

Note that AICR's version also unions across GOOS/GOARCH because its binaries ship for darwin+linux x amd64+arm64 and build-tagged sources import different transitive dependencies per platform. If the nodewright operator image is linux-only, that part is not needed here.

Scope

• Affects operator/THIRD_PARTY_NOTICES.md and the root rollup.
• agent/THIRD_PARTY_NOTICES.md is Python (pip-licenses) and is not affected by this bug.