-
Notifications
You must be signed in to change notification settings - Fork 1
378 lines (367 loc) · 17.7 KB
/
Copy pathci.yml
File metadata and controls
378 lines (367 loc) · 17.7 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
name: CI
on:
push:
branches: [main]
pull_request:
branches: [main]
# Tier B of the phase 4 gate is far too slow for the push path but it is the only
# thing that runs a full-length RFC vector through one encoder lifetime, so it runs
# nightly. See the gate-long job.
schedule:
- cron: '30 3 * * *'
workflow_dispatch:
env:
GO_VERSION: '1.27'
# Pinned to match the local toolchain so git hooks, `task lint`, and CI all
# run the identical linter version.
GOLANGCI_LINT_VERSION: 'v2.13.2'
permissions:
contents: read
jobs:
test:
name: Test
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-24.04-arm, ubuntu-latest]
include:
# The race detector is arch-agnostic in what it detects (a happens-before
# violation is the same bug on either arch), so it runs once, on the fast
# arm runner. amd64 runs the same suite WITHOUT -race: that still exercises
# amd64 codegen and the SSE2/AVX2 SIMD kernels (now in the tphakala/simd
# library, plus the SIMD-vs-scalar tests) without the race detector's 2-10x
# overhead on the slower runner.
# Coverage rides on the amd64 leg only (no -race there, so -covermode=count
# is enough) and uploads to codecov; the arm leg leaves `cover` empty. This
# keeps the coverage upload on the fast runner instead of a separate slow
# job, while the reftest job stays the real CI long pole either way.
- os: ubuntu-24.04-arm
race: '-race'
cover: ''
- os: ubuntu-latest
race: ''
cover: '-coverpkg=./... -covermode=count -coverprofile=coverage.out'
# The exhaustive SIMD differential sweep in internal/celt (pitch_simd_test.go)
# walks every length 0..600 against 12 adversarial patterns, which is the only
# place the hand-written vector kernels' scalar tail epilogue is checked at every
# remainder class. That breadth is deliberate but too slow to pay on the PR
# feedback loop, so pull_request runs it with -short (a boundary-focused subset
# that still hits every tail remainder). Every other trigger runs the full sweep:
# push to main verifies it on both arches on every merge, and the nightly schedule
# plus manual workflow_dispatch re-run it in full. -short only affects that sweep;
# no other test in this job gates on testing.Short(). See issue #15.
env:
SHORT_FLAG: ${{ github.event_name == 'pull_request' && '-short' || '' }}
steps:
- name: Checkout code
uses: actions/checkout@v7
with:
submodules: recursive
# No job here pushes, so the GITHUB_TOKEN has no business sitting in
# .git/config for the life of the job. The libopus submodule is a public
# https remote and needs no credential.
persist-credentials: false
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version: ${{ env.GO_VERSION }}
# The RFC 6716 / RFC 8251 conformance vectors are ~75 MB and are never
# committed (.gitignore), so without this step opus/conformance_test.go SKIPS
# and the RFC gate is not running on any push at all: 12/12 vectors passing
# locally, nothing enforcing it in CI. Only the archive is cached (the
# extracted 121 MB of PCM re-expands from it in a couple of seconds), and the
# cache key is the checksum file itself, so a re-pinned vector set invalidates
# the cache by construction. fetch-vectors.sh sha256-verifies before extracting
# and refuses to extract a mismatching archive.
- name: Cache conformance vectors
uses: actions/cache@v6
with:
path: testdata/vectors/opus_testvectors-rfc8251.tar.gz
key: opus-vectors-rfc8251-${{ hashFiles('testdata/vectors.sha256') }}
- name: Fetch conformance vectors
run: scripts/fetch-vectors.sh
- name: Download dependencies
run: go mod download
- name: Run tests
run: go test ${{ matrix.race }} ${{ matrix.cover }} $SHORT_FLAG -timeout 20m ./...
- name: Run codec tests with the scratch poison harness
run: go test -tags poison $SHORT_FLAG -count=1 -timeout 15m ./internal/celt/... ./internal/packet/... ./opus/...
# The SIMD differential suites diff each dispatcher against a scalar reference
# that is also the dispatcher's fallback, so a green suite cannot tell a working
# vector kernel from one that never ran (issue #72). The dispatchcount build
# arms a per-kernel counter at the exact library call site; the Dispatch tests
# then assert each dispatcher reaches its vector kernel (and, where an in-tree
# fallback exists, that the fallback shape does not). Runs on both arches.
- name: Verify the SIMD dispatchers reach their vector kernels
run: go test -tags dispatchcount -count=1 -run Dispatch ./internal/celt/
# Coverage is informational (codecov, fail_ci_if_error: false), never a gate.
# Only the amd64 leg sets `cover`, so it is the only leg with a coverage.out to
# upload; the arm leg skips this step.
- name: Upload coverage
if: matrix.os == 'ubuntu-latest'
uses: codecov/codecov-action@v7
with:
files: coverage.out
fail_ci_if_error: false
# Forced pure-Go SIMD fallback (#38). The pitch (i16), haar1/band-gain/comb
# (i32), and FFT butterfly (cint) kernels are backed by tphakala/simd, which runs
# hand-written NEON/SSE2/AVX2 on the arm64/amd64 CI hosts and a pure-Go
# fallback on every other GOARCH (riscv64, ppc64le, s390x, loong64, wasm). The
# `test` job therefore only ever exercises the assembly; the Go fallback's
# bit-exactness on those arches would otherwise rest solely on the simd
# library's own suite, not go-opus's differential gate. SIMD_DISABLE=all makes
# the simd cpu package clear every detected feature at init, so every dispatch
# falls to the exact Go functions the exotic arches run, letting the existing
# differential suites (pitch/haar1/band-gain/comb/FFT-cint) cover the fallback on a
# native amd64 runner. -count=1 is mandatory: the outcome depends on an env var
# the go test cache does not track, so a cached SIMD result would defeat the
# leg. TestSIMDFallbackForced asserts the features really were cleared, so a
# green run that quietly kept running SIMD fails instead of passing hollow.
forcego:
name: Test (forced Go SIMD fallback)
runs-on: ubuntu-latest
env:
SIMD_DISABLE: all
GOOPUS_ASSERT_SIMD_DISABLED: '1'
steps:
- name: Checkout code
uses: actions/checkout@v7
with:
persist-credentials: false
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version: ${{ env.GO_VERSION }}
- name: Download dependencies
run: go mod download
# Full sweep (no -short): the exhaustive length 0..600 differential walk is
# the only place the fallback's scalar tail epilogue is checked at every
# remainder class, and it is the whole point of this leg, so it runs in full
# on every trigger. It is a single pure-Go package, so the cost is small.
- name: Run the celt differential suite against the forced Go fallback
run: go test -count=1 -timeout 20m ./internal/celt/...
# Differential round-trip against the pinned libopus oracle (FIXED_POINT +
# DISABLE_FLOAT_API). The cgo harness compiles the vendored libopus sources
# directly, so only a C compiler is needed (no autotools). This job carries the
# phase 4 encoder gate; it runs on amd64 and arm64.
reftest:
name: Reftest (libopus oracle)
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, ubuntu-24.04-arm]
steps:
- name: Checkout code
uses: actions/checkout@v7
with:
submodules: recursive
# No job here pushes, so the GITHUB_TOKEN has no business sitting in
# .git/config for the life of the job. The libopus submodule is a public
# https remote and needs no credential.
persist-credentials: false
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version: ${{ env.GO_VERSION }}
# Persist the cgo-compiled libopus objects across runs. setup-go's built-in
# cache keys on go.sum, which is identical for every job, so the first job to
# finish claims that key and the reftest leg's refc cgo objects (built only
# here, behind CGO_ENABLED=1 + the refc tag) never get saved, forcing a full
# libopus recompile every run. A dedicated key fixes that. Correctness does
# not rest on the key: Go's build cache is content-addressed on the exact C
# sources, cgo flags AND the C compiler binary, so a stale entry is rehashed
# and rebuilt, never linked. The key is still conservative (libopus submodule
# commit, the shim sources and cgo CFLAGS, and the cc version) so a submodule
# bump, a shim edit, or a runner GCC bump misses cleanly; without the cc
# version a compiler bump would let Go rebuild while actions/cache kept the old
# key and never re-saved, stranding the job in a permanent recompile. The
# restore-keys fall back to a partial cache the rebuild reuses.
- name: Resolve reftest cache-key inputs
id: cachekey
run: |
{
echo "libopus=$(git -C internal/reftest/libopus rev-parse HEAD)"
echo "cc=$("$(go env CC)" -dumpfullversion)"
echo "gocache=$(go env GOCACHE)"
} >> "$GITHUB_OUTPUT"
- name: Cache libopus cgo build objects
uses: actions/cache@v6
with:
path: ${{ steps.cachekey.outputs.gocache }}
key: reftest-cgo-${{ runner.os }}-${{ runner.arch }}-go${{ env.GO_VERSION }}-cc${{ steps.cachekey.outputs.cc }}-libopus${{ steps.cachekey.outputs.libopus }}-${{ hashFiles('internal/reftest/oracle/*.c', 'internal/reftest/oracle/*.h', 'internal/reftest/oracle/oracle_cgo.go') }}
restore-keys: |
reftest-cgo-${{ runner.os }}-${{ runner.arch }}-go${{ env.GO_VERSION }}-cc${{ steps.cachekey.outputs.cc }}-libopus${{ steps.cachekey.outputs.libopus }}-
reftest-cgo-${{ runner.os }}-${{ runner.arch }}-go${{ env.GO_VERSION }}-cc${{ steps.cachekey.outputs.cc }}-
# The vectors unlock the gate's REAL-AUDIO tier (corpus_test.go tier 2 reads
# testvectorNN.dec as 48 kHz PCM) and the packet-inspection differential's RFC
# corpus. Both t.Skip cleanly without them, which is precisely the failure mode
# this step removes: a gate that quietly grades itself on synthetic signal only.
- name: Cache conformance vectors
uses: actions/cache@v6
with:
path: testdata/vectors/opus_testvectors-rfc8251.tar.gz
key: opus-vectors-rfc8251-${{ hashFiles('testdata/vectors.sha256') }}
- name: Fetch conformance vectors
run: scripts/fetch-vectors.sh
- name: Run differential harness
env:
CGO_ENABLED: '1'
run: go test -tags "refc poison" -timeout 15m ./internal/reftest/...
gate-long:
name: Phase 4 gate, tier B (full-length corpus)
# Tier A of the gate runs on every push, inside the reftest job, over 2 s excerpts.
# Tier B is the only thing that pushes a full-length RFC vector (21 to 32 s) through
# a single encoder lifetime, which is where a one-LSB error in the delay ring, the
# dc_reject memory, the CELT energy history or the VBR reservoir would finally
# accumulate into a divergence. Without this job that tier is dead code: it skips
# unless -gate.long is passed, and nothing passed it.
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v7
with:
submodules: recursive
# No job here pushes, so the GITHUB_TOKEN has no business sitting in
# .git/config for the life of the job. The libopus submodule is a public
# https remote and needs no credential.
persist-credentials: false
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version: ${{ env.GO_VERSION }}
- name: Cache conformance vectors
uses: actions/cache@v6
with:
path: testdata/vectors/opus_testvectors-rfc8251.tar.gz
key: opus-vectors-rfc8251-${{ hashFiles('testdata/vectors.sha256') }}
# Tier B IS the real audio, so a missing corpus is a failure here, not a skip.
- name: Fetch conformance vectors
run: scripts/fetch-vectors.sh
- name: Run tier B
env:
CGO_ENABLED: '1'
run: |
go test -tags refc -count=1 -timeout 150m -v \
./internal/reftest/oracle/ -run 'TestOpusencPhase4GateLong' -gate.long
lint:
name: Lint
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v7
with:
submodules: recursive
# No job here pushes, so the GITHUB_TOKEN has no business sitting in
# .git/config for the life of the job. The libopus submodule is a public
# https remote and needs no credential.
persist-credentials: false
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version: ${{ env.GO_VERSION }}
# skip-cache because the action's shared analysis cache poisons this job. On
# 2026-09-02 every run, on main and on every PR branch, began failing with
# four SA5011 "possible nil pointer dereference" reports in test files whose
# nil check is a t.Fatalf immediately above the use. The same commit had
# passed hours earlier, reruns failed identically, and every failing run
# logged a hit on the same restored cache key. Running the pinned linter
# (v2.12.2) against the pinned Go (1.26.7) on a clean cache reports zero
# issues for both main and the branch, so the findings come from the cache
# rather than from the code. A lint gate has to reflect the tree; the cache
# saves seconds on a job the reftest dwarfs anyway.
- name: Run golangci-lint
uses: golangci/golangci-lint-action@v9
with:
version: ${{ env.GOLANGCI_LINT_VERSION }}
skip-cache: true
vet:
name: Vet
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
goarch: [amd64, arm64]
steps:
- name: Checkout code
uses: actions/checkout@v7
with:
submodules: recursive
# No job here pushes, so the GITHUB_TOKEN has no business sitting in
# .git/config for the life of the job. The libopus submodule is a public
# https remote and needs no credential.
persist-credentials: false
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version: ${{ env.GO_VERSION }}
- name: Run go vet
env:
GOARCH: ${{ matrix.goarch }}
run: go vet ./...
build:
name: Build
runs-on: ubuntu-latest
strategy:
matrix:
goos: [linux, darwin, windows]
goarch: [amd64, arm64]
steps:
- name: Checkout code
uses: actions/checkout@v7
with:
submodules: recursive
# No job here pushes, so the GITHUB_TOKEN has no business sitting in
# .git/config for the life of the job. The libopus submodule is a public
# https remote and needs no credential.
persist-credentials: false
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version: ${{ env.GO_VERSION }}
- name: Build
env:
GOOS: ${{ matrix.goos }}
GOARCH: ${{ matrix.goarch }}
run: go build ./...
fuzz:
name: Fuzz smoke
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v7
with:
persist-credentials: false
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version: ${{ env.GO_VERSION }}
# A short bounded run of each decode-path fuzz target on every build. The
# packet and Ogg readers parse untrusted bytes, so this guards against
# panics and unbounded allocation on malformed input; a crash a run finds
# is committed under testdata/fuzz as a regression seed. The last four targets
# fuzz a SIMD kernel against its scalar reference. Smoke runs, not a
# substitute for a long local fuzz session. The libopus submodule is not
# needed: these targets are pure Go, the C reference is build-tagged out.
- name: Fuzz Opus packet parser
run: go test -run='^$' -fuzz='^FuzzParsePacket$' -fuzztime=20s ./internal/packet
- name: Fuzz Opus decode
run: go test -run='^$' -fuzz='^FuzzDecode$' -fuzztime=20s ./opus
- name: Fuzz Ogg page reader
run: go test -run='^$' -fuzz='^FuzzReadPage$' -fuzztime=15s ./oggopus
- name: Fuzz OpusHead parser
run: go test -run='^$' -fuzz='^FuzzParseOpusHead$' -fuzztime=15s ./oggopus
- name: Fuzz OpusTags parser
run: go test -run='^$' -fuzz='^FuzzParseOpusTags$' -fuzztime=15s ./oggopus
- name: Fuzz Ogg container reader
run: go test -run='^$' -fuzz='^FuzzContainerReader$' -fuzztime=15s ./oggopus
- name: Fuzz xcorr SIMD differential
run: go test -run='^$' -fuzz='^FuzzXcorrKernel$' -fuzztime=15s ./internal/celt
- name: Fuzz comb filter SIMD differential
run: go test -run='^$' -fuzz='^FuzzCombFilterConst$' -fuzztime=15s ./internal/celt
- name: Fuzz haar1 SIMD differential
run: go test -run='^$' -fuzz='^FuzzHaar1$' -fuzztime=15s ./internal/celt
- name: Fuzz band-gain SIMD differential
run: go test -run='^$' -fuzz='^FuzzBandGainRequant$' -fuzztime=15s ./internal/celt