Skip to content

Commit 726de2d

Browse files
yhakbaryhakbar
committed
chore: Adding sandboxed unit tests in CI
1 parent fd3df71 commit 726de2d

6 files changed

Lines changed: 196 additions & 7 deletions

File tree

‎.github/scripts/ci/no-network-exec.sh‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
#!/usr/bin/env bash
2+
set -euo pipefail
3+
4+
# Run a command with no network reach beyond loopback.
5+
#
6+
# Written for `go test -exec`, which splits the flag value on spaces, so this
7+
# has to be a single path taking the test binary and its arguments:
8+
#
9+
# go test -exec "$PWD/.github/scripts/ci/no-network-exec.sh" ./...
10+
#
11+
# Loopback stays reachable so tests can stand up httptest servers. Everything
12+
# else is refused, which turns any unvirtualized network call in the unit suite
13+
# into a test failure.
14+
#
15+
# With --check, confirm the sandbox is actually in force.
16+
PROFILE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/no-network.sb"
17+
18+
run_sandboxed() {
19+
case "$(uname -s)" in
20+
Darwin)
21+
exec sandbox-exec -f "$PROFILE" "$@"
22+
;;
23+
Linux)
24+
# A fresh network namespace starts with `lo` down, so bring it up before
25+
# handing over to the real command.
26+
exec unshare --map-root-user --net -- \
27+
sh -c 'ip link set lo up && exec "$@"' sh "$@"
28+
;;
29+
*)
30+
echo "no-network-exec.sh: no sandbox mechanism for $(uname -s)" >&2
31+
exit 1
32+
;;
33+
esac
34+
}
35+
36+
run_check() {
37+
local script="${BASH_SOURCE[0]}"
38+
39+
if ! command -v curl >/dev/null 2>&1; then
40+
echo "no-network-exec.sh: --check needs curl" >&2
41+
exit 1
42+
fi
43+
44+
# A sandbox that cannot start makes every command under it fail, which would
45+
# otherwise read as "egress is blocked" below and hand back a green run.
46+
if ! "$script" true; then
47+
echo "no-network-exec.sh: check failed, the sandbox cannot run a command at all" >&2
48+
exit 1
49+
fi
50+
51+
if "$script" curl --silent --show-error --max-time 15 --output /dev/null https://example.com; then
52+
echo "no-network-exec.sh: check failed, https://example.com is still reachable inside the sandbox" >&2
53+
exit 1
54+
fi
55+
56+
echo "no-network-exec.sh: check passed, egress is blocked"
57+
}
58+
59+
if [[ "${1:-}" == "--check" ]]; then
60+
run_check
61+
exit 0
62+
fi
63+
64+
run_sandboxed "$@"

‎.github/scripts/ci/no-network.sb‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
;; macOS seatbelt profile that denies a process (and everything it spawns) any
2+
;; network reach beyond loopback. Loopback stays open so tests can stand up
3+
;; httptest servers and talk to them; every other socket operation is refused.
4+
;;
5+
;; Unix domain sockets stay open because the platform reaches system daemons
6+
;; (mDNSResponder, notifyd) through them. Name resolution therefore still
7+
;; succeeds; connecting to what it resolves does not.
8+
(version 1)
9+
(allow default)
10+
(deny network*)
11+
(allow network-bind (local ip "localhost:*"))
12+
(allow network-inbound (local ip "localhost:*"))
13+
(allow network-outbound (remote ip "localhost:*"))
14+
(allow network-outbound (remote unix-socket))

‎.github/workflows/ci.yml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,13 @@ jobs:
6767
checks: write
6868
secrets: inherit
6969

70+
sandboxed_tests:
71+
needs: [lint, precommit, go_mod_tidy_check]
72+
uses: ./.github/workflows/sandboxed-test.yml
73+
permissions:
74+
contents: read
75+
secrets: inherit
76+
7077
build:
7178
needs: [lint, precommit, go_mod_tidy_check]
7279
uses: ./.github/workflows/build.yml

‎.github/workflows/sandboxed-test.yml‎

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
name: Sandboxed Tests
2+
3+
on:
4+
workflow_call:
5+
6+
# Run the suite with every socket except loopback closed. Terragrunt virtualizes
7+
# its side effects through the venv bundle, so a test that reaches the network is
8+
# either exercising an unvirtualized code path or is an integration test hiding in
9+
# the untagged suite. Both are worth catching here rather than as a flake on a day
10+
# the registry is slow.
11+
jobs:
12+
test:
13+
name: Sandboxed Test
14+
runs-on: ubuntu-latest
15+
env:
16+
GOGC: "400"
17+
18+
steps:
19+
- name: Checkout code
20+
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v6
21+
with:
22+
fetch-depth: 1
23+
24+
- name: Use mise to install dependencies
25+
uses: jdx/mise-action@1648a7812b9aeae629881980618f079932869151 # v4.0.1
26+
with:
27+
version: 2026.5.12
28+
env:
29+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
30+
31+
- id: go-cache-paths
32+
run: |
33+
echo "go-build=$(go env GOCACHE)" >> "$GITHUB_OUTPUT"
34+
echo "go-mod=$(go env GOMODCACHE)" >> "$GITHUB_OUTPUT"
35+
shell: bash
36+
37+
- name: Go Build Cache
38+
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v5
39+
with:
40+
path: ${{ steps.go-cache-paths.outputs.go-build }}
41+
key: ${{ runner.os }}-go-build-${{ hashFiles('**/go.sum') }}-ubuntu-amd64
42+
restore-keys: |
43+
${{ runner.os }}-go-build-
44+
45+
- name: Go Mod Cache
46+
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v5
47+
with:
48+
path: ${{ steps.go-cache-paths.outputs.go-mod }}
49+
key: ${{ runner.os }}-go-mod-${{ hashFiles('**/go.sum') }}-ubuntu-amd64
50+
restore-keys: |
51+
${{ runner.os }}-go-mod-
52+
53+
# Ubuntu 24.04 confines unprivileged user namespaces with AppArmor, which is
54+
# what `unshare --map-root-user --net` needs. The check below is what
55+
# actually gates the run, so a runner image that never restricted them is
56+
# free to ignore this.
57+
- name: Permit unprivileged user namespaces
58+
run: sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0 || true
59+
shell: bash
60+
61+
# `go test` compiles outside the sandbox, so warm the module cache first and
62+
# keep the sandbox scoped to what the test binaries do.
63+
- name: Warm build cache
64+
run: go build ./... && go mod download
65+
shell: bash
66+
67+
- name: Verify the sandbox blocks egress
68+
run: ./.github/scripts/ci/no-network-exec.sh --check
69+
shell: bash
70+
71+
- name: Run Tests
72+
run: go test -exec "$PWD/.github/scripts/ci/no-network-exec.sh" -timeout 45m ./...
73+
shell: bash

‎docs/bun.lock‎

Lines changed: 2 additions & 6 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎docs/src/content/docs/05-community/01-contributing.mdx‎

Lines changed: 36 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ sidebar:
66
order: 1
77
---
88

9-
import { Aside } from '@astrojs/starlight/components';
9+
import { Aside, Tabs, TabItem } from '@astrojs/starlight/components';
1010

1111
## Contribution Guidelines
1212

@@ -333,6 +333,41 @@ We can do a better job of finding candidates for additional testing here, so if
333333

334334
The convention we use for race tests is to prefix them with `WithRacing`. The Terragrunt Continuous Integration workflow will run these tests with the `-race` flag as part of the test suite.
335335

336+
#### Sandboxed tests
337+
338+
Terragrunt routes its side effects through a virtualized environment, so tests can drive filesystem, subprocess and HTTP behavior without touching the real thing. A test that still reaches the network is either exercising a code path that bypasses that environment, or is an integration test sitting in the untagged suite. Either way, it makes the unit suite slower and flakier. Every run pays for the round trip, and a slow registry or a rate-limited git host turns into a red build that has nothing to do with the change under review.
339+
340+
To check for that, run the suite with every socket except loopback closed:
341+
342+
```bash
343+
go test -exec "$PWD/.github/scripts/ci/no-network-exec.sh" ./...
344+
```
345+
346+
Before reading anything into a passing run, confirm the sandbox took effect:
347+
348+
```bash
349+
./.github/scripts/ci/no-network-exec.sh --check
350+
```
351+
352+
Loopback stays reachable, so tests that stand up an `httptest` server keep working, and compilation happens outside the sandbox, so module downloads are unaffected. How the sandbox itself is built differs by platform:
353+
354+
<Tabs syncKey="operating-systems">
355+
<TabItem label="Linux">
356+
Each test binary runs in its own network namespace, created with `unshare`.
357+
358+
Ubuntu 24.04 and later confine unprivileged user namespaces with AppArmor, which stops `unshare` from creating that namespace. If the check reports that the sandbox cannot run a command at all, lift the restriction:
359+
360+
```bash
361+
sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
362+
```
363+
</TabItem>
364+
<TabItem label="macOS">
365+
Each test binary runs under the seatbelt profile at `.github/scripts/ci/no-network.sb`, applied with `sandbox-exec`.
366+
367+
Nothing to set up. `sandbox-exec` ships with macOS.
368+
</TabItem>
369+
</Tabs>
370+
336371
#### Benchmark tests
337372

338373
Benchmark tests are tests that are run with the `-bench` flag to the `go test` command. They are used to measure the performance of a particular function or set of functions.

0 commit comments

Comments
 (0)