Skip to content

Commit 849c0c6

Browse files
james00012james00012
authored
Automate v2 release into a single create-release workflow (#1874)
* Automate the v2 release into a single create-release workflow Replace the two-step release process (manual scripts/release-prep-pin on a throwaway branch, then a tag-only workflow) with one workflow, .github/workflows/create-release.yml, dispatched with a version and a ref (default main). It runs the pre-flight gates and the GOWORK=off release-mode build, pins every submodule's cross-module require to the release version and commits that in CI, then pushes the 16 tags in dependency order and verifies the proxy. dry_run previews the tags without pushing. - Move the pin logic into scripts/release-prep-pin.sh so the workflow (and a human, if ever needed) share one deterministic implementation. - The intentionally-broken pinned state now lives only on an ephemeral in-CI commit that the tags point at, instead of a pushed release-prep branch whose build is red. Removes the last manual release step and the failing-CI branch. - Update docs/v2-release-runbook.md to describe the single-workflow flow. * Fix v2 tag format: modules/<name>/<version>, not modules/<name>/v2/<version> The submodule go.mod files live at modules/<name>/ (the /v2 is the SIV in the module path, not a directory), so Go resolves each module's version tag at <subdir>/<version> = modules/<name>/<version> (e.g. modules/aws/v2.0.0-beta.1). The workflow and runbook incorrectly put the /v2 in the tag itself (modules/aws/v2/v2.0.0-beta.1), which the module proxy cannot associate with the module path, so no consumer could resolve the tags. Caught by an end-to-end consumer test against the pushed beta tags: core (no cross-module deps) resolved, every other module failed with 'ssh/v2@v2.0.0-00010101...: unknown revision' because the placeholder-free tag was published under the wrong name. The proxy-verify URL is unchanged (it correctly uses the full module path, which does include /v2). * create-release: publish one GitHub Release per version After the lockstep tag push, create a single human-facing GitHub Release for the version (attached to the core tag; -beta/-rc suffixes marked prerelease), so the Releases page stays in sync instead of showing 16 per-module entries or nothing. Only runs on a real (non-dry-run) push.
1 parent 74bb0d6 commit 849c0c6

4 files changed

Lines changed: 230 additions & 131 deletions

File tree

‎.github/workflows/create-release.yml‎

Lines changed: 178 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,178 @@
1+
name: Create v2 Release (pin + lockstep tags)
2+
3+
# One manual workflow that cuts a full v2 release from main:
4+
# 1. Pre-flight: the modularization gates + the GOWORK=off release-mode build.
5+
# 2. Pin every submodule's cross-module require to the release version and
6+
# commit that in-CI (the commit the tags point at). Deterministic; runs
7+
# scripts/release-prep-pin.sh.
8+
# 3. Push the 16 per-module tags in dependency order, then verify the proxy.
9+
#
10+
# Supersedes the old two-step flow (manual release-prep-pin.sh on a throwaway
11+
# branch + a tag-only workflow). Procedure documented in
12+
# docs/v2-release-runbook.md. Run with dry_run=true first to preview the tags.
13+
14+
on:
15+
workflow_dispatch:
16+
inputs:
17+
version:
18+
description: 'Version to tag (e.g. v2.0.0-beta.1). Applied to every submodule.'
19+
required: true
20+
type: string
21+
ref:
22+
description: 'Base ref/SHA to release from.'
23+
required: false
24+
type: string
25+
default: main
26+
dry_run:
27+
description: 'If true, pin and print the tag commands without pushing.'
28+
required: false
29+
type: boolean
30+
default: true
31+
32+
# Default to read-only. Only the tagging job is granted write, so the pre-flight
33+
# job cannot push even though it runs scripts from a dispatched ref.
34+
permissions:
35+
contents: read
36+
37+
jobs:
38+
verify:
39+
name: Pre-flight checks
40+
runs-on: ubuntu-latest
41+
timeout-minutes: 20
42+
steps:
43+
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
44+
with:
45+
ref: ${{ inputs.ref }}
46+
persist-credentials: false
47+
- run: bash scripts/check-acyclic-deps.sh
48+
- run: bash scripts/check-single-source.sh
49+
- run: bash scripts/check-siv-placement.sh
50+
51+
- name: Set up mise
52+
uses: jdx/mise-action@5228313ee0372e111a38da051671ca30fc5a96db # v3
53+
env:
54+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
55+
# Validates the exact release-mode build (pin + GOWORK=off external
56+
# consumer) in a throwaway, so it runs against the un-pinned ref.
57+
- name: Release-mode build (GOWORK=off external consumer)
58+
run: bash scripts/check-release-mode.sh
59+
60+
tag-and-push:
61+
name: Pin, tag, push
62+
needs: verify
63+
runs-on: ubuntu-latest
64+
timeout-minutes: 10
65+
permissions:
66+
contents: write
67+
env:
68+
VERSION: ${{ inputs.version }}
69+
# Single source of truth for the module list and tag order (mirrors ORDER
70+
# in scripts/check-release-mode.sh and the runbook tag sequence).
71+
MODULE_PATHS: >-
72+
modules/core
73+
modules/ssh modules/httphelper modules/dnshelper
74+
modules/docker modules/packer modules/database modules/opa
75+
modules/aws modules/azure modules/gcp modules/k8s modules/helm
76+
modules/terraform modules/terragrunt modules/teststructure
77+
steps:
78+
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
79+
with:
80+
ref: ${{ inputs.ref }}
81+
fetch-depth: 0
82+
83+
- name: Validate version
84+
run: |
85+
if [[ ! "$VERSION" =~ ^v2\.[0-9]+\.[0-9]+(-[0-9A-Za-z.]+)?$ ]]; then
86+
echo "::error::version '$VERSION' must look like v2.<minor>.<patch>[-suffix] (e.g. v2.0.0-beta.1)"
87+
exit 1
88+
fi
89+
90+
- name: Set up mise
91+
uses: jdx/mise-action@5228313ee0372e111a38da051671ca30fc5a96db # v3
92+
env:
93+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
94+
95+
- name: Configure git
96+
run: |
97+
git config user.name "github-actions[bot]"
98+
git config user.email "github-actions[bot]@users.noreply.github.com"
99+
100+
# Pin the cross-module requires to the real version and commit in-CI. This
101+
# is the commit the tags point at. It intentionally breaks the go.work
102+
# build (the tags do not exist yet); that only matters until the push.
103+
- name: Pin cross-module requires and commit
104+
run: |
105+
bash scripts/release-prep-pin.sh "$VERSION"
106+
if git diff --quiet; then
107+
echo "no placeholders to pin (ref already pinned to $VERSION)"
108+
else
109+
git commit -am "release-prep: pin cross-module requires to $VERSION"
110+
fi
111+
echo "PINNED_SHA=$(git rev-parse HEAD)" >> "$GITHUB_ENV"
112+
113+
- name: Tag in dependency order
114+
env:
115+
DRY_RUN: ${{ inputs.dry_run }}
116+
run: |
117+
set -e
118+
for path in $MODULE_PATHS; do
119+
tag="${path}/${VERSION}"
120+
if [ "$DRY_RUN" = "true" ]; then
121+
echo "DRY-RUN would tag ${PINNED_SHA:0:12}: $tag"
122+
else
123+
git tag -a "$tag" -m "v2 lockstep release $VERSION"
124+
git push origin "$tag"
125+
echo "pushed $tag"
126+
fi
127+
done
128+
129+
- name: Verify proxy serves each tag
130+
if: ${{ inputs.dry_run == false }}
131+
run: |
132+
# Allow the proxy a moment to populate after push.
133+
sleep 30
134+
fail=0
135+
for path in $MODULE_PATHS; do
136+
url="https://proxy.golang.org/github.com/gruntwork-io/terratest/${path}/v2/@v/${VERSION}.info"
137+
code=$(curl -s -o /dev/null -w '%{http_code}' "$url")
138+
if [ "$code" = "200" ]; then
139+
echo " $path: 200"
140+
else
141+
echo "::warning::$path: $code from $url"
142+
fail=1
143+
fi
144+
done
145+
if [ "$fail" -ne 0 ]; then
146+
echo "::warning::Proxy has not served every tag yet. The tags are already pushed and immutable; the proxy usually populates within a few minutes. Re-run this check if needed."
147+
fi
148+
149+
# One human-facing GitHub Release for the whole version (not 16). Attached
150+
# to the core tag; -beta/-rc suffixes are marked prerelease.
151+
- name: Create GitHub Release
152+
if: ${{ inputs.dry_run == false }}
153+
env:
154+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
155+
run: |
156+
prerelease=""
157+
case "$VERSION" in *-*) prerelease="--prerelease" ;; esac
158+
{
159+
echo "Terratest v2 lockstep release. All 16 submodules are tagged at ${VERSION}."
160+
echo
161+
echo "## Install"
162+
echo
163+
echo '```'
164+
echo "go get github.com/gruntwork-io/terratest/modules/aws/v2@${VERSION}"
165+
echo '```'
166+
echo
167+
echo "## Modules"
168+
echo
169+
echo "All at \`github.com/gruntwork-io/terratest/modules/<name>/v2\`:"
170+
echo
171+
for p in $MODULE_PATHS; do printf '%s ' "${p#modules/}"; done
172+
echo
173+
} > release_notes.md
174+
# shellcheck disable=SC2086
175+
gh release create "modules/core/${VERSION}" \
176+
--title "Terratest ${VERSION}" \
177+
--notes-file release_notes.md \
178+
$prerelease

‎.github/workflows/v2-release.yml‎

Lines changed: 0 additions & 121 deletions
This file was deleted.

‎docs/v2-release-runbook.md‎

Lines changed: 21 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,15 @@
22

33
How to cut a coordinated release of the v2 submodules. Read this before tagging.
44

5-
The tag push and proxy verification are automated by the `v2 Release` workflow
6-
(`.github/workflows/v2-release.yml`, run via workflow_dispatch). This runbook
7-
explains the procedure that workflow follows and how to do it by hand if needed.
5+
The whole release is one manual workflow: `Create v2 Release`
6+
(`.github/workflows/create-release.yml`, run via workflow_dispatch). Give it a
7+
`version` (e.g. `v2.0.0-beta.1`) and a `ref` (default `main`); it runs the
8+
pre-flight checks, pins the cross-module requires, commits that in CI, pushes the
9+
16 tags in dependency order, and verifies the proxy. Run it once with
10+
`dry_run=true` to preview the exact tags, then again with `dry_run=false` to push.
11+
12+
The rest of this runbook explains what that workflow does under the hood and how
13+
to do each step by hand if you ever need to.
814

915
## Layout
1016

@@ -22,14 +28,16 @@ resolves; it is never pinned to a real, to-be-published version. Pinning a sibli
2228
version (e.g. `core/v2 v2.0.0-beta.1`) BREAKS the workspace build until that tag
2329
actually exists: `go.work` does not shadow an unpublished required version, so
2430
`go build` and `go work sync` try to fetch the missing revision and fail. The pin
25-
must therefore happen on a short-lived release-prep branch, immediately before the
26-
tags are pushed, never on the modularization PR.
31+
must therefore happen immediately before the tags are pushed, never on `main` or
32+
a modularization PR. The `Create v2 Release` workflow does this pin in CI (via
33+
`scripts/release-prep-pin.sh`) on an ephemeral commit that only the tags point
34+
at, so the broken-build state never lands on a branch.
2735

2836
CI validates the release-mode build continuously without committing the pin: the
2937
`GOWORK=off` check generates the pinned state with throwaway `replace` directives,
3038
builds a consumer, and discards it (see `scripts/`).
3139

32-
## Pre-flight (on a release-prep branch)
40+
## Pre-flight (what the workflow does before tagging)
3341

3442
1. Choose the version, e.g. `v2.0.0-beta.1`.
3543
2. Pin each module, in dependency order (core first, then helpers, tooling,
@@ -60,11 +68,14 @@ build is validated continuously without committing the pin or needing tags.
6068

6169
## Tag push order
6270

63-
All tags point at the same release commit. Each tag name puts the `/v2` SIV in the
64-
tag itself: `modules/<name>/v2/<version>` (NOT `modules/<name>/<version>`, which
65-
the proxy cannot associate with the `/v2` module path). Push in dependency order:
71+
All tags point at the same release commit. The tag name is the module's
72+
on-disk subdirectory (which does NOT include the `/v2`, since `/v2` is only the
73+
SIV in the module path, not a directory) followed by the version:
74+
`modules/<name>/<version>`, e.g. `modules/aws/v2.0.0-beta.1`. Do NOT put the
75+
`/v2` in the tag (`modules/<name>/v2/<version>`), Go looks for the tag at
76+
`<subdir>/<version>` and will not find it. Push in dependency order:
6677

67-
1. `modules/core/v2/v2.0.0-beta.1`
78+
1. `modules/core/v2.0.0-beta.1`
6879
2. helpers: `ssh`, `httphelper`, `dnshelper`
6980
3. tooling: `docker`, `packer`, `database`, `opa`
7081
4. platforms: `aws`, `azure`, `gcp`, then `k8s`, `helm`

‎scripts/release-prep-pin.sh‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
#!/usr/bin/env bash
2+
# release-prep-pin.sh <version> e.g. release-prep-pin.sh v2.0.0-beta.1
3+
#
4+
# Pins every submodule's cross-module /v2 require from the go.work dev
5+
# placeholder (v2.0.0-00010101000000-000000000000) to the real release version,
6+
# in place. The v2 release workflow runs this in CI right before the lockstep
7+
# tag push; the pinned commit is what the tags point at.
8+
#
9+
# This intentionally breaks the go.work workspace build (the tags do not exist
10+
# yet) — that is expected and only lasts until the tags are pushed. Root go.mod
11+
# and go.work are left untouched (the root module is never published in v2).
12+
set -uo pipefail
13+
14+
VERSION="${1:?usage: release-prep-pin.sh <version, e.g. v2.0.0-beta.1>}"
15+
if [[ ! "$VERSION" =~ ^v2\.[0-9]+\.[0-9]+(-[0-9A-Za-z.]+)?$ ]]; then
16+
echo "bad version: '$VERSION' (want v2.<minor>.<patch>[-suffix])" >&2
17+
exit 1
18+
fi
19+
20+
PLACEHOLDER='v2.0.0-00010101000000-000000000000'
21+
count=0
22+
for d in modules/*/; do
23+
[ -f "${d}go.mod" ] || continue
24+
mods=$(grep -oE "github.com/gruntwork-io/terratest/modules/[a-z0-9]+/v2 ${PLACEHOLDER}" "${d}go.mod" 2>/dev/null | awk '{print $1}' || true)
25+
for mod in $mods; do
26+
go -C "$d" mod edit -require="${mod}@${VERSION}"
27+
echo " ${d}go.mod: ${mod} -> ${VERSION}"
28+
count=$((count + 1))
29+
done
30+
done
31+
echo "pinned ${count} cross-module require(s) to ${VERSION}"

0 commit comments

Comments
 (0)