Skip to content

Commit 7ed43c3

Browse files
efiacorCopilot
andauthored
Add release artifact verification step (#1066)
* Add release artifact verification to release workflow Add a post-goreleaser step that verifies all expected artifacts are published to the GitHub release and validates their checksums. This catches silent goreleaser failures that previously resulted in empty release pages. The verification script: - Checks all expected assets are present (porchctl binaries, blueprint, checksums.txt) - Downloads artifacts and validates SHA256 checksums - Supports retry logic for release propagation delays - Works on both Linux (CI) and macOS (local dev) Signed-off-by: Fiachra Corcoran <fiachra.corcoran@est.tech> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Fiachra Corcoran <fiachra.corcoran@est.tech> * fix: address review comments on verify script - Stop suppressing stderr in gh retry loop so auth/API errors are visible - Fail on missing files in checksums.txt instead of skipping - Verify all expected assets have entries in checksums.txt Signed-off-by: Fiachra Corcoran <fiachra.corcoran@est.tech> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Fiachra Corcoran <fiachra.corcoran@est.tech> * fix: address second round of review comments - Fix requirements comment to reference GH_TOKEN (matches workflow) - Stop capturing stderr into ASSETS variable (let it pass through) - Let checksums.txt download fail fast instead of suppressing errors - Use portable mktemp template for macOS compatibility Signed-off-by: Fiachra Corcoran <fiachra.corcoran@est.tech> * Fix linter error Signed-off-by: Fiachra Corcoran <fiachra.corcoran@est.tech> * Fix lint errors Signed-off-by: Fiachra Corcoran <fiachra.corcoran@est.tech> --------- Signed-off-by: Fiachra Corcoran <fiachra.corcoran@est.tech> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
1 parent 0e60ef8 commit 7ed43c3

4 files changed

Lines changed: 243 additions & 0 deletions

File tree

.github/workflows/release.yaml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,3 +50,8 @@ jobs:
5050
args: release --skip=validate -f release/tag/goreleaser.yaml
5151
env:
5252
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
53+
54+
- name: Verify release artifacts
55+
env:
56+
GH_TOKEN: ${{ github.token }}
57+
run: ./scripts/verify-release-artifacts.sh "${{ github.ref_name }}"

api/go.mod

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ go 1.26.0
44

55
require (
66
github.com/kptdev/kpt/api v0.0.1
7+
github.com/pkg/errors v0.9.1
78
github.com/stretchr/testify v1.11.1
89
k8s.io/api v0.36.1
910
k8s.io/apiextensions-apiserver v0.36.1

api/go.sum

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,8 @@ github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq
5757
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
5858
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
5959
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
60+
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
61+
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
6062
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
6163
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
6264
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
Lines changed: 235 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,235 @@
1+
#!/usr/bin/env bash
2+
# Copyright 2026 The kpt Authors
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
16+
# verify-release-artifacts.sh
17+
#
18+
# Verifies that all expected artifacts are published to a GitHub release
19+
# and that their checksums match the published checksums.txt.
20+
#
21+
# Usage:
22+
# ./scripts/verify-release-artifacts.sh <tag>
23+
#
24+
# Requirements:
25+
# - gh CLI installed and authenticated (gh auth login)
26+
# - For CI: set GH_TOKEN in the workflow step env (or GITHUB_TOKEN)
27+
#
28+
# Example:
29+
# ./scripts/verify-release-artifacts.sh v1.5.11
30+
31+
set -euo pipefail
32+
33+
TAG="${1:-}"
34+
if [[ -z "${TAG}" ]]; then
35+
echo "ERROR: tag argument is required"
36+
echo "Usage: $0 <tag>"
37+
exit 1
38+
fi
39+
40+
# Ensure gh CLI is available
41+
if ! command -v gh &>/dev/null; then
42+
echo "ERROR: gh CLI is not installed. Install from https://cli.github.com/"
43+
exit 1
44+
fi
45+
46+
# Determine the checksum command (Linux: sha256sum, macOS: shasum -a 256)
47+
if command -v sha256sum &>/dev/null; then
48+
SHA256_CMD="sha256sum"
49+
elif command -v shasum &>/dev/null; then
50+
SHA256_CMD="shasum -a 256"
51+
else
52+
echo "ERROR: neither sha256sum nor shasum found"
53+
exit 1
54+
fi
55+
56+
# Strip leading 'v' to get the version number (goreleaser uses version without 'v')
57+
VERSION="${TAG#v}"
58+
59+
# Repository in owner/name format — detect from env or infer from gh
60+
REPO="${GITHUB_REPOSITORY:-}"
61+
if [[ -z "${REPO}" ]]; then
62+
REPO=$(gh repo view --json nameWithOwner -q '.nameWithOwner' 2>/dev/null || true)
63+
fi
64+
if [[ -z "${REPO}" ]]; then
65+
echo "ERROR: could not determine repository. Set GITHUB_REPOSITORY or run from a git checkout."
66+
exit 1
67+
fi
68+
69+
echo "==> Verifying release artifacts for ${TAG} in ${REPO}"
70+
71+
# Expected artifact names based on goreleaser.yaml configuration
72+
EXPECTED_ASSETS=(
73+
"porchctl_${VERSION}_darwin_amd64.tar.gz"
74+
"porchctl_${VERSION}_darwin_arm64.tar.gz"
75+
"porchctl_${VERSION}_linux_amd64.tar.gz"
76+
"porchctl_${VERSION}_linux_arm64.tar.gz"
77+
"porch_blueprint.tar.gz"
78+
"checksums.txt"
79+
)
80+
81+
# Retry logic for asset listing — the release may take a moment to propagate
82+
MAX_RETRIES=5
83+
RETRY_DELAY=10
84+
ASSETS=""
85+
86+
for ((i = 1; i <= MAX_RETRIES; i++)); do
87+
echo " Attempt ${i}/${MAX_RETRIES}: listing release assets..."
88+
if ASSETS=$(gh release view "${TAG}" --repo "${REPO}" --json assets -q '.assets[].name'); then
89+
if [[ -n "${ASSETS}" ]]; then
90+
break
91+
fi
92+
fi
93+
ASSETS=""
94+
if [[ ${i} -lt ${MAX_RETRIES} ]]; then
95+
echo " Release not found yet, retrying in ${RETRY_DELAY}s..."
96+
sleep "${RETRY_DELAY}"
97+
fi
98+
done
99+
100+
if [[ -z "${ASSETS}" ]]; then
101+
echo "ERROR: could not retrieve release assets for tag ${TAG}"
102+
echo " The release may not exist or has no assets."
103+
exit 1
104+
fi
105+
106+
echo ""
107+
echo "==> Assets found on release:"
108+
while IFS= read -r asset; do
109+
echo " ${asset}"
110+
done <<< "${ASSETS}"
111+
echo ""
112+
113+
# Check all expected assets are present
114+
MISSING=()
115+
for expected in "${EXPECTED_ASSETS[@]}"; do
116+
if ! grep -qxF "${expected}" <<< "${ASSETS}"; then
117+
MISSING+=("${expected}")
118+
fi
119+
done
120+
121+
if [[ ${#MISSING[@]} -gt 0 ]]; then
122+
echo "ERROR: the following expected artifacts are MISSING from the release:"
123+
for m in "${MISSING[@]}"; do
124+
echo " - ${m}"
125+
done
126+
exit 1
127+
fi
128+
129+
echo "==> All expected artifacts are present."
130+
131+
# Verify that checksums.txt has an entry for each expected asset (excluding checksums.txt itself)
132+
echo ""
133+
echo "==> Verifying checksums.txt coverage..."
134+
if ! CHECKSUMS_CONTENT=$(gh release download "${TAG}" --repo "${REPO}" --pattern "checksums.txt" --output -); then
135+
echo "ERROR: could not download checksums.txt to validate coverage"
136+
exit 1
137+
fi
138+
if [[ -z "${CHECKSUMS_CONTENT}" ]]; then
139+
echo "ERROR: checksums.txt is empty"
140+
exit 1
141+
fi
142+
143+
UNCOVERED=()
144+
for expected in "${EXPECTED_ASSETS[@]}"; do
145+
[[ "${expected}" == "checksums.txt" ]] && continue
146+
if ! grep -qF "${expected}" <<< "${CHECKSUMS_CONTENT}"; then
147+
UNCOVERED+=("${expected}")
148+
fi
149+
done
150+
151+
if [[ ${#UNCOVERED[@]} -gt 0 ]]; then
152+
echo "ERROR: the following expected artifacts are NOT listed in checksums.txt:"
153+
for u in "${UNCOVERED[@]}"; do
154+
echo " - ${u}"
155+
done
156+
exit 1
157+
fi
158+
echo " All expected artifacts have checksum entries."
159+
160+
# Download artifacts and verify checksums
161+
WORKDIR=$(mktemp -d "${TMPDIR:-/tmp}/verify-release.XXXXXX")
162+
trap 'rm -rf "${WORKDIR}"' EXIT
163+
164+
echo ""
165+
echo "==> Downloading artifacts to verify checksums..."
166+
gh release download "${TAG}" --repo "${REPO}" --dir "${WORKDIR}"
167+
168+
echo ""
169+
echo "==> Verifying checksums..."
170+
cd "${WORKDIR}"
171+
172+
if [[ ! -f "checksums.txt" ]]; then
173+
echo "ERROR: checksums.txt was not found after download"
174+
exit 1
175+
fi
176+
177+
echo " Contents of checksums.txt:"
178+
sed 's/^/ /' < checksums.txt
179+
echo ""
180+
181+
# Verify each file listed in checksums.txt
182+
FAILED=0
183+
CHECKED=0
184+
while IFS= read -r line; do
185+
# Skip empty lines
186+
[[ -z "${line}" ]] && continue
187+
# Each line is: <hash> <filename> (two spaces between hash and name)
188+
CHECKSUM=$(awk '{print $1}' <<< "${line}")
189+
FILENAME=$(awk '{print $2}' <<< "${line}")
190+
191+
if [[ -z "${FILENAME}" || -z "${CHECKSUM}" ]]; then
192+
echo " ! Skipping malformed line: ${line}"
193+
continue
194+
fi
195+
196+
if [[ -f "${FILENAME}" ]]; then
197+
ACTUAL=$(${SHA256_CMD} "${FILENAME}" | awk '{print $1}')
198+
if [[ "${ACTUAL}" == "${CHECKSUM}" ]]; then
199+
echo "${FILENAME}"
200+
CHECKED=$((CHECKED + 1))
201+
else
202+
echo "${FILENAME} — CHECKSUM MISMATCH"
203+
echo " expected: ${CHECKSUM}"
204+
echo " actual: ${ACTUAL}"
205+
FAILED=1
206+
fi
207+
else
208+
echo "${FILENAME} — MISSING from download directory"
209+
FAILED=1
210+
fi
211+
done < checksums.txt
212+
213+
# Ensure checksums.txt covers all expected artifacts (except itself)
214+
CHECKSUM_FILENAMES=$(awk 'NF {print $2}' checksums.txt)
215+
for expected in "${EXPECTED_ASSETS[@]}"; do
216+
[[ "${expected}" == "checksums.txt" ]] && continue
217+
if ! grep -qxF "${expected}" <<< "${CHECKSUM_FILENAMES}"; then
218+
echo "${expected} — missing from checksums.txt"
219+
FAILED=1
220+
fi
221+
done
222+
223+
echo ""
224+
if [[ ${FAILED} -ne 0 ]]; then
225+
echo "ERROR: one or more checksum verifications FAILED"
226+
exit 1
227+
fi
228+
229+
if [[ ${CHECKED} -eq 0 ]]; then
230+
echo "ERROR: no files were checked against checksums — something is wrong"
231+
exit 1
232+
fi
233+
234+
echo "==> All checksums verified successfully (${CHECKED} files)."
235+
echo "==> Release ${TAG} is complete and valid."

0 commit comments

Comments
 (0)