Skip to content

Commit 2498735

Browse files
cowwoccowwoc
committed
feature: add deterministic SHA256 manifest ordering to build-git-filter-repo workflow
1 parent 7a92da9 commit 2498735

3 files changed

Lines changed: 114 additions & 6 deletions

File tree

‎.cat/issues/v2/v2.1/validate-release-artifact-ordering/index.json‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1,5 @@
1-
{"status": "open"}
1+
{
2+
"status" : "closed",
3+
"resolution" : "implemented",
4+
"target_branch" : "v2.1"
5+
}

‎.cat/issues/v2/v2.1/validate-release-artifact-ordering/plan.md‎

Lines changed: 73 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -11,17 +11,85 @@ a SHA256 manifest. Matrix job completion order is non-deterministic (depends on
1111
so the manifest may be generated in different orders across runs. This produces unnecessary diffs and
1212
makes the manifest harder to review.
1313

14+
Currently the workflow creates individual per-platform `.sha256` files but does not produce a combined
15+
`SHA256SUMS.txt` manifest. This issue adds that combined manifest with deterministic ordering.
16+
1417
## Changes Required
1518

16-
1. In the manifest assembly step of `build-git-filter-repo.yml`, sort the platform entries before
17-
writing the final manifest file (e.g., using `sort` or explicit ordering).
18-
2. Document the expected manifest ordering in a comment in the workflow file.
19-
3. Optionally add a CI validation step that verifies the manifest is in sorted order as a
20-
post-condition check.
19+
1. In the `release` job, add a "Create combined SHA256 manifest" step (after "Read SHA256 checksums")
20+
that builds `artifacts/SHA256SUMS.txt` with entries sorted alphabetically by platform name.
21+
2. Add a "Verify manifest ordering" step after creation as a post-condition CI check.
22+
3. Add `artifacts/SHA256SUMS.txt` to the files list in the "Create release" step.
23+
4. Document the expected manifest ordering in a comment in the new step.
24+
25+
## Research Findings
26+
27+
Each per-platform `.sha256` file contains the bare binary name `git-filter-repo` (not the platform
28+
name), because the checksum is generated in the `build` job before the binary is renamed. The combined
29+
manifest must therefore use the platform-qualified name. The correct approach is:
30+
- Read the hash from each `.sha256` file using `awk '{print $1}'`
31+
- Pair each hash with its platform-qualified binary name (e.g., `git-filter-repo-linux-x64`)
32+
- Sort alphabetically by filename column to produce deterministic order:
33+
`linux-aarch64`, `linux-x64`, `macos-aarch64`, `macos-x64`
2134

2235
## Post-conditions
2336

2437
- [ ] SHA256 manifest entries are always written in the same deterministic order
2538
- [ ] The ordering logic is documented in the workflow file
2639
- [ ] Two consecutive runs with the same inputs produce identical manifest files
2740
- [ ] No regressions introduced to the build or download flow
41+
42+
## Jobs
43+
44+
### Job 1
45+
- Modify `.github/workflows/build-git-filter-repo.yml` in the `release` job:
46+
1. After the "Read SHA256 checksums" step (line ~195), add a "Create combined SHA256 manifest" step:
47+
```yaml
48+
- name: Create combined SHA256 manifest
49+
run: |
50+
set -euo pipefail
51+
# Combine all platform SHA256 checksums into a single manifest file.
52+
# Entries are sorted alphabetically by platform filename for deterministic ordering
53+
# regardless of which matrix jobs complete first.
54+
# Expected sorted order: linux-aarch64, linux-x64, macos-aarch64, macos-x64
55+
{
56+
printf '%s %s\n' "$(awk '{print $1}' artifacts/git-filter-repo-linux-aarch64.sha256)" \
57+
"git-filter-repo-linux-aarch64"
58+
printf '%s %s\n' "$(awk '{print $1}' artifacts/git-filter-repo-linux-x64.sha256)" \
59+
"git-filter-repo-linux-x64"
60+
printf '%s %s\n' "$(awk '{print $1}' artifacts/git-filter-repo-macos-aarch64.sha256)" \
61+
"git-filter-repo-macos-aarch64"
62+
printf '%s %s\n' "$(awk '{print $1}' artifacts/git-filter-repo-macos-x64.sha256)" \
63+
"git-filter-repo-macos-x64"
64+
} > artifacts/SHA256SUMS.txt
65+
cat artifacts/SHA256SUMS.txt
66+
```
67+
2. After that, add a "Verify manifest ordering" step:
68+
```yaml
69+
- name: Verify manifest ordering
70+
run: |
71+
set -euo pipefail
72+
# Verify that SHA256SUMS.txt entries are in sorted order (second column = filename).
73+
EXPECTED=$(sort -k2 artifacts/SHA256SUMS.txt)
74+
ACTUAL=$(cat artifacts/SHA256SUMS.txt)
75+
if [[ "$EXPECTED" != "$ACTUAL" ]]; then
76+
echo "ERROR: SHA256SUMS.txt entries are not in sorted order." >&2
77+
echo "Expected:" >&2
78+
echo "$EXPECTED" >&2
79+
echo "Actual:" >&2
80+
echo "$ACTUAL" >&2
81+
exit 1
82+
fi
83+
echo "OK: SHA256SUMS.txt entries are in sorted order."
84+
```
85+
3. In the "Create release" step, add `artifacts/SHA256SUMS.txt` to the `files:` list (after the
86+
existing `.sha256` entries).
87+
- Update index.json: set status to `closed`, progress to 100%.
88+
89+
## Success Criteria
90+
91+
- `artifacts/SHA256SUMS.txt` is created with exactly 4 entries (one per platform)
92+
- Entries are sorted alphabetically: `linux-aarch64`, `linux-x64`, `macos-aarch64`, `macos-x64`
93+
- "Verify manifest ordering" step passes (no exit code 1)
94+
- `SHA256SUMS.txt` is attached to the GitHub release alongside the individual `.sha256` files
95+
- `git diff` of the workflow shows only the three additions described above

‎.github/workflows/build-git-filter-repo.yml‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -205,6 +205,41 @@ jobs:
205205
echo "macos_x64=${MACOS_X64}" >> $GITHUB_OUTPUT
206206
echo "macos_aarch64=${MACOS_AARCH64}" >> $GITHUB_OUTPUT
207207
208+
- name: Create combined SHA256 manifest
209+
run: |
210+
set -euo pipefail
211+
# Combine all platform SHA256 checksums into a single manifest file.
212+
# Entries are sorted alphabetically by platform filename for deterministic ordering
213+
# regardless of which matrix jobs complete first.
214+
# Expected sorted order: linux-aarch64, linux-x64, macos-aarch64, macos-x64
215+
{
216+
printf '%s %s\n' "$(awk '{print $1}' artifacts/git-filter-repo-linux-aarch64.sha256)" \
217+
"git-filter-repo-linux-aarch64"
218+
printf '%s %s\n' "$(awk '{print $1}' artifacts/git-filter-repo-linux-x64.sha256)" \
219+
"git-filter-repo-linux-x64"
220+
printf '%s %s\n' "$(awk '{print $1}' artifacts/git-filter-repo-macos-aarch64.sha256)" \
221+
"git-filter-repo-macos-aarch64"
222+
printf '%s %s\n' "$(awk '{print $1}' artifacts/git-filter-repo-macos-x64.sha256)" \
223+
"git-filter-repo-macos-x64"
224+
} > artifacts/SHA256SUMS.txt
225+
cat artifacts/SHA256SUMS.txt
226+
227+
- name: Verify manifest ordering
228+
run: |
229+
set -euo pipefail
230+
# Verify that SHA256SUMS.txt entries are in sorted order (second column = filename).
231+
EXPECTED=$(sort -k2 artifacts/SHA256SUMS.txt)
232+
ACTUAL=$(cat artifacts/SHA256SUMS.txt)
233+
if [[ "$EXPECTED" != "$ACTUAL" ]]; then
234+
echo "ERROR: SHA256SUMS.txt entries are not in sorted order." >&2
235+
echo "Expected:" >&2
236+
echo "$EXPECTED" >&2
237+
echo "Actual:" >&2
238+
echo "$ACTUAL" >&2
239+
exit 1
240+
fi
241+
echo "OK: SHA256SUMS.txt entries are in sorted order."
242+
208243
- name: Create release
209244
uses: softprops/action-gh-release@153bb8e04406b158c6c84fc1615b65b24149a1fe # v2.6.1
210245
with:
@@ -238,5 +273,6 @@ jobs:
238273
artifacts/git-filter-repo-linux-aarch64.sha256
239274
artifacts/git-filter-repo-macos-x64.sha256
240275
artifacts/git-filter-repo-macos-aarch64.sha256
276+
artifacts/SHA256SUMS.txt
241277
draft: false
242278
prerelease: false

0 commit comments

Comments
 (0)