Skip to content

Commit 02f6294

Browse files
MaximusHaximusMaximusHaximus
committed
fix(apps-e2e): address PR #43 API review feedback
Correctness: - Guard apps-e2e-trigger.yml against firing on pipelines' own PRs (`github.repository != '0xPolygon/pipelines'`) — pipelines has no `pnpm run e2e` to run. No-op in any consumer, so still copy-verbatim. - restore.sh: `docker create --entrypoint /bin/true` so the addresses sidecar extraction overrides (not appends to) any ENTRYPOINT a future snapshot image declares. - cold-start.sh: verify the EL resource scale-down sed actually applied (grep the override marker, exit 1 if the file moved or vars renamed on an unexpected kurtosis_pos_ref) instead of silently OOM-killing. - restore.sh: fold state_hex_file / body_file into the cleanup trap so the ~2 MB temps don't leak when curl or the result check exits non-zero. - Drop the stale "compare boolean AND string" comment in the trigger (the expression only compares the toJSON string). Flexibility / standards: - Add REGISTRY_USERNAME / REGISTRY_PASSWORD (required: false) secrets to apps-e2e.yml and forward into the snapshot composite, falling back to the actor + GITHUB_TOKEN when unset. Unblocks a cross-org consumer (matic.js) pulling a private snapshot with its own credentials. - Remove the `secrets: inherit` recommendation from the header comment (prohibited by Team Standards). - Extract the on-failure dump shell into a new e2e-dump composite action (bundled script via github.action_path) rather than inline YAML — a reusable workflow can't reach a .github/scripts/ helper cross-repo. - Document the dual-path adapter contract and the secret-injection limitation on the e2e_command input; cold-start composite now exports E2E_KURTOSIS_ENCLAVE=pos for a symmetric, single-path consumer adapter. Deferred (design polish, tracked for follow-up): runner/fetch-depth inputs, jq-over-python, install-after-composite, enclave-name parameterisation, required snapshot_image. Generic suite-secret injection stays out — named secrets only, no secrets: inherit.
1 parent 9508bcc commit 02f6294

7 files changed

Lines changed: 191 additions & 44 deletions

File tree

‎.github/actions/e2e-cold-start/cold-start.sh‎

Lines changed: 31 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -77,16 +77,41 @@ fi
7777
# developer machine with more headroom this rewrite is a no-op from the
7878
# suite's perspective — the services just use less of the available
7979
# capacity. Remove once kurtosis-pos exposes these as inputs.
80+
#
81+
# This patch is brittle by nature: it depends on the file path and the
82+
# variable names. `kurtosis_pos_ref` is consumer-overridable, so an
83+
# unexpected ref could rename either and make the sed a silent no-op —
84+
# the enclave would then request 16 GB and the runner OOM-kills it with a
85+
# confusing failure. Verify the override actually landed and fail loud if
86+
# not, so the cause is unambiguous.
8087
shared_star="$WORK_DIR/src/el/shared.star"
81-
if [ -f "$shared_star" ]; then
82-
sed -i.bak -E \
83-
-e 's/^MAX_CPU = [0-9]+.*/MAX_CPU = 1800 # CI override (ubuntu-latest 2-vCPU private-repo runner)/' \
84-
-e 's/^MAX_MEM = [0-9]+.*/MAX_MEM = 4096 # CI override (ubuntu-latest 7-GB private-repo runner)/' \
85-
"$shared_star"
86-
rm -f "${shared_star}.bak"
88+
if [ ! -f "$shared_star" ]; then
89+
echo "[e2e-cold-start] ERROR: expected ${shared_star} not found — kurtosis-pos@${KURTOSIS_POS_REF} may have moved the EL resource config. Update cold-start.sh for this ref." >&2
90+
exit 1
91+
fi
92+
sed -i.bak -E \
93+
-e 's/^MAX_CPU = [0-9]+.*/MAX_CPU = 1800 # CI override (ubuntu-latest 2-vCPU private-repo runner)/' \
94+
-e 's/^MAX_MEM = [0-9]+.*/MAX_MEM = 4096 # CI override (ubuntu-latest 7-GB private-repo runner)/' \
95+
"$shared_star"
96+
rm -f "${shared_star}.bak"
97+
# Confirm both overrides applied — grep for the marker comment the sed adds.
98+
# Two matches expected (MAX_CPU + MAX_MEM); anything less means a var was
99+
# renamed and we'd silently run at full resource request.
100+
override_count="$(grep -c 'CI override (ubuntu-latest' "$shared_star" || true)"
101+
if [ "$override_count" -ne 2 ]; then
102+
echo "[e2e-cold-start] ERROR: EL resource scale-down did not apply (${override_count}/2 markers found in ${shared_star}). kurtosis-pos@${KURTOSIS_POS_REF} likely renamed MAX_CPU/MAX_MEM — the runner would OOM. Update the sed patterns for this ref." >&2
103+
exit 1
87104
fi
88105

89106
echo "[e2e-cold-start] running enclave 'pos' (this takes ~5–10m)"
90107
( cd "$WORK_DIR" && kurtosis run --enclave pos --args-file "$ARGS_FILE_ABS" . )
91108

109+
# Export the enclave name so the consumer's e2e suite can locate it via the
110+
# kurtosis CLI in cold-start mode (symmetric with snapshot mode exporting
111+
# E2E_SNAPSHOT_ADDRESSES_JSON). Lets a consumer write a single env-driven
112+
# adapter instead of hardcoding the enclave name per mode.
113+
if [ -n "${GITHUB_ENV:-}" ]; then
114+
echo "E2E_KURTOSIS_ENCLAVE=pos" >> "$GITHUB_ENV"
115+
fi
116+
92117
echo "[e2e-cold-start] ready"

‎.github/actions/e2e-dump/action.yml‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
name: E2E devnet dump
2+
description: >
3+
Capture devnet state into /tmp/devnet-dump for post-mortem after an e2e
4+
failure. Snapshot mode dumps the docker-compose stack's ps + logs and the
5+
compose file; cold-start mode dumps the kurtosis enclave. Best-effort —
6+
individual failures are swallowed so a half-restored run still yields
7+
whatever artefacts it can.
8+
9+
Bundled as a composite (not a reusable-workflow inline step or a
10+
.github/scripts/ helper) so the shell logic is locally runnable AND
11+
reachable when apps-e2e.yml runs in a consumer's checkout — a raw script
12+
under .github/scripts/ is not on disk cross-repo; ${{ github.action_path }}
13+
is.
14+
15+
inputs:
16+
use_snapshot:
17+
description: '"1" for snapshot mode (compose dump), "0" for cold-start (enclave dump).'
18+
required: true
19+
compose_file:
20+
description: >
21+
Snapshot mode: path to the restored docker-compose.yaml (the restore
22+
composite's compose_file_path output). May be empty if the restore
23+
failed before writing it — the dump guards for that.
24+
required: false
25+
default: ''
26+
enclave:
27+
description: 'Cold-start mode: kurtosis enclave name to dump.'
28+
required: false
29+
default: pos
30+
31+
runs:
32+
using: composite
33+
steps:
34+
- name: Dump devnet state
35+
shell: bash
36+
env:
37+
E2E_USE_SNAPSHOT: ${{ inputs.use_snapshot }}
38+
COMPOSE_FILE: ${{ inputs.compose_file }}
39+
E2E_KURTOSIS_ENCLAVE: ${{ inputs.enclave }}
40+
run: bash "${GITHUB_ACTION_PATH}/dump-devnet.sh"

‎.github/actions/e2e-dump/dump-devnet.sh‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
#!/usr/bin/env bash
2+
#
3+
# Capture devnet state into /tmp/devnet-dump for post-mortem after an e2e
4+
# failure. Invoked by the e2e-dump composite action. Two modes, picked by
5+
# E2E_USE_SNAPSHOT:
6+
#
7+
# - Snapshot mode (=1): dump the docker-compose stack's `ps` + `logs` and
8+
# copy the compose file itself. COMPOSE_FILE is the path the restore
9+
# composite emitted; it may be unset/absent if the restore failed before
10+
# writing it, so every step is guarded.
11+
# - Cold-start mode (=0): dump the kurtosis enclave named by
12+
# E2E_KURTOSIS_ENCLAVE (default `pos`).
13+
#
14+
# Best-effort throughout: a half-restored run should still yield whatever
15+
# artefacts it can, so individual failures are swallowed with `|| true`
16+
# rather than aborting the dump.
17+
#
18+
# Environment (set by action.yml):
19+
# E2E_USE_SNAPSHOT "1" for snapshot mode, "0" for cold-start.
20+
# COMPOSE_FILE snapshot mode: path to the restored docker-compose.
21+
# E2E_KURTOSIS_ENCLAVE cold-start mode: enclave name (default `pos`).
22+
23+
set -euo pipefail
24+
25+
DUMP_DIR="/tmp/devnet-dump"
26+
mkdir -p "$DUMP_DIR"
27+
28+
if [ "${E2E_USE_SNAPSHOT:-1}" = "1" ]; then
29+
compose_file="${COMPOSE_FILE:-}"
30+
if [ -n "$compose_file" ] && [ -f "$compose_file" ]; then
31+
docker compose --file "$compose_file" ps > "${DUMP_DIR}/compose-ps.txt" 2>&1 || true
32+
docker compose --file "$compose_file" logs --no-color > "${DUMP_DIR}/compose-logs.txt" 2>&1 || true
33+
cp "$compose_file" "${DUMP_DIR}/docker-compose.yaml" || true
34+
else
35+
echo "[dump-devnet] no compose file at '${compose_file:-<unset>}' — restore likely failed before writing it" >&2
36+
fi
37+
else
38+
enclave="${E2E_KURTOSIS_ENCLAVE:-pos}"
39+
kurtosis enclave dump "$enclave" "${DUMP_DIR}/${enclave}" || true
40+
fi

‎.github/actions/e2e-snapshot-restore/restore.sh‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,10 @@ cleanup() {
4848
if [ -n "${WORK_DIR:-}" ] && [ -d "$WORK_DIR" ]; then
4949
rm -rf "$WORK_DIR"
5050
fi
51+
# state_hex_file / body_file are ~2 MB temp files created in step 6. The
52+
# happy path rm's them inline, but curl or the JSON-RPC result check can
53+
# exit non-zero before that — clean them here so no temp leaks on failure.
54+
rm -f "${state_hex_file:-}" "${body_file:-}"
5155
exit "$exit_code"
5256
}
5357
trap cleanup EXIT
@@ -88,7 +92,11 @@ echo "[restore] extracting ${IMAGE} -> ${OUT_DIR_ABS}"
8892

8993
ADDRS_FILE="${OUT_DIR_ABS}/e2e-snapshot-addresses.json"
9094
echo "[restore] extracting addresses sidecar -> ${ADDRS_FILE}"
91-
addr_extract_cid="$(docker create "$IMAGE" /bin/true)"
95+
# `--entrypoint /bin/true` OVERRIDES any ENTRYPOINT the image declares; the
96+
# bare `docker create "$IMAGE" /bin/true` form only sets CMD, so an image with
97+
# an entrypoint would run `<entrypoint> /bin/true` instead and the create could
98+
# behave unexpectedly. We only need a stopped container to `docker cp` from.
99+
addr_extract_cid="$(docker create --entrypoint /bin/true "$IMAGE")"
92100
# Older snapshot images built before bake-addresses landed do not have this
93101
# file; tolerate that (the consumer suite then falls back to its kurtosis
94102
# CLI path or fails with a clear error).

‎.github/workflows/apps-e2e-trigger.yml‎

Lines changed: 9 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -47,23 +47,18 @@ permissions:
4747

4848
jobs:
4949
e2e:
50-
# Skip on PRs that don't carry the label. workflow_dispatch always runs.
50+
# `github.repository != '0xPolygon/pipelines'` keeps this template
51+
# inert in the pipelines repo itself — pipelines has no `pnpm run e2e`,
52+
# so a `run-e2e` label or a dispatch here would spawn an apps-e2e@main
53+
# run with nothing to execute. The guard is a no-op in any consumer
54+
# repo (its name is never `0xPolygon/pipelines`), so copy it verbatim.
55+
# Beyond that: skip PRs without the label; workflow_dispatch always runs.
5156
if: >-
52-
github.event_name != 'pull_request' ||
53-
contains(github.event.pull_request.labels.*.name, 'run-e2e')
57+
github.repository != '0xPolygon/pipelines' &&
58+
(github.event_name != 'pull_request' ||
59+
contains(github.event.pull_request.labels.*.name, 'run-e2e'))
5460
uses: 0xPolygon/pipelines/.github/workflows/apps-e2e.yml@main
5561
with:
56-
# Snapshot mode is the strong default. Cold-start is opt-in via
57-
# an explicit `use_snapshot=false` workflow_dispatch — every other
58-
# case (null on `pull_request: labeled`, true, "true", and a
59-
# "false" string passed via `gh workflow run -f`) resolves to
60-
# snapshot. The naive `inputs.use_snapshot == null && true ||
61-
# inputs.use_snapshot` looks correct but `false || false` in GH
62-
# Actions doesn't reliably round-trip a literal boolean false
63-
# through workflow_call's typed input — the called workflow saw
64-
# `use_snapshot: true` empirically. Compare against the boolean
65-
# AND the string explicitly so an explicit cold-start dispatch
66-
# actually flips the bit.
6762
# GH Actions' `==`/`!=` use loose equality where `null == false`
6863
# evaluates to true — so a `inputs.use_snapshot == null` check
6964
# also matches an explicit boolean false. Stringify via toJSON

‎.github/workflows/apps-e2e.yml‎

Lines changed: 56 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -12,9 +12,11 @@ name: E2E (kurtosis devnet)
1212
# snapshotted yet.
1313
#
1414
# Trigger files in consumer repos are thin: they specify the event
15-
# (workflow_dispatch + pull_request labeled), forward inputs, and call
16-
# this workflow with `secrets: inherit`. See apps-e2e-trigger.yml in
17-
# this repo for the canonical example.
15+
# (workflow_dispatch + pull_request labeled) and forward inputs. If the
16+
# consumer's snapshot image is in a private registry its own token can't
17+
# reach, the trigger passes REGISTRY_USERNAME / REGISTRY_PASSWORD by name
18+
# under `secrets:` (never `secrets: inherit` — see Team Standards). See
19+
# apps-e2e-trigger.yml in this repo for the canonical example.
1820

1921
on:
2022
workflow_call:
@@ -56,6 +58,31 @@ on:
5658
there. Wrap any pre-commands with && (e.g. starting a Firestore
5759
emulator) when needed; vitest globalSetup handlers are usually
5860
the cleaner home for those.
61+
62+
63+
ADAPTER CONTRACT — the suite this command runs must read the
64+
devnet's connection details from the environment this workflow
65+
sets, and the two modes expose them differently:
66+
67+
- Snapshot mode: contract addresses are in the JSON file at
68+
`$E2E_SNAPSHOT_ADDRESSES_JSON` (exported by the restore
69+
composite); RPCs are at the fixed host ports
70+
`127.0.0.1:8545` (L1) / `127.0.0.1:9545` (L2).
71+
- Cold-start mode: no addresses file. `$E2E_KURTOSIS_ENCLAVE`
72+
(default `pos`) names the running enclave; query it via the
73+
kurtosis CLI for addresses and dynamically-mapped ports.
74+
75+
A consumer therefore needs a dual-path adapter — see lst-api's
76+
`packages/e2e/scripts/up.ts` for the reference. This is real
77+
porting work, not a thin trigger.
78+
79+
80+
SECRETS — the suite step runs with only the registry secrets
81+
(below) and whatever `GITHUB_TOKEN` provides. There is no
82+
generic mechanism to inject arbitrary consumer secrets into the
83+
suite (`secrets: inherit` is prohibited by Team Standards). A
84+
consumer needing, e.g., an RPC token must have it added as a
85+
named secret on this workflow — open a PR here when that lands.
5986
type: string
6087
required: false
6188
default: pnpm run e2e
@@ -66,6 +93,20 @@ on:
6693
type: number
6794
required: false
6895
default: 45
96+
secrets:
97+
REGISTRY_USERNAME:
98+
description: >
99+
Username for the snapshot-image registry login. Optional —
100+
omit to use the workflow actor + GITHUB_TOKEN, which works for
101+
same-org GHCR pulls. A cross-org consumer (e.g. matic.js in
102+
maticnetwork pulling the 0xpolygon snapshot) passes its own
103+
credentials here so the private package resolves.
104+
required: false
105+
REGISTRY_PASSWORD:
106+
description: >
107+
Token for the snapshot-image registry login. Pair with
108+
REGISTRY_USERNAME. Optional; defaults to GITHUB_TOKEN.
109+
required: false
69110

70111
# `packages: read` lets the GITHUB_TOKEN pull from GHCR in snapshot mode;
71112
# ignored by cold-start runs that never touch the registry.
@@ -109,6 +150,11 @@ jobs:
109150
with:
110151
image: ${{ inputs.snapshot_image }}
111152
kurtosis_pos_ref: ${{ inputs.kurtosis_pos_ref }}
153+
# Fall back to the workflow actor + GITHUB_TOKEN when the caller
154+
# passes no registry secrets — preserves the same-org GHCR pull
155+
# the composite defaults to. A cross-org caller supplies its own.
156+
registry_username: ${{ secrets.REGISTRY_USERNAME || github.actor }}
157+
registry_password: ${{ secrets.REGISTRY_PASSWORD || github.token }}
112158

113159
- name: Cold-start kurtosis
114160
if: env.E2E_USE_SNAPSHOT == '0'
@@ -122,26 +168,15 @@ jobs:
122168
E2E_COMMAND: ${{ inputs.e2e_command }}
123169
run: bash -c "$E2E_COMMAND"
124170

125-
# On failure, capture devnet state for post-mortem. Snapshot mode
126-
# has a docker-compose stack we know the path to (via the composite
127-
# action's compose_file_path output); cold-start has a kurtosis
128-
# enclave to dump. `|| true` everywhere so a half-restored run
129-
# still produces whatever artefacts it can.
171+
# On failure, capture devnet state for post-mortem. Logic lives in
172+
# the e2e-dump composite (bundled script) rather than inline here —
173+
# keeps this YAML thin and the shell locally runnable.
130174
- name: Dump devnet on failure
131175
if: failure()
132-
env:
133-
COMPOSE_FILE: ${{ steps.snapshot.outputs.compose_file_path }}
134-
run: |
135-
mkdir -p /tmp/devnet-dump
136-
if [ "${E2E_USE_SNAPSHOT}" = "1" ]; then
137-
if [ -f "$COMPOSE_FILE" ]; then
138-
docker compose --file "$COMPOSE_FILE" ps > /tmp/devnet-dump/compose-ps.txt 2>&1 || true
139-
docker compose --file "$COMPOSE_FILE" logs --no-color > /tmp/devnet-dump/compose-logs.txt 2>&1 || true
140-
cp "$COMPOSE_FILE" /tmp/devnet-dump/docker-compose.yaml || true
141-
fi
142-
else
143-
kurtosis enclave dump pos /tmp/devnet-dump/pos || true
144-
fi
176+
uses: 0xPolygon/pipelines/.github/actions/e2e-dump@main
177+
with:
178+
use_snapshot: ${{ env.E2E_USE_SNAPSHOT }}
179+
compose_file: ${{ steps.snapshot.outputs.compose_file_path }}
145180

146181
- name: Upload devnet dump
147182
if: failure()

‎README.md‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,7 @@ system state automatically.
5858
| `actions/upsert-changeset-comment` | Post or remove changeset nag comment on a PR | Yes — ncc bundle |
5959
| `actions/e2e-snapshot-restore` | Restore a kurtosis-pos devnet from a published GHCR snapshot image | No — pure shell |
6060
| `actions/e2e-cold-start` | Bring up a kurtosis-pos devnet from source | No — pure shell |
61+
| `actions/e2e-dump` | Capture devnet state (compose logs or kurtosis enclave) on e2e failure | No — pure shell |
6162

6263
### `actions/ci`
6364

@@ -111,12 +112,15 @@ reusable workflow context, so they must be compiled ncc bundles (raw
111112
`.github/scripts/` files are not accessible in the calling repo's workspace).
112113
See [Adding a new composite action with compiled dist](#adding-a-new-composite-action-with-compiled-dist).
113114

114-
### `actions/e2e-snapshot-restore` and `actions/e2e-cold-start`
115+
### `actions/e2e-snapshot-restore`, `actions/e2e-cold-start`, `actions/e2e-dump`
115116

116117
Used by `apps-e2e.yml`. Each composite action ships its bash helper next to
117118
its `action.yml` and references it via `${{ github.action_path }}`, so no ncc
118119
bundle is needed — composite actions can read sibling files even when called
119-
cross-repo. Consumers should call `apps-e2e.yml` rather than the composites
120+
cross-repo. (A reusable workflow can't run a repo-local `.github/scripts/`
121+
helper cross-repo for the same reason, which is why the on-failure dump is a
122+
composite — `e2e-dump` — rather than an inline `run:` block or a scripts
123+
file.) Consumers should call `apps-e2e.yml` rather than the composites
120124
directly; the workflow handles the snapshot/cold-start switch and on-failure
121125
devnet artefact upload.
122126

0 commit comments

Comments
 (0)