|
| 1 | +--- |
| 2 | +name: portal-ponder-release |
| 3 | +description: Cut a new @subsquid/ponder release — the Portal fork of ponder, versioned <ponder-version>-sqd.<rev>. Bump/confirm the target ponder version in versions.json, trigger the release workflow (manual dispatch or a v<version> tag), watch it build+test+publish to npm via Trusted Publishing, then flip the versions.json row to published and write GitHub release notes for the v<version> tag. Use when the user asks to "release", "publish", "cut a version", "ship", or "re-cut a fork revision" for @subsquid/ponder / portal-ponder. |
| 4 | +--- |
| 5 | + |
| 6 | +# @subsquid/ponder release |
| 7 | + |
| 8 | +End-to-end release procedure for **`@subsquid/ponder`** — the drop-in Portal fork of `ponder`. |
| 9 | +A release is `ponder@<version>` + the Portal layer (`portal/`), published to npm as |
| 10 | +`@subsquid/ponder@<version>-sqd.<rev>`. The bin stays `ponder`, so it is a drop-in install. |
| 11 | + |
| 12 | +The fork is **generated, not hand-maintained**: [`scripts/sync-upstream.sh <ver>`](../../../scripts/sync-upstream.sh) |
| 13 | +clones `ponder@<ver>`, drops in the two Portal modules + the per-version `portal/wiring/<ver>.patch`, |
| 14 | +renames the package, and builds. The publishable artifact lives in the generated tree at |
| 15 | +`packages/core`, **not** in this repo — this repo only carries the Portal layer. |
| 16 | + |
| 17 | +Publishing runs on **npm Trusted Publishing** (OIDC, no `NPM_TOKEN`) via |
| 18 | +[`.github/workflows/release.yml`](../../../.github/workflows/release.yml). The workflow guards that |
| 19 | +the requested version is declared in [`versions.json`](../../../versions.json) before it builds. |
| 20 | + |
| 21 | +Full background: [`PUBLISHING.md`](../../../PUBLISHING.md) and `versions.json` are the sources of truth. |
| 22 | + |
| 23 | +> **Scope:** this releases `@subsquid/ponder` only. Do not confuse the **git tag** (`v<ponder-version>`, |
| 24 | +> e.g. `v0.16.6`) with the **npm version** (`<ponder-version>-sqd.<rev>`, e.g. `0.16.6-sqd.1`) — see |
| 25 | +> [The two version forms](#the-two-version-forms). |
| 26 | +
|
| 27 | +## Preconditions |
| 28 | + |
| 29 | +Confirm before starting: |
| 30 | + |
| 31 | +- The user named a **ponder version** to release (e.g. `0.16.6`). If not, ask. It must already exist |
| 32 | + in `versions.json` with a `portal/wiring/<ver>.patch`. If it doesn't, this is an *add a version* |
| 33 | + task first — see [Adding a new ponder version](#adding-a-new-ponder-version). |
| 34 | +- The target row's wiring patch **applies + builds + tests green** locally: |
| 35 | + `SYNC_REV=<rev> scripts/sync-upstream.sh <ver> --test`. CI runs this for every row on each push, |
| 36 | + so a green `ci.yml` on the branch is a good proxy — but re-run it if the patch or Portal modules |
| 37 | + changed since the last CI run. |
| 38 | +- You know the **fork revision** (`rev`). First cut of a ponder version is `1`. Bump it (`2`, `3`, …) |
| 39 | + only to re-cut a fork-side fix on the **same** ponder version — npm permanently retires an |
| 40 | + unpublished version number, so `0.16.6-sqd.1` can never be re-published after a bad build. |
| 41 | +- A **trusted publisher** for `@subsquid/ponder` is configured on npmjs.com pointing at |
| 42 | + `subsquid-labs/portal-ponder` → `release.yml`. One-time setup — see |
| 43 | + [First-time setup](#first-time-setup-trusted-publishing). Without it, the publish step 404s. |
| 44 | + |
| 45 | +## The two version forms |
| 46 | + |
| 47 | +This trips people up. There are two distinct strings: |
| 48 | + |
| 49 | +| Form | Example | Where it lives | |
| 50 | +|------|---------|----------------| |
| 51 | +| **git tag** | `v0.16.6` | what you push to trigger the workflow; the bare ponder version, prefixed `v` | |
| 52 | +| **npm version** | `0.16.6-sqd.1` | what actually publishes; ponder version + `-sqd.<rev>` | |
| 53 | + |
| 54 | +The workflow strips `v` off the tag → `0.16.6`, guards it against `versions.json` (which stores |
| 55 | +**bare** ponder versions), then `sync-upstream.sh` appends `-sqd.<rev>` at build time. |
| 56 | + |
| 57 | +**Consequence:** tag `v0.16.6`, never `v0.16.6-sqd.1` — the `-sqd.1` form would fail the |
| 58 | +`versions.json` guard (no such row). And the **tag-push path always publishes `rev` = 1** |
| 59 | +(`SYNC_REV` defaults to `1` on a tag push). To publish `rev` ≥ 2, you **must** use the manual |
| 60 | +`workflow_dispatch` and pass the `rev` input — a tag push can't express a revision bump. |
| 61 | + |
| 62 | +## Steps |
| 63 | + |
| 64 | +### 1. Pick the trigger |
| 65 | + |
| 66 | +Two ways in, same `publish` job: |
| 67 | + |
| 68 | +- **Manual (preferred, and required for rev ≥ 2):** Actions → **release** → *Run workflow* → |
| 69 | + enter `version` (e.g. `0.16.6`) and `rev` (e.g. `1`). From the CLI: |
| 70 | + |
| 71 | + ```sh |
| 72 | + gh workflow run release.yml --repo subsquid-labs/portal-ponder -f version=0.16.6 -f rev=1 |
| 73 | + ``` |
| 74 | + |
| 75 | +- **Tag push (rev 1 only):** push a `v<version>` tag on the release branch. |
| 76 | + |
| 77 | + ```sh |
| 78 | + git tag v0.16.6 |
| 79 | + git push origin v0.16.6 |
| 80 | + ``` |
| 81 | + |
| 82 | + The workflow triggers on `tags: ['v*.*.*']`. This path is `rev` = 1 by construction. |
| 83 | + |
| 84 | +### 2. Watch the workflow |
| 85 | + |
| 86 | +```sh |
| 87 | +gh run watch --repo subsquid-labs/portal-ponder --exit-status |
| 88 | +``` |
| 89 | + |
| 90 | +Or grab the latest run explicitly: |
| 91 | + |
| 92 | +```sh |
| 93 | +RUN_ID=$(gh run list --repo subsquid-labs/portal-ponder --workflow=release.yml --limit 1 --json databaseId --jq '.[0].databaseId') |
| 94 | +gh run watch "$RUN_ID" --repo subsquid-labs/portal-ponder --exit-status |
| 95 | +``` |
| 96 | + |
| 97 | +The single `publish` job runs: resolve version → **guard against `versions.json`** → |
| 98 | +`sync-upstream.sh <ver> --test` (apply layer, build, run Portal unit tests) → `npm publish`. If the |
| 99 | +publish step fails on Trusted Publishing, the npm-side config is missing/misconfigured — surface the |
| 100 | +error, don't retry blindly. See [Failure modes](#failure-modes). |
| 101 | + |
| 102 | +### 3. Confirm the publish and dist-tag |
| 103 | + |
| 104 | +The workflow tags `latest` **only** for the newest ponder version in `versions.json`; publishing an |
| 105 | +older version (e.g. `0.15.17` after `0.16.6`) publishes it under `ponder-<version>` so it does not |
| 106 | +clobber the default `npm install`. Verify where it landed: |
| 107 | + |
| 108 | +```sh |
| 109 | +npm view @subsquid/ponder dist-tags |
| 110 | +npm view @subsquid/ponder@<version>-sqd.<rev> version # confirm the exact build exists |
| 111 | +``` |
| 112 | + |
| 113 | +`latest` should point at the newest ponder version's `-sqd` build; an older release should sit under |
| 114 | +`ponder-<version>` and users pin the exact `@subsquid/ponder@<version>-sqd.<rev>`. |
| 115 | + |
| 116 | +### 4. Flip the versions.json row to published |
| 117 | + |
| 118 | +After a successful publish, update the row in [`versions.json`](../../../versions.json): set |
| 119 | +`"published": true` (leave `"status"` as the verification state — `"verified"`; the schema tracks |
| 120 | +publish state in the separate `published` boolean, not via a `status` value), bump `"rev"` if you cut |
| 121 | +a new revision, and trim the `note` to reflect reality (drop stale "publish candidate / BROKEN seed" |
| 122 | +caveats once they no longer apply). Commit: |
| 123 | + |
| 124 | +```sh |
| 125 | +git add versions.json |
| 126 | +git commit -m "chore(release): @subsquid/ponder <version>-sqd.<rev> published" |
| 127 | +git push |
| 128 | +``` |
| 129 | + |
| 130 | +This is a real state change other releases and CI read — don't skip it. |
| 131 | + |
| 132 | +### 5. Write the GitHub release notes |
| 133 | + |
| 134 | +**Required — every release gets one.** `release.yml` publishes to npm only; it does **not** create a |
| 135 | +GitHub Release, so this is a manual step you always do. Write curated, Portal-layer-scoped highlights |
| 136 | +(what the fork changed vs plain `ponder@<version>`: the Portal backfill seam, wiring touch-points, |
| 137 | +correctness fixes, tests) — **not** a restatement of ponder's own changelog. Use |
| 138 | +[release-template.md](release-template.md) as the shape, and source highlights from the commits since |
| 139 | +the previous `-sqd` cut. Apply from a file to avoid heredoc escaping around code fences: |
| 140 | + |
| 141 | +```sh |
| 142 | +# Tag-push release: the v<version> tag already exists — just attach the release. |
| 143 | +gh release create v<version> --repo subsquid-labs/portal-ponder \ |
| 144 | + --title "@subsquid/ponder <version>-sqd.<rev>" --notes-file notes.md |
| 145 | + |
| 146 | +# If the release already exists (e.g. re-editing notes): |
| 147 | +gh release edit v<version> --repo subsquid-labs/portal-ponder --notes-file notes.md |
| 148 | +``` |
| 149 | + |
| 150 | +**Manual-dispatch releases have no tag** (a `workflow_dispatch` run doesn't push one). `gh release |
| 151 | +create` will create the `v<version>` tag for you, but it defaults to the branch HEAD — pin it to the |
| 152 | +exact commit that was released with `--target`: |
| 153 | + |
| 154 | +```sh |
| 155 | +gh release create v<version> --repo subsquid-labs/portal-ponder \ |
| 156 | + --target <released-commit-sha> \ |
| 157 | + --title "@subsquid/ponder <version>-sqd.<rev>" --notes-file notes.md |
| 158 | +``` |
| 159 | + |
| 160 | +For a re-cut (`rev` ≥ 2), the `v<version>` tag from the earlier revision already exists — don't move |
| 161 | +it. Either append the revision's notes to the existing release, or create a distinct |
| 162 | +`v<version>-sqd.<rev>` release tag for that revision. |
| 163 | + |
| 164 | +### 6. Print the summary |
| 165 | + |
| 166 | +```sh |
| 167 | +echo "npm: @subsquid/ponder@<version>-sqd.<rev>" |
| 168 | +echo "tag: https://github.com/subsquid-labs/portal-ponder/releases/tag/v<version>" |
| 169 | +npm view @subsquid/ponder dist-tags |
| 170 | +``` |
| 171 | + |
| 172 | +## Re-cutting a fork revision (same ponder version) |
| 173 | + |
| 174 | +A bad build on `<version>-sqd.1` can't be re-published under the same number. To ship a fork-side fix |
| 175 | +on the **same** ponder version: |
| 176 | + |
| 177 | +1. Fix the Portal layer (`portal/*.ts` or `portal/wiring/<ver>.patch`) and land it on the branch. |
| 178 | +2. `SYNC_REV=2 scripts/sync-upstream.sh <ver> --test` — confirm green. |
| 179 | +3. Bump the row's `"rev"` in `versions.json`. |
| 180 | +4. Release via **manual dispatch** with `rev=2` (a tag push can't — it's always rev 1). |
| 181 | + |
| 182 | +## Adding a new ponder version |
| 183 | + |
| 184 | +If the requested ponder version has no `versions.json` row / wiring patch, it can't be released yet. |
| 185 | +Per [`PUBLISHING.md`](../../../PUBLISHING.md): |
| 186 | + |
| 187 | +1. Author `portal/wiring/<ver>.patch` — the 4 touch-points (`internal/types.ts`, `config/index.ts`, |
| 188 | + `build/config.ts`, `runtime/historical.ts`). See an existing patch for the shape. |
| 189 | +2. `scripts/sync-upstream.sh <ver> --test` — applies, builds, Portal tests pass. |
| 190 | +3. Add a `{ "ponder": "<ver>", "wiring": "wiring/<ver>.patch", "rev": 1, "status": "verified" }` row |
| 191 | + and add `<ver>` to `compat.tested`. |
| 192 | +4. Commit (CI proves the seam on every push). Then release with the steps above. |
| 193 | + |
| 194 | +## First-time setup (Trusted Publishing) |
| 195 | + |
| 196 | +Publishing uses OIDC — there is no `NPM_TOKEN`. Once, before the first automated release, a maintainer |
| 197 | +with publish rights on the `@subsquid` scope configures a trusted publisher on npmjs.com: |
| 198 | + |
| 199 | +- `@subsquid/ponder` → **Settings** → **Trusted Publisher** → GitHub Actions. |
| 200 | +- Repository: `subsquid-labs/portal-ponder` |
| 201 | +- Workflow filename: `release.yml` |
| 202 | + |
| 203 | +If npm requires the package to exist before a trusted publisher can be added, **seed it with one |
| 204 | +manual publish**, then configure the publisher — every release after that is tokenless: |
| 205 | + |
| 206 | +```sh |
| 207 | +SYNC_REV=1 SYNC_WORKDIR=/tmp/sqd-fork scripts/sync-upstream.sh <ver> --test # build + test locally |
| 208 | +cd /tmp/sqd-fork/<ver>/packages/core && npm publish --access public --tag latest # requires npm login |
| 209 | +``` |
| 210 | + |
| 211 | +For this **manual seed publish only**, add `--provenance` when the repo is public for a signed |
| 212 | +provenance attestation. The automated `release.yml` path needs nothing — npm attaches provenance |
| 213 | +automatically under OIDC Trusted Publishing. |
| 214 | + |
| 215 | +## Failure modes |
| 216 | + |
| 217 | +- **Guard fails: "`<ver>` not in versions.json"**: the version has no row / wiring patch. Add it first |
| 218 | + — see [Adding a new ponder version](#adding-a-new-ponder-version). Don't hand-edit the workflow to |
| 219 | + skip the guard. |
| 220 | +- **Tag pushed as `v<ver>-sqd.<rev>`**: it fails the `versions.json` guard (rows are bare ponder |
| 221 | + versions). Delete the tag, push `v<ver>` instead. For a revision bump, use manual dispatch with the |
| 222 | + `rev` input, not a tag. |
| 223 | +- **`sync-upstream.sh` fails (patch won't apply / build breaks)**: the wiring patch drifted against |
| 224 | + that ponder version, or the Portal modules broke the build. Reproduce locally with |
| 225 | + `SYNC_REV=<rev> scripts/sync-upstream.sh <ver> --test`, fix the patch/modules, re-run. This is the |
| 226 | + same step CI runs, so a red `ci.yml` points at the same fix. |
| 227 | +- **Publish fails on Trusted Publishing (404/401)**: the trusted publisher for `@subsquid/ponder` is |
| 228 | + missing or points at the wrong repo/workflow. See |
| 229 | + [First-time setup](#first-time-setup-trusted-publishing). Don't fall back to a token-based |
| 230 | + `npm publish` unless the user explicitly asks. |
| 231 | +- **npm >= 11.5.1 required**: OIDC publishing needs a recent npm; node 22 ships an older one. The |
| 232 | + workflow already does `npm install -g npm@latest` before publishing — if you publish locally, do |
| 233 | + the same or the OIDC step is unavailable. |
| 234 | +- **Version already published**: `0.16.6-sqd.1` exists on npm and can't be re-published. Bump the |
| 235 | + `rev` and re-cut via manual dispatch — see |
| 236 | + [Re-cutting a fork revision](#re-cutting-a-fork-revision-same-ponder-version). |
| 237 | +- **`latest` clobbered by an older release**: publishing an older ponder version should route to |
| 238 | + `ponder-<version>`, not `latest` — the workflow computes the max from `versions.json`. If `latest` |
| 239 | + drifted (e.g. a manual publish), repoint it: `npm dist-tag add @subsquid/ponder@<newest-sqd> latest` |
| 240 | + (requires npm auth — a manual recovery outside the OIDC workflow). |
0 commit comments