Skip to content

Commit 4014419

Browse files
committed
New lint-deps command
1 parent 5f4a06a commit 4014419

15 files changed

Lines changed: 1462 additions & 28 deletions

File tree

.claude/skills/repomatic-ship/SKILL.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -116,6 +116,7 @@ The checks:
116116
- **Tests**: `uv run pytest --no-header -q`. Exception: an integration-heavy suite driving real external tooling can outrun a local background timeout and need tools not installed locally, so it is not a fast gate. Skip it, keep the rest of the gate, and treat the CI matrix on the exact commit as the authoritative test signal (step 6 covers dispatching one). Between the extremes, a suite whose local runtime approaches CI's fast platforms (~5-8 minutes from push) stops blocking. Start it with the gate, push once every fast check is green, and fold the still-running suite into step 6 as the first babysit channel: a failure lands as an immediate tight-loop fix at the same absolute time CI would have reported it, while a pass cost zero wall-clock.
117117
- **Types**: mypy over an **explicit list of tracked files**, the shape CI's lint job uses (it pipes `metadata`'s `python_files` through `xargs`): `git ls-files '*.py' | xargs <cmd> run mypy --`. Do **not** pass directory names instead (`<cmd> run mypy -- repomatic tests docs`): directories change module resolution enough that mypy follows an installed dependency's own source, so a package pulled in by the `docs` group and written for a newer Python fails the run under `--python-version 3.10` with a syntax error in a file this project does not own, which reads as a real failure and is not one. Either way the scope must cover every tracked `*.py`, `docs/conf.py` included, or the gate stays green on a `docs/` error that reddens Lint post-push.
118118
- **Changelog**: `<cmd> lint-changelog`. A `⚠ X.Y.Z: not found on PyPI` warning for the still-unreleased version is expected and not a blocker.
119+
- **Shippable deps**: `<cmd> lint-deps`. Offline and instant, and it covers the one release failure nothing else in this gate can see: a `[tool.uv.sources]` override never reaches the published metadata, so tests, types, formatting and the build all pass on a tree whose wheel every user then fails to install. Run it even on a docs-only cycle, where a lockfile bump can still have moved a source. A blocker naming a git source paired with a `.dev` floor is the `sync-dep-sources` idiom mid-flight: the fix is to wait for that swap PR, not to edit `pyproject.toml` by hand. The release lane carries the same check as a hard gate, but it fires after the freeze commit is already on `main`, where the only recovery is to burn the version and ship the next one, so a red here is cheap and a red there is not.
119120
- **Formatting**, reproduced with the **pinned** tools, never the dev-env `uv run ruff` (a newer local ruff once silently disagreed on a `PERF401` fix): `<cmd> run autopep8 --` over the cycle's changed Python files (it wraps long-line comments ruff leaves), fed the way the Types gate feeds mypy — `git diff --name-only HEAD -- '*.py' | xargs <cmd> run autopep8 --` — and never as a shell variable holding the space-separated list, which the runner takes as one literal path and rejects with `[Errno 2] No such file or directory: 'a.py b.py c.py '`, then `<cmd> run ruff -- check` and `<cmd> run ruff -- format` (all write in place; the runner injects `--fix`), then read `git diff`: write-mode output is the reliable signal, `--check` is not. An empty diff past your reconciliation edits is green; fold a legitimate style fix into the reconciliation. For any Markdown the reconciliation touched (`changelog.md`, `docs/`), verify with the pinned `<cmd> run mdformat -- <file>`, never a bare `mdformat`/`mdformat --with mdformat-myst`: the bare form rewrites MyST directive colon-options (a `{list-table}`'s `:header-rows:`/`:widths:`) to `---` frontmatter form, diverging from CI's autofix and injecting a spurious reflow you would then have to revert.
120121
- Landmine: autopep8 relocates a trailing `# type: ignore[...]` off a >88-char line onto its own line, voiding the suppression (Lint red under `warn_unused_ignores`); ruff format usually reverts the relocation, so only wraps that survive the full pinned sequence are real formatting debt. Never commit the relocation: fix the line length at the source so the comment rides the opening line.
121122
- **Autofix externals**: smoke-run `<cmd> run typos`, **every formatter that downloads a checksum-pinned binary** (`<cmd> run biome` and peers), and the vulnerable-deps scan `<cmd> audit --fix` (parses live `uv audit` output). An upstream re-publish flips a pinned SHA-256 and kills the step; pytest mocks these, so the drift (or a changed output schema) surfaces only here or in CI's `autofix` run. A pin living upstream in `repomatic` breaks every downstream repo and cannot be patched here: surface it for the step-8 upstream report. Invocation rules:

.github/workflows/_release-build.yaml

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -134,9 +134,47 @@ jobs:
134134
is_python_project cli_scripts package_name current_version released_version
135135
release_notes release_notes_with_admonition skip_binary_build
136136
137+
lint-deps:
138+
name: 🔗 Lint deps
139+
# Refuse to publish a project whose dependencies do not all resolve from the index its users
140+
# install from. A `[tool.uv.sources]` override never reaches the published metadata, so the
141+
# wheel builds and uploads exactly as it would have and only the install fails: nothing else
142+
# in the lane can see it. Fatal only on a release commit, so test-driving a git branch during
143+
# development stays frictionless while still annotating every push.
144+
# See repomatic/dep_sources.py for the rules and the failure classes.
145+
needs:
146+
- metadata
147+
if: fromJSON(needs.metadata.outputs.metadata).is_python_project
148+
runs-on: ubuntu-slim
149+
steps:
150+
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
151+
with:
152+
# The release commit, not the push head: a rebase-merged release PR delivers the freeze
153+
# and the post-release bump together, so `main` HEAD already carries the next `.devN`.
154+
# The tree being published is the one that has to pass. Most recent release commit
155+
# first, as in release.yaml's pack-plugin; a push carrying two of them checks only the
156+
# newest, which is the same assumption the rest of the lane makes.
157+
ref: >-
158+
${{ fromJSON(needs.metadata.outputs.metadata).release_commits_matrix
159+
&& fromJSON(needs.metadata.outputs.metadata).release_commits_matrix.commit[0]
160+
|| github.sha }}
161+
- uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
162+
with:
163+
version: "0.12.1"
164+
- name: Check dependency sources
165+
run: >
166+
uv --no-progress run --frozen -- repomatic lint-deps
167+
${{ fromJSON(needs.metadata.outputs.metadata).release_commits_matrix
168+
&& '--fatal' || '--no-fatal' }}
169+
137170
build-package:
138171
name: 📦 Build package (${{ matrix.short_sha }})
172+
# `lint-deps` is a hard gate, not a report: a failure here skips this job, which leaves
173+
# `package_built` false so the caller's publish-pypi never fires, and fails the whole lane so
174+
# the engine's create-tag / create-release / publish-release are skipped with it. Nothing
175+
# reaches PyPI or GitHub. tests/test_workflows.py holds this edge.
139176
needs:
177+
- lint-deps
140178
- metadata
141179
if: fromJSON(needs.metadata.outputs.metadata).is_python_project
142180
strategy:

changelog.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@
88
- Teach the `repomatic-ship` skill to reconcile bundled skills and agents as a third pass, judge a false-positive autofix PR against current `main` before writing a lint rule for it, and tell a superseded intra-cycle measurement from a genuine contradiction.
99
- **Breaking:** the `update-dep-graph` job moved from `autofix.yaml` into the release engine, its only firing moment being a release push. A required check or `needs:` edge naming it under Autofix must follow.
1010
- **Breaking:** manual version-bump commits now read `` [changelog] Bump minor version to `vX.Y.0` ``: every version-machinery commit carries the `[changelog] ` prefix, and anything matching the old unprefixed titles must follow.
11+
- New `lint-deps` command and release-lane job, blocking a release whose dependencies do not all resolve from PyPI: git branches, forks, local paths, direct URLs and private indexes. Exempt a package with `[tool.repomatic] lint-deps.allow`.
1112
- New `runner-images` job and CLI command, opening an issue that lists GitHub's open runner-image announcements and flags the ones retiring an image the repo runs on. Opt out with `runner-images = false`.
1213
- Autofix jobs no longer run on version-bump pushes: the 16 remaining jobs only re-checked machine-generated, ship-gated commits, and drift stays covered by the next push and the weekly sweep.
1314
- The tool runner now retries a download up to 3 times on transient network failures, instead of failing the job on a one-off TLS or truncation error.

claude.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -106,6 +106,14 @@ exclude-newer-package = { plumage = "0 days" }
106106

107107
Declaring it in `pyproject.toml` rather than in a machine's `~/.config/uv/uv.toml` is what makes a fresh clone resolve the same way, and keeps the exemption reviewable in a diff. It stays a bypass and not a hole: the transitive tree that release pulls in is still gated, and that tree is the part the maintainer did not publish.
108108

109+
## A release ships only released dependencies
110+
111+
Consuming a dependency from a git branch, a fork, a local path or a private index is the right move mid-cycle, and `[tool.uv.sources]` exists for it. Releasing while one is in place is not: source overrides never reach the published metadata, so the wheel builds and uploads exactly as it would have, and only the *install* fails. Nothing in this repository can feel it, because every workflow here installs from `uv.lock` and resolves straight through the override.
112+
113+
`repomatic lint-deps` is the gate, and it blocks everything off-index, `[dependency-groups]` included. Do not weaken a finding to a warning to get a release out; the two legitimate moves are to land the swap (`sync-dep-sources` automates the git-branch-plus-`.dev`-floor idiom on its own) or to name the package in `[tool.repomatic] lint-deps.allow` with the reason it is safe. Adding a rule here means adding it to {func}`repomatic.dep_sources.scan_pyproject` or {func}`~repomatic.dep_sources.scan_lock`, never to a call site.
114+
115+
The gate runs in four places, and the release lane's copy is the backstop, not the mechanism: by the time it fires the freeze commit is already on `main` and the recovery is to burn the version per [§ Skip and move forward](#skip-and-move-forward-dont-rewrite-history). The layer that prevents that is the `prepare-release` PR banner, which is regenerated on every push. See [`docs/dependencies.md` § Shippable sources](https://kdeldycke.github.io/repomatic/dependencies.html#shippable-sources) for the rules and the failure classes.
116+
109117
## Documentation requirements
110118

111119
### Keeping `claude.md` lean

docs/dependencies.md

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,70 @@ This page documents the version specifier conventions and dependency audit proce
4141
- **Extras syntax** is fine: `"coverage[toml]>=7.11"`.
4242
- **One dependency per line** for readable diffs. Short groups that fit on one line are acceptable: the `format-json` workflow normalizes layout automatically.
4343

44+
## Shippable sources
45+
46+
A dependency is **shippable** when whoever installs the published artifact gets the same code the release was tested against. `repomatic lint-deps` checks that, and the release lane refuses to build a package while any dependency fails it.
47+
48+
The check is offline, reading `pyproject.toml` and `uv.lock`. The lockfile pass is what makes it complete rather than a list of hand-written rules: it records the resolved origin of every package in the tree, so a git dependency pulled in by another git dependency shows up even though no table names it.
49+
50+
### Why a gate rather than a code review
51+
52+
Two failure classes hide behind "I'll remember to swap it back before releasing", and they fail very differently.
53+
54+
**A direct reference is refused at upload.** PyPI rejects a distribution whose `Requires-Dist` carries a PEP 508 direct reference (`mango @ git+https://…`). That detection exists already, it just happens far too late: the tag is created, the GitHub release is published and the binaries are built before `publish-pypi` fails. `repomatic`'s own `5.0.0` shipped this way, carrying `mdformat-pelican @ git+…` in an extra. It is a GitHub release with no PyPI counterpart to this day, and `5.0.1` shipped the same afternoon to "fix publishing to PyPI by removing URL-based dependency on `mdformat-pelican`". The version number is burned permanently.
55+
56+
**A `[tool.uv.sources]` override is not detectable anywhere.** Source overrides never reach published metadata, so this:
57+
58+
```toml
59+
[project]
60+
dependencies = ["mango>=2.1.0.dev0"]
61+
62+
[tool.uv.sources]
63+
mango = { git = "https://github.com/acme/mango", branch = "main" }
64+
```
65+
66+
builds a valid wheel, uploads cleanly, and declares a requirement PyPI cannot satisfy. Every install of that release fails. The repository's own CI stays green throughout, because it installs from `uv.lock` and resolves straight through the override. Nothing short of a gate catches it.
67+
68+
### What blocks
69+
70+
| Source | Example |
71+
| :-------------------------- | :-------------------------------------------------------------------------------------------- |
72+
| A git repository | `{ git = "…", branch = "main" }`, and the `tag` and `rev` forms |
73+
| A local path | `{ path = "../mango" }`, with or without `editable` |
74+
| A direct artifact URL | `{ url = "https://…/mango-2.1-py3-none-any.whl" }` |
75+
| A workspace member | `{ workspace = true }` |
76+
| An index other than PyPI | `{ index = "internal" }`, or a `default = true` index pointing elsewhere |
77+
| A PEP 508 direct reference | `mango @ git+https://…`, in any requirement array |
78+
| A floor inside the cooldown | see [§ `exclude-newer-package` cooldown overrides](#exclude-newer-package-cooldown-overrides) |
79+
80+
Everything blocks, including a source on a package declared only in `[dependency-groups]`, which never reaches an installer. `uv.lock` pins a tracked branch to a revision, and once that branch is force-pushed or the fork disappears, the released tag no longer builds for contributors.
81+
82+
`override-dependencies` and `constraint-dependencies` are reported as warnings instead. They name a published version rather than an unreleased artifact, so the result still installs; what they change is the tree the release was tested against.
83+
84+
The report says which of these applies to each finding, because the remedies differ. A git source paired with a `.dev` floor is the managed `sync-dep-sources` idiom described at the end of [§ `exclude-newer-package` cooldown overrides](#exclude-newer-package-cooldown-overrides): the swap PR opens on its own once the awaited release ships, so the fix is to wait rather than to edit anything.
85+
86+
### Where it runs
87+
88+
Four layers, ordered by how cheap the failure is:
89+
90+
1. **Locally**, through the `/repomatic-ship` pre-push gate.
91+
2. **The release PR body**, regenerated on every push to `main`. A blocker replaces the checklist's "This PR is ready to be merged" opener with a `[!CAUTION]` block naming each offending dependency. This is the layer that matters: it reaches the maintainer at the moment they decide to merge.
92+
3. **The release lane**, as `_release-build.yaml`'s `lint-deps` job. Fatal only on a release commit, so test-driving a git branch mid-cycle stays frictionless. `build-package` depends on it, so a failure skips the wheel build, which leaves `package_built` false and skips `publish-pypi`, and fails the lane, which skips the engine's tag, release and publish jobs with it.
93+
4. **The test suite**, upstream only, as a millisecond-latency copy of the same check.
94+
95+
Layer 3 is a backstop that should never fire. By the time it does, the freeze commit is already on `main` and the recovery is to burn the version per the skip-and-move-forward rule.
96+
97+
### Allowing an exception
98+
99+
Some arrangements are legitimate: a monorepo member published under its own name, a private mirror an internal project genuinely targets. Name each one in `[tool.repomatic]`, mapped to the reason it is safe:
100+
101+
```toml
102+
[tool.repomatic]
103+
lint-deps.allow = { papaya = "monorepo workspace member, published separately" }
104+
```
105+
106+
A mapping rather than a list, because the reason is the point: an exemption without one is indistinguishable from a forgotten development shortcut six months later. The reason renders in the report and in the release PR banner, so an accepted exception stays visible rather than disappearing. Listing a package does not exempt its transitive dependencies, which stay gated on their own.
107+
44108
## Floor verification
45109

46110
Comments and changelogs can lie; the codebase is the source of truth. For each dependency with a weak or suspicious comment, verify the floor against actual usage:

0 commit comments

Comments
 (0)