Skip to content

README.md

Latest commit

 

History

History
130 lines (95 loc) · 5.31 KB

README.md

File metadata and controls

130 lines (95 loc) · 5.31 KB
Raw

vendored_invoke

A portable invoke you can drop into any repository to build a development workflow. Consumers need nothing but Python — no virtualenv, no pip install, no lockfile. _CI/lib/vendor/ holds the libraries, and workflow runs them.

Maintaining this repository needs one thing: uv.

What is in the vendored tree, and why

Everything in _CI/lib/vendor/ is copied into every repository that adopts this template, so the bar for adding something is high: the shipped code under _CI/ must import it.

Package Why
invoke The task runner. The whole point.
coloredlogs Log formatting, imported by _CI/tasks/__init__.py.
humanfriendly Not a choice — coloredlogs requires it.

That is 80 files and ~886 KiB, all pure Python.

Two things are deliberately not vendored:

  • The vendoring toolchain (vendoring, pip-tools). It used to be, which made re-vendoring self-hosting — but it also shipped ~5.8 MB of maintainer tooling, plus rich, requests, jsonschema, pygments, urllib3 and friends, to every consumer. It is now run on demand with uvx.
  • tomli. configuration.py parses TOML, but tomli only publishes mypyc-compiled wheels, so vendoring it produced a __mypyc.cpython-314-darwin.so — a 427 KiB binary tied to one OS, architecture and Python version, in a tree whose purpose is running anywhere. stdlib tomllib does the same job for free, which is why requires-python is >=3.11.

Re-vendoring from scratch

Four steps. Run them from _CI/, except where noted.

1. Declare what you want

_CI/pyproject.toml has two extras, and the distinction matters:

[project.optional-dependencies]
vendor      = ['invoke', 'coloredlogs']   # content — ends up in lib/vendor/
maintenance = ['vendoring==1.2.0']        # tooling — runs the vendoring, never vendored

Anything you add to vendor is resolved with its full dependency tree and copied in. Adding one convenience library can pull a dozen packages; check what you are signing up for before committing the result.

2. Resolve the dependency tree

uv pip compile --extra=vendor -o lib/vendor.txt pyproject.toml

This replaces pip-compile. It writes the exact command into the file header using the relative path you gave it, so there is no absolute path to strip afterwards.

3. Sync the tree

uvx --from vendoring==1.2.0 --with pip vendoring sync .

--with pip is required: vendoring shells out to pip to download, and a uvx tool environment has no pip of its own.

This clears lib/vendor/, downloads everything in lib/vendor.txt, rewrites absolute imports into the lib.vendor namespace, applies drop rules and lib/patches/, and generates .pyi stubs. Configuration lives in [tool.vendoring] in _CI/pyproject.toml.

4. Generalise the shebangs

python - <<'PY'
import pathlib
for script in sorted(pathlib.Path('lib/vendor/bin').iterdir()):
    lines = script.read_text(encoding='utf-8').splitlines(keepends=True)
    lines[0] = '#!/usr/bin/env python\n'
    script.write_text(''.join(lines), encoding='utf-8')
    script.chmod(0o755)
PY

pip writes the absolute path of whatever interpreter did the install into each console script. Left alone, the committed tree only runs on the machine that generated it.

Then verify before committing:

python lib/vendor/bin/invoke --version
git diff --stat            # from the repository root

The path-injection patch

lib/patches/path_inject.patch is the one hand-maintained piece. It inserts a sys.path prelude into lib/vendor/bin/inv and lib/vendor/bin/invoke so the launcher can import both the vendored libraries and the consuming project's _CI package without anything installed.

vendoring applies it during step 3. Two constraints if you ever regenerate it:

  • Every surviving script in bin/ needs a hunk. A hunk for a script that no longer exists makes vendoring sync fail outright. This is why unused console entry points are listed under drop rather than left in place.
  • Keep the shebang out of the diff context. Line 1 holds a machine-specific interpreter path, so a patch generated with the default three lines of context only applies on the machine that made it. Generate it with one line of context.

To regenerate: move the patch aside, run step 3 to get unpatched scripts, insert the prelude, then diff the before and after with -U1, writing hunks with a/ and b/ prefixes relative to the repository root.

Adding a library

  1. Add it to the vendor extra.
  2. Run steps 2–4.
  3. Check the diff. If the tree grew far more than the library itself, its dependencies came too — decide whether that is acceptable before committing.
  4. If vendoring sync leaves a module importing a bare top-level name, add a rule to [tool.vendoring.transformations].substitute. The list is empty on purpose; every rule it used to hold existed to prop up the accidentally-vendored toolchain.
  5. If the library ships a console script you never run, add it to drop — each one otherwise needs its own patch hunk.

Consuming this template

Copy _CI/ and workflow into your repository. Define tasks under _CI/tasks/. Run them with ./workflow <task>. Nothing needs installing.