Add a skip option - #1
Open
orien wants to merge 2 commits into
Open
Conversation
Some steps never read the codebase: gates that only echo a message, or steps whose whole job is done by another plugin. They still pay for a clone, which on an agent with no existing checkout is the slowest part of the step. Clearing BUILDKITE_REPO is the agent's own mechanism for skipping the checkout. With no repository to clone it allocates a fresh temporary directory for the job and removes it afterwards, leaving any existing checkout on the agent alone so the next job there still gets an incremental fetch. Prefer this to the agent's BUILDKITE_SKIP_CHECKOUT, which skips the git work but keeps using the normal checkout directory. That directory is not branch-scoped, so a tree left by an earlier job still supplies .buildkite/hooks to a step that asked for no codebase at all.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
Some Buildkite steps never read the codebase. Gates that only
echoa message are the clearest case; so are steps whose whole job is done by another plugin, such as one that posts a deploy webhook. They still clone the repository first, and on an agent with no existing checkout that clone is the slowest part of the step. Inenvato/sso-serverthe two SOX gates have a median duration of 2.2s but a p90 of around 47s and a maximum of 78s, and the difference is whether the agent had to do a full clone.The agent offers
BUILDKITE_SKIP_CHECKOUTfor this, but it only skips the git work: the job still runs in the agent's normalbuilds/<agent>/<org>/<pipeline>directory. That path is not branch-scoped, so a tree left behind by an earlier job remains, and the agent resolves repository-local hooks from it. A step that asked for no codebase can end up running.buildkite/hooks/pre-commandfrom another branch, or from a partial clone, where a missing file makes the hook fail underset -eand takes the step with it.Change
Adds a
skipoption that clearsBUILDKITE_REPO. With no repository to clone the agent allocates a fresh temporary directory for the job, skips the checkout, and removes the directory afterwards. Two consequences follow, and both are the point:.buildkite/hooks, so the staleness problem above cannot arise.skiptakes precedence overpreset,clone,fetchandclean, and logs a line when it ignores them. It acceptstrue,onand1.The hook's existing logic is unchanged, but moving it into the
elsebranch reindents it —git diff -wshows the real change.Considerations
This is the same mechanism
docker-compose-buildkite-pluginuses for its ownskip-checkoutoption, which is well proven in Envato pipelines. The agent supports it deliberately:CheckoutPhasecomments that "environment and pre-checkout hooks can changeBUILDKITE_REPO",applyEnvironmentChangesfeeds hook env changes throughReadFromEnvironmentback into the executor config, andBUILDKITE_REPOis markedmutableFromWithinJobin the agent's protected-env table precisely so hooks and plugins may set it.A step's own
env:cannot do this — that same table blocks job-level env and secrets from writingBUILDKITE_REPO— which is why it has to live in a plugin.The alternative considered was pairing
BUILDKITE_SKIP_CHECKOUTwithBUILDKITE_CLEAN_CHECKOUT, which also yields an empty working directory. It was rejected because the clean discards the agent's cached checkout, so the next job to land on that agent pays the full clone the change was trying to avoid. This option moves the cost rather than relocating it.The 23 bats tests pass, along with shellcheck and the plugin linter.
v1.2.0will need tagging after merge;envato/sso-serveris the first intended consumer.