feat(tests): CI Workflow to run basic checks against the templates#32
Merged
Conversation
Adds the first milestone of the template testing pipeline: a thin test image over ghcr.io/dfinity/icp-dev-env-all and a bash harness that renders a template from the local checkout via `icp new`, builds, deploys to a local network, and verifies a backend call — all inside the container, against the icp CLI pinned in the image. Covers a single hardcoded permutation (rust template, defaults); the full matrix follows in M2. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replaces the hardcoded M1 permutation with a declarative matrix (tests/matrix.txt) covering rust, motoko, and all four hello-world backend/frontend combinations. The driver gains filtering by permutation-id prefix, a --fail-fast flag, and a summary table. Each permutation now verifies every canister in the rendered project: asset canisters get an HTTP 200 check through the name-based gateway domain (curl --resolve <name>.local.localhost:8000), everything else gets the greet call. hello-world renders additionally assert the variant directory renames. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds a GitHub Actions workflow that runs the full template test suite as a single sequential job on pull requests and pushes to main. The suite stays sequential in one container: dependency caches warm up on the first permutation, making the full matrix faster than parallel cold containers would be. If the matrix grows, CI can split into a job matrix via the harness's permutation filter without changes. Local `make test` runs now mount named volumes for the cargo, npm and mops caches so repeated runs skip re-downloading dependencies. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The only thing the custom image added was yq, which the base image will eventually ship. The harness now installs yq at startup if it's missing (a no-op once the image includes it) and exports the env vars the image doesn't set (USER for cargo-generate, DO_NOT_TRACK). The icp CLI pin moves to the Makefile's IMAGE variable; an alternate CLI can be tested with `make test IMAGE=...`. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replaces recipe-type discovery with explicit per-template test
configuration. matrix.txt becomes matrix.yaml, where each template
declares its permutations and, per canister, what to verify after
deploy: a gateway curl (curl: true) and/or a canister call with
method, args, and an expected reply substring.
A configured canister name of {{project-name}} is substituted with
the rendered project's name, for templates whose icp.yaml names the
canister after the project.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
marc0olo
approved these changes
Jun 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Can be simplified a little if we add yq to the base dev image.