Update Workflows to Version 1.0.1 (#53)

Co-authored-by: connoraird <61978554+connoraird@users.noreply.github.com>
Co-authored-by: connoraird <c.aird@ucl.ac.uk>

Author: carpentries-bot

.github/workflows/README.md

@@ -1,12 +1,19 @@
-# Carpentries Workflows
+# Workflow Documentation
 
-This directory contains workflows to be used for Lessons using the {sandpaper}
-lesson infrastructure. Two of these workflows require R (`sandpaper-main.yaml`
-and `pr-receive.yaml`) and the rest are bots to handle pull request management.
+## Managing Workflow Updates
 
-These workflows will likely change as {sandpaper} evolves, so it is important to
-keep them up-to-date. To do this in your lesson you can do the following in your
-R console:
+By using prebuilt Docker containers that are managed by the Carpentries core Workbench maintainers, these workflows are designed
+to be rarely updated.
+
+However, is important to be able to keep them up-to-date when appropriate.
+You can do this locally using your own R and Workbench installation, or via the "04 Maintain: Update Workflow Files"
+(`update-workflows.yaml`) GitHub Action.
+
+### Updating locally
+
+In a terminal/git bash, navigate to the lesson folder where you want to update the workflows.
+
+Then, start an R session and:
 
 ```r
 # Install/Update sandpaper
@@ -16,21 +23,31 @@ install.packages("sandpaper")
 
 # update the workflows in your lesson
 library("sandpaper")
-update_github_workflows()
+sandpaper::update_github_workflows()
+quit()
 ```
 
-Inside this folder, you will find a file called `sandpaper-version.txt`, which
-will contain a version number for sandpaper. This will be used in the future to
-alert you if a workflow update is needed.
+And then in a bash prompt/git bash terminal:
 
-What follows are the descriptions of the workflow files:
+```bash
+git add .github/workflows
+git commit -m "Manual update to docker workflows"
+git push origin main
+```
 
-## Deployment
+> [!NOTE]
+> For non-renv lessons, this is all the setup you need!
+>
+> For renv-enabled lessons:
+>
+> - Cancel any "01 Maintain: Build and Deploy Site" workflow currently running
+> - Run the "02 Maintain: Check for Updated Packages" workflow and merge any PR opened to update the renv lockfile
+> - This should automatically run the "03 Maintain: Apply Package Cache" workflow to install packages and build the cache
+> - A successful cache build should then trigger the "01 Maintain: Build and Deploy Site" workflow
 
-### 01 Build and Deploy (sandpaper-main.yaml)
+### Updating using GitHub
 
-This is the main driver that will only act on the main branch of the repository.
-This workflow does the following:
+#### Official lessons
 
  1. checks out the lesson
  2. provisions the following resources
@@ -40,7 +57,10 @@ This workflow does the following:
     - lesson dependencies if needed (stored in a cache)
  3. builds the lesson via `sandpaper:::ci_deploy()`
 
-#### Caching
+To update the workflows, either:
+
+- wait for the scheduled run of the "04 Maintain: Update Workflow Files" at approximately midnight every Tuesday
+- go to the Actions tab on GitHub, click "04 Maintain: Update Workflow Files" on the left, then "Run Workflow" on the right
 
 This workflow has two caches; one cache is for the lesson infrastructure and
 the other is for the lesson dependencies if the lesson contains rendered
@@ -52,19 +72,20 @@ and invalidate the failing cache](https://github.blog/changelog/2022-10-20-manag
 or by setting the `CACHE_VERSION` secret to the current date (which will
 invalidate all of the caches).
 
-## Updates
+#### Your own lessons
+
+This presumes you:
 
-### Setup Information
+- already have a lesson repository available on GitHub
+- have enabled workflows in the lesson repo
+- have set up a SANDPAPER_WORKFLOW personal access token (PAT) in the lesson repo
 
 These workflows run on a schedule and at the maintainer's request. Because they
 create pull requests that update workflows/require the downstream actions to run,
 they need a special repository/organization secret token called
 `SANDPAPER_WORKFLOW` and it must have the `public_repo` and `workflow` scope.
 
-This can be an individual user token, OR it can be a trusted bot account. If you
-have a repository in one of the official Carpentries accounts, then you do not
-need to worry about this token being present because the Carpentries Core Team
-will take care of supplying this token.
+Once set up, run the "04 Maintain: Update Workflow Files" (`update-workflows.yaml`) action.
 
 If you want to use your personal account: you can go to
 <https://github.com/settings/tokens/new?scopes=public_repo,workflow&description=Sandpaper%20Token>
@@ -75,7 +96,7 @@ create or edit the `SANDPAPER_WORKFLOW` secret, pasting in the generated token.
 If you do not specify your token correctly, the runs will not fail and they will
 give you instructions to provide the token for your repository.
 
-### 02 Maintain: Update Workflow Files (update-workflow.yaml)
+## Package Caches for RMarkdown Lessons
 
 The {sandpaper} repository was designed to do as much as possible to separate
 the tools from the content. For local builds, this is true, but
@@ -86,17 +107,18 @@ This workflow ensures that the workflow files are up-to-date. The way it work is
 to download the update-workflows.sh script from GitHub and run it. The script
 will do the following:
 
-1. check the recorded version of sandpaper against the current version on github
-2. update the files if there is a difference in versions
+### Caching
 
-After the files are updated, if there are any changes, they are pushed to a
-branch called `update/workflows` and a pull request is created. Maintainers are
-encouraged to review the changes and accept the pull request if the outputs
-are okay.
+The two cache management workflows are separated to ensure that once you have a successful build with a working renv cache, this
+cache is stored and will be reused by the Workbench Docker container.
+This means that lesson builds will be faster once an renv cache is created and reused by the Docker container.
 
-This update is run weekly or on demand.
+Another major bonus of this setup is that you can keep using this cache indefinitely to build your lesson.
+This is important if you need very specific versions of R packages ("pinning").
 
-### 03 Maintain: Update Package Cache (update-cache.yaml)
+If and when you want to perform an update to the cache, you can re-run the "02 Maintain: Check for Updated Packages" and verify
+that your lesson still builds with the new packages.
+If all looks good, re-run the "03 Maintain: Apply Package Cache" workflow, and this will write a new renv cache file to GitHub.
 
 For lessons that have generated content, we use {renv} to ensure that the output
 is stable. This is controlled by a single lockfile which documents the packages
@@ -125,8 +147,8 @@ diagram and the below sections:
 
 ### Pre Flight Pull Request Validation (pr-preflight.yaml)
 
-This workflow runs every time a pull request is created and its purpose is to
-validate that the pull request is okay to run. This means the following things:
+This workflow runs every time a pull request is created and its purpose is to validate that the pull request is okay to run.
+This means the following things:
 
 1. The pull request does not contain modified workflow files
 2. If the pull request contains modified workflow files, it does not contain
@@ -140,25 +162,24 @@ Once the checks are finished, a comment is issued to the pull request, which
 will allow maintainers to determine if it is safe to run the
 "Receive Pull Request" workflow from new contributors.
 
-### Receive Pull Request (pr-receive.yaml)
+### Receive Pull Request (docker_pr_receive.yaml)
 
 **Note of caution:** This workflow runs arbitrary code by anyone who creates a
 pull request. GitHub has safeguarded the token used in this workflow to have no
 privileges in the repository, but we have taken precautions to protect against
 spoofing.
 
-This workflow is triggered with every push to a pull request. If this workflow
-is already running and a new push is sent to the pull request, the workflow
-running from the previous push will be cancelled and a new workflow run will be
-started.
+This workflow is triggered with every push to a pull request.
+If this workflow is already running and a new push is sent to the pull request, the workflow running from the previous push will
+be cancelled and a new workflow run will be started.
 
 The first step of this workflow is to check if it is valid (e.g. that no
 workflow files have been modified). If there are workflow files that have been
 modified, a comment is made that indicates that the workflow is not run. If
 both a workflow file and lesson content is modified, an error will occur.
 
-The second step (if valid) is to build the generated content from the pull
-request. This builds the content and uploads three artifacts:
+The second step (if valid) is to build the generated content from the pull request.
+This builds the content and uploads three artifacts:
 
 1. The pull request number (pr)
 2. A summary of changes after the rendering process (diff)
@@ -171,26 +192,23 @@ The artifacts produced are used by the next workflow.
 
 ### Comment on Pull Request (pr-comment.yaml)
 
-This workflow is triggered if the `pr-receive.yaml` workflow is successful.
+This workflow is triggered if the `docker_pr_receive.yaml` workflow is successful.
 The steps in this workflow are:
 
-1. Test if the workflow is valid and comment the validity of the workflow to the
-   pull request.
-2. If it is valid: create an orphan branch with two commits: the current state
-   of the repository and the proposed changes.
+1. Test if the workflow is valid and comment the validity of the workflow to the pull request.
+2. If it is valid: create an orphan branch with two commits: the current state of the repository and the proposed changes.
 3. If it is valid: update the pull request comment with the summary of changes
 
-Importantly: if the pull request is invalid, the branch is not created so any
-malicious code is not published.
+Importantly: if the pull request is invalid, the branch is not created so any malicious code is not published.
 
 From here, the maintainer can request changes from the author and eventually
 either merge or reject the PR. When this happens, if the PR was valid, the
 preview branch needs to be deleted.
 
 ### Send Close PR Signal (pr-close-signal.yaml)
 
-Triggered any time a pull request is closed. This emits an artifact that is the
-pull request number for the next action
+Triggered any time a pull request is closed.
+This emits an artifact that is the pull request number for the next action.
 
 ### Remove Pull Request Branch (pr-post-remove-branch.yaml)