Bug Report: <BUG>

[morrisongraeme]

Apologies, I would submit a PR to fix this if I could but I'm unsure how to resolve it. Thanks in advance for your help!

Describe the bug
Cannot build image or run apt update on built image due to expired apt key

To Reproduce

Option 1 (failure on build): clone the repo as per the README instructions and run the build script ./build-images.sh. Fails with the error:

9.779 W: GPG error: https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.29/deb  InRelease: The following signatures were invalid: EXPKEYSIG 234654DA9A296436 isv:kubernetes OBS Project <isv:kubernetes@build.opensuse.org>
9.779 E: The repository 'https://pkgs.k8s.io/core:/stable:/v1.29/deb  InRelease' is not signed.
------
Dockerfile:10
--------------------
   9 |     # Install GCP related tools
  10 | >>> RUN echo "deb [signed-by=/etc/apt/trusted.gpg.d/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list && \
  11 | >>> 	curl -fsSLo - https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/cloud.google.gpg && \
  12 | >>> 	sudo apt-get update && sudo apt-get install --yes --no-install-recommends \
  13 | >>> 		google-cloud-cli \
  14 | >>> 		google-cloud-cli-app-engine-go \
  15 | >>> 		google-cloud-cli-app-engine-python \
  16 | >>> 		google-cloud-cli-gke-gcloud-auth-plugin \
  17 | >>> 		google-cloud-cli-kubectl-oidc \
  18 | >>> 	&& \
  19 | >>> 	sudo rm -rf /var/lib/apt/lists/* && \
  20 | >>> 	gcloud --version
  21 |     
--------------------
ERROR: failed to solve: process "/bin/bash -exo pipefail -c echo \"deb [signed-by=/etc/apt/trusted.gpg.d/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main\" | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list && \tcurl -fsSLo - https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/cloud.google.gpg && \tsudo apt-get update && sudo apt-get install --yes --no-install-recommends \t\tgoogle-cloud-cli \t\tgoogle-cloud-cli-app-engine-go \t\tgoogle-cloud-cli-app-engine-python \t\tgoogle-cloud-cli-gke-gcloud-auth-plugin \t\tgoogle-cloud-cli-kubectl-oidc \t&& \tsudo rm -rf /var/lib/apt/lists/* && \tgcloud --version" did not complete successfully: exit code: 100

Option 2 (failure on use): run a CircleCI pipeline with cimg/gcp:2024.08 as the docker image and sudo apt update as the first command. Fails with the error:

W: https://download.docker.com/linux/ubuntu/dists/jammy/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.
W: GPG error: https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.29/deb  InRelease: The following signatures were invalid: EXPKEYSIG 234654DA9A296436 isv:kubernetes OBS Project <isv:kubernetes@build.opensuse.org>
E: The repository 'https://pkgs.k8s.io/core:/stable:/v1.29/deb  InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

Exited with code exit status 100

Expected behavior
Option 1: build should succeed
Option 2: pipeline should pass

Workarounds
I haven't found one myself, but I expect someone more competent than me in this area can probably find one.

Screenshots and Build Links
N/A

Additional context
I believe this is related to: https://www.reddit.com/r/kubernetes/comments/1giyr87/are_the_kubernetes_apt_repositories_down_right_now/

[prakash-fb]

Since we encountered the same error, I am not opening a new bug ticket. Upon running sudo apt update, we received the following errors and warnings. I hope someone from the CircleCI team can address this promptly.

Err:2 https://repo.saltproject.io/py3/ubuntu/20.04/amd64/latest focal InRelease
  Could not resolve 'repo.saltproject.io'
Reading package lists... Done     

The error occurred because repo.saltproject.io was migrated to packages.broadcom.com. To solve this we have to update the repo. More info here: https://saltproject.io/blog/salt-project-package-repo-migration-and-guidance/

curl -fsSL https://packages.broadcom.com/artifactory/api/security/keypair/SaltProjectKey/public | sudo tee /etc/apt/keyrings/salt-archive-keyring-2023.pgp
echo "deb [signed-by=/etc/apt/keyrings/salt-archive-keyring-2023.pgp arch=amd64] https://packages.broadcom.com/artifactory/saltproject-deb/ stable main" | sudo tee /etc/apt/sources.list.d/salt.list

W: GPG error: https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.29/deb  InRelease: The following signatures were invalid: EXPKEYSIG 234654DA9A296436 isv:kubernetes OBS Project <isv:kubernetes@build.opensuse.org>
E: The repository 'https://pkgs.k8s.io/core:/stable:/v1.29/deb  InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

The solution is to update the GPG key

curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.29/deb/Release.key | sudo gpg --dearmor -o /usr/share/keyrings/kubernetes-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.29/deb/ /" | sudo tee /etc/apt/sources.list.d/kubernetes.list

W: https://download.docker.com/linux/ubuntu/dists/jammy/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.

This is just a warning, and someone from circleci team may address this. We solved it by running the following command

sudo cp /etc/apt/trusted.gpg /etc/apt/trusted.gpg.d

Finally, run

sudo apt update