Integrated code lifecycle: Stop offering build agent password authentication when build agents use SSH (#13503)

Author: krusche

documentation/docs/admin/production-setup/multiple-artemis-instances.mdx

@@ -542,7 +542,8 @@ Build agents can be added to and removed from the server cluster depending on th
 assessment of programming exercises. If desired, build agents can execute multiple build jobs concurrently. In this case, you need to make sure that the server node your build agents is running on has enough resources.
 We recommend at least 2 CPUs and 2 GB of RAM for each concurrently running build job.
 
-Build agents do **not** require access to the Shared File System as the repositories used in the build jobs are cloned using HTTPS. Furthermore, as Build Agents do not handle client requests,
+Build agents do **not** require access to the Shared File System, because they clone the repositories used in the build jobs over the network: over SSH when
+`artemis.version-control.build-agent-use-ssh` is `true`, and otherwise over HTTPS. Furthermore, as Build Agents do not handle client requests,
 they should be left out from the [nginx configuration](#nginx-configuration).
 
 
@@ -588,19 +589,29 @@ You can make following adaptations to the `application-prod.yml`:
 
 3. Furthermore, you will need some configuration related to version control and continuous integration.
 
-    Build agents require access to the VC server. Therefore, you need to add credentials so the build agent can access the repositories.
-    These credentials are used to clone repositories via HTTPS. You must also add these credentials to the localvc nodes.
+    Build agents require access to the VC server. How they authenticate for it is selected by
+    `artemis.version-control.build-agent-use-ssh`, and the credentials below belong to only one of the two options:
+
+    - `true` (**recommended**): each agent generates an SSH key pair at startup and authenticates with it, so there is no
+      shared credential to configure and `build-agent-git-username`/`build-agent-git-password` may be omitted entirely.
+    - `false` (default, **deprecated**): the agents clone over HTTPS with the credentials below, which you must add to the
+      build agents and to the localvc nodes alike.
+
+    The HTTPS credentials are therefore required only for that second configuration; the SSH setup is described after the
+    example below.
 
     ```yaml
     artemis:
         version-control:
             url: <url-to-your-vc-server>
             default-branch: main # The branch that should be used as default branch for all newly created repositories. This does NOT have to be equal to the default branch of the VCS
             # Artemis admin credentials
-            build-agent-git-username: buildjob_user # Replace with more secure credentials for production. Required for https access to localvc
-            build-agent-git-password: buildjob_password # Replace with more secure credentials for production. Required for https access to localvc. You can otherwise use an ssh key
+            build-agent-git-username: buildjob_user # Replace with more secure credentials for production. Required for https access to localvc, and ignored when build-agent-use-ssh is true
+            build-agent-git-password: buildjob_password # Replace with more secure credentials for production. Required for https access to localvc, and ignored when build-agent-use-ssh is true
     ```
 
+    The order matters when you switch an existing installation over. Set `artemis.version-control.build-agent-use-ssh: true` on the build agents first — localvc nodes accept a registered agent's key regardless of their own setting, so builds keep working, and you can move agents one at a time — and only afterwards on the localvc nodes, where it stops that credential pair from granting build agents read access to every repository. Doing it in the other order breaks every agent still cloning over HTTPS. See [Build Agent Authentication](/admin/production-setup/security#build-agent-authentication).
+
 4. Configuration related to the execution of build jobs:
 
     ```yaml

documentation/docs/admin/production-setup/security.mdx

@@ -16,6 +16,8 @@ artemis:
             username: "artemis-admin"
             password: "artemis-admin"
     version-control:
+        # Deprecated, and both may be omitted entirely when build-agent-use-ssh is true.
+        # Prefer SSH keys for build agents, see "Build Agent Authentication" below.
         build-agent-git-username: "buildagent_user"
         build-agent-git-password: "buildagent_password"
 jhipster:
@@ -49,6 +51,8 @@ jhipster:
 
 Under the `prod` profile, Artemis refuses to start when one of these properties still holds a value that is published in this repository: the JWT signing key, the internal admin password (this includes a password identical to the username), and the build-agent git password. The startup error names the property and how to supply a value.
 
+The build-agent git password is exempt when `artemis.version-control.build-agent-use-ssh` is `true`, because the core nodes then stop granting that credential pair read access to every repository, which is the access this check exists to protect. The pair is not rejected everywhere: it still runs through ordinary Basic authentication afterwards, where it opens exactly what the named account may access - nothing at all for `buildjob_user`, which is not an Artemis account. The check runs on every startup, so it applies again on the first start after the property goes back to `false`.
+
 The signing key is checked through both `jhipster.security.authentication.jwt.base64-secret` and the plain `jhipster.security.authentication.jwt.secret`, because either one can be the key that is actually used. It is also rejected when it is absent altogether, and when it decodes to fewer than 64 bytes, which is the minimum HS512 accepts. The comparison happens on the decoded key as well as on the configured spelling, so re-encoding a published value - different padding, an added line break - does not get past it. Generate a key with:
 
 ```bash
@@ -65,6 +69,41 @@ ARTEMIS_VERSIONCONTROL_BUILDAGENTGITPASSWORD="<a unique password, kept in sync w
 
 Artemis also pins the restrictive rendering profile of the diagram renderer at startup, in every profile. If `PLANTUML_SECURITY_PROFILE` is set explicitly, it has to name either `SANDBOX` or `ALLOWLIST`; any other value is rejected. Artemis passes its diagram theme as inline content, so no Artemis feature depends on the other profiles.
 
+### Build Agent Authentication
+
+Build agents clone repositories from the core nodes in one of two ways, selected by `artemis.version-control.build-agent-use-ssh`:
+
+- `false` (default, **deprecated**): the agents authenticate over HTTPS with `build-agent-git-username` and `build-agent-git-password`. A core node accepts that credential pair for read access to *every* repository, ahead of the rate limit, the repository authorization checks and the VCS access log. Treat it as a system-wide credential and give it the same care as the internal admin password.
+- `true` (**recommended**): each agent generates an SSH key pair at startup and publishes its public key to the core nodes, which authenticate agents by key. Core nodes then **stop honouring the build-agent shortcut**, so that credential pair no longer opens every repository. This also requires `ssh-template-clone-url` and `ssh-private-key-folder-path` on the agents.
+
+<Callout variant={CalloutVariant.warning}>
+    HTTPS authentication for build agents is deprecated. Configure new installations with
+    `artemis.version-control.build-agent-use-ssh: true`, and move existing ones over in the order given below.
+
+    One static secret, shared by every build agent and every core node and usually written into several configuration
+    files, opens every repository in the installation, and it does so ahead of the rate limit, the authorization checks
+    and the access log, which makes a leak both unlimited and invisible. An SSH key is per agent, is generated at
+    startup rather than configured, never leaves the agent that holds it, and reaches a core node only through an agent
+    that has joined the cluster.
+
+    The default stays `false` for now because SSH mode still needs `ssh-template-clone-url` and
+    `ssh-private-key-folder-path` to be set explicitly, and because Jenkins uses the same credential pair (see the
+    rollout note below).
+</Callout>
+
+The property does different things on the two node roles. On a build agent it selects the mechanism the agent uses. On a core node it controls only the build-agent shortcut over HTTPS: `true` stops that pair from granting repository-wide read. The pair is still processed as ordinary Basic credentials afterwards, which matters only if you pointed `build-agent-git-username` at a real Artemis account — that person then authenticates as themselves and gets exactly their own access, subject to the rate limit, the authorization checks and the access log. Core nodes always accept a registered build agent's public key, whatever the property says there, because a key is per-agent and only reaches a core node through a build agent that has joined the cluster — unlike the credential pair, it is not a shared secret worth switching off.
+
+That asymmetry decides the order in which you roll the change out:
+
+1. Set `build-agent-use-ssh: true` on the build agents, together with `ssh-template-clone-url` and `ssh-private-key-folder-path`, and restart them. They now clone by key, and the core nodes accept that immediately, so builds keep working while the core nodes are still on `false`. You can move agents one at a time.
+2. Once no agent clones over HTTPS any more, set the property on the core nodes and restart them. This is the step that closes the shortcut.
+
+<Callout variant={CalloutVariant.warning}>
+    Do not do it in the other order. A core node with `build-agent-use-ssh: true` no longer honours the build-agent shortcut, and `buildjob_user` is not a real account, so every agent still configured for HTTPS fails to clone from it — which is also what happens if you set the property on the core nodes while any agent is still on `false`.
+
+    Leave it at `false` when Jenkins is your CI backend. The [Jenkins with LocalVC](/admin/jenkins-localvc) setup has Jenkins clone over HTTPS with this same credential pair, and Jenkins is not an Artemis build agent, so it has no key to fall back to.
+</Callout>
+
 ### Database Credentials
 
 The database services used by `docker/artemis-prod-postgres.yml` and `docker/artemis-prod-mysql.yml` extend the shared `docker/postgres.yml`/`docker/mysql.yml` service definitions, which default to `docker/postgres/default.env` and `docker/mysql/default.env` for local development and testing. Those default env files intentionally use no real authentication (Postgres `POSTGRES_HOST_AUTH_METHOD=trust`, MySQL `MYSQL_ALLOW_EMPTY_PASSWORD=yes` with an empty root password) so a fresh local setup works without any configuration.

src/main/java/de/tum/cit/aet/artemis/buildagent/service/BuildJobGitService.java

@@ -1,5 +1,6 @@
 package de.tum.cit.aet.artemis.buildagent.service;
 
+import static de.tum.cit.aet.artemis.core.config.Constants.BUILD_AGENT_USE_SSH_PROPERTY_NAME;
 import static de.tum.cit.aet.artemis.core.config.Constants.PROFILE_BUILDAGENT;
 
 import java.io.IOException;
@@ -42,6 +43,7 @@
 import org.springframework.context.annotation.Lazy;
 import org.springframework.context.annotation.Profile;
 import org.springframework.stereotype.Service;
+import org.springframework.util.StringUtils;
 
 import de.tum.cit.aet.artemis.localvc.service.AbstractGitService;
 import de.tum.cit.aet.artemis.localvc.service.LocalVCRepositoryUri;
@@ -53,10 +55,10 @@
  * <p>
  * This service extends {@link AbstractGitService} and provides build-agent-specific Git operations,
  * including repository cloning and commit checkout. It supports two authentication mechanisms:
- * <ol>
- * <li><b>SSH authentication</b>: Uses SSH keys for secure access (preferred when available)</li>
- * <li><b>Username/password authentication</b>: Falls back to HTTP(S) with credentials</li>
- * </ol>
+ * <ul>
+ * <li><b>SSH authentication</b>: the key pair the agent generates at startup and publishes to the core nodes</li>
+ * <li><b>Username/password authentication</b>: HTTP(S) with the configured build-agent git credentials</li>
+ * </ul>
  * <p>
  * <b>Usage in Build Jobs:</b>
  * <ul>
@@ -66,10 +68,12 @@
  * <li>{@link #deleteLocalRepository} - Cleans up cloned repositories after build completion</li>
  * </ul>
  * <p>
- * <b>Authentication Priority:</b>
- * The service uses SSH if {@code artemis.version-control.build-agent-use-ssh=true} and required
- * SSH configuration (private key path, URL template) is provided. Otherwise, it uses the configured
- * username and password from application properties.
+ * <b>Choosing a mechanism:</b>
+ * {@code artemis.version-control.build-agent-use-ssh} selects one of the two, and there is no fallback between them:
+ * with ssh, a missing private key path or url template fails startup rather than reverting to https, and without ssh,
+ * missing credentials fail startup rather than reverting to a key. On a core node the same property governs only
+ * whether the credential pair still grants build agents repository-wide read; a registered agent's public key is
+ * accepted either way.
  *
  * @see BuildJobExecutionService
  * @see AbstractGitService
@@ -81,10 +85,12 @@ public class BuildJobGitService extends AbstractGitService {
 
     private static final Logger log = LoggerFactory.getLogger(BuildJobGitService.class);
 
-    @Value("${artemis.version-control.build-agent-git-username}")
+    // Optional on purpose: an agent that authenticates with an ssh key never uses this credential pair, and then must
+    // not have to configure one. {@link #init} rejects a missing value for the https case, where it is required.
+    @Value("${artemis.version-control.build-agent-git-username:}")
     private String buildAgentGitUsername;
 
-    @Value("${artemis.version-control.build-agent-git-password}")
+    @Value("${artemis.version-control.build-agent-git-password:}")
     private String buildAgentGitPassword;
 
     @Value("${artemis.version-control.build-agent-use-ssh:false}")
@@ -105,11 +111,9 @@ public class BuildJobGitService extends AbstractGitService {
     private SshdSessionFactory sshdSessionFactory;
 
     /**
-     * initialize the BuildJobGitService, in particular which authentication mechanism should be used
-     * Artemis uses the following order for authentication:
-     * 1. ssh key (if available)
-     * 2. username + personal access token (if available)
-     * 3. username + password
+     * Initializes the BuildJobGitService, in particular which authentication mechanism is used, and fails startup when
+     * the configuration for the selected mechanism is incomplete. Failing here rather than at clone time keeps the
+     * error next to the setting that caused it.
      * EventListener cannot be used here, as the bean is lazy
      * <a href="https://docs.spring.io/spring-framework/reference/core/beans/context-introduction.html#context-functionality-events-annotation">Spring Docs</a>
      */
@@ -124,7 +128,18 @@ public void init() {
                 throw new RuntimeException("No SSH private key folder was set but should use SSH for build agent authentication.");
             }
             configureSsh();
+            return;
+        }
+        // The mirror of the checks above. Both credentials are optional so that an ssh installation does not have to
+        // configure them, which means the https case has to reject a missing value here instead: the core node stops
+        // accepting a blank credential pair, so an agent that started without one would fail every clone at build time
+        // with an authentication error, far away from the configuration that caused it.
+        if (!StringUtils.hasText(buildAgentGitUsername) || !StringUtils.hasText(buildAgentGitPassword)) {
+            throw new IllegalStateException("No build agent git username and password were set, and " + BUILD_AGENT_USE_SSH_PROPERTY_NAME
+                    + " is false. Configure both credentials, or set that property to true on the build agents and on every core node to authenticate with an ssh key instead.");
         }
+        log.info("BuildJobGitService will use the configured git username and password as authentication method to interact with remote git repositories. This mechanism is "
+                + "deprecated; set {} to true here and on every core node to authenticate with an ssh key instead.", BUILD_AGENT_USE_SSH_PROPERTY_NAME);
     }
 
     protected boolean useSsh() {