Infrastructure: Add Kubernetes-native LocalCI execution - #13375
Open
krusche wants to merge 20 commits into
Open
Conversation
github-actions
Bot
added
tests
server
krusche
temporarily deployed
to
playwright-e2e-tests
August 1, 2026 18:11 — with 
GitHub Actions
Inactive
GitHub Actions
Inactive
End-to-End Test Results
❌ Failed Tests
Test Strategy: Running all tests (configuration or infrastructure changes detected) Overall: ❌ E2E: real (non-flaky) test failure 🔗 Workflow Run · 📊 Test Report |
Member
Author
|
@Claudia-Anthropica review |
Contributor
|
@krusche acknowledged, queuing this up for immediate review! |
Contributor
There was a problem hiding this comment.
Pull request overview
Adds an MVP Kubernetes-native execution path for LocalCI build agents while preserving existing queueing, retry/cancel, result processing, and statistics behavior. The PR introduces a runner abstraction (Docker default, Kubernetes optional), adds Kubernetes Job-based execution with a trusted helper sidecar, and ships a Helm-based multi-node deployment + acceptance test workflow for Docker Desktop’s managed Kubernetes.
Changes:
- Introduce
BuildJobRunnerabstraction and implement Docker + Kubernetes execution paths, including improved cancellation/requeue handoff safety. - Expose build runner metadata (type/version) in build-agent APIs/UI and add a Playwright multi-node Kubernetes LocalCI acceptance test.
- Add a Helm chart + helper script for multi-node Kubernetes deployments, plus Fabric8 Kubernetes client dependencies.
Reviewed changes
Copilot reviewed 78 out of 80 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| src/test/playwright/support/requests/ExerciseAPIRequests.ts | Allows creating programming exercises with a serialized LocalCI build plan config in Playwright tests. |
| src/test/playwright/e2e/localci/KubernetesLocalCI.spec.ts | Adds multi-node Playwright acceptance test covering queueing, results, logs, and cancellation for Kubernetes Jobs. |
| src/test/java/de/tum/cit/aet/artemis/core/security/jwt/JWTCookieServiceTest.java | Adds tests for configurable secure-cookie behavior across profiles. |
| src/test/java/de/tum/cit/aet/artemis/buildagent/service/SharedQueueProcessingServiceTest.java | Adds unit coverage for result publication gating and internal requeue/completion races. |
| src/test/java/de/tum/cit/aet/artemis/buildagent/service/runner/KubernetesBuildJobRunnerTest.java | Adds tests for Kubernetes runner execution lifecycle, log buffering, cancellation scoping, and safety helpers. |
| src/test/java/de/tum/cit/aet/artemis/buildagent/service/runner/KubernetesBuildJobFactoryTest.java | Tests Kubernetes Job spec generation, isolation flags, proxy/env mapping, and rejected run flags. |
| src/test/java/de/tum/cit/aet/artemis/buildagent/service/runner/KubernetesBuildArchiveServiceTest.java | Tests input-archive construction, path validation, symlink handling, and cleanup-on-failure. |
| src/test/java/de/tum/cit/aet/artemis/buildagent/service/runner/DockerBuildJobRunnerTest.java | Tests Docker runner error classification and cancellation behavior via the runner abstraction. |
| src/test/java/de/tum/cit/aet/artemis/buildagent/service/BuildJobManagementServiceTest.java | Adds tests ensuring cancellation completion waits for callable exit and preserves ownership until release. |
| src/test/java/de/tum/cit/aet/artemis/buildagent/service/BuildAgentConfigurationTest.java | Verifies dedicated executor for build-result waits and queueing behavior under concurrency. |
| src/main/webapp/i18n/en/buildAgents.json | Adds UI label for build runner metadata. |
| src/main/webapp/i18n/de/buildAgents.json | Adds UI label for build runner metadata (German locale). |
| src/main/webapp/app/localci/shared/entities/build-agent-information.model.ts | Extends build-agent details model with runner type/version fields. |
| src/main/webapp/app/localci/build-agent-details/build-agent-details.component.html | Displays build runner + version instead of Docker-only version field. |
| src/main/resources/config/application-k8s.yml | Adds k8s profile defaults selecting Kubernetes runner and runner-specific settings. |
| src/main/resources/config/application-buildagent.yml | Documents/sets default runner selection (Docker) and supported runner values. |
| src/main/java/de/tum/cit/aet/artemis/localci/exception/DockerImagePullException.java | Introduces typed exception for Docker image pull failures (non-agent-health failures). |
| src/main/java/de/tum/cit/aet/artemis/core/service/TempFileUtilService.java | Makes TempFileUtilService available for both core and buildagent profiles. |
| src/main/java/de/tum/cit/aet/artemis/core/security/jwt/JWTCookieService.java | Adds configurable cookie security (cookieSecure) with profile-based default fallback. |
| src/main/java/de/tum/cit/aet/artemis/core/config/ArtemisProperties.java | Adds nullable cookieSecure property under JWT security settings. |
| src/main/java/de/tum/cit/aet/artemis/buildagent/service/SharedQueueProcessingService.java | Refactors for runner abstraction and tightens attempt ownership/requeue/cancellation race handling. |
| src/main/java/de/tum/cit/aet/artemis/buildagent/service/runner/PreparedBuildJob.java | Adds runner-neutral representation of prepared/cloned repositories. |
| src/main/java/de/tum/cit/aet/artemis/buildagent/service/runner/KubernetesBuildJobFactory.java | Builds Kubernetes Job specs for LocalCI attempts (workload isolation, resources, env/proxy mapping). |
| src/main/java/de/tum/cit/aet/artemis/buildagent/service/runner/KubernetesBuildArchiveService.java | Creates runner-neutral input tar archives for Kubernetes helper ingestion. |
| src/main/java/de/tum/cit/aet/artemis/buildagent/service/runner/DockerBuildJobRunner.java | Encapsulates existing Docker execution flow behind the runner abstraction. |
| src/main/java/de/tum/cit/aet/artemis/buildagent/service/runner/BuildRunnerType.java | Defines supported runner types and display naming. |
| src/main/java/de/tum/cit/aet/artemis/buildagent/service/runner/BuildRunnerStatus.java | Defines runner availability/version status model. |
| src/main/java/de/tum/cit/aet/artemis/buildagent/service/runner/BuildJobRunnerResult.java | Defines AutoCloseable runner result (archive stream + cleanup hook). |
| src/main/java/de/tum/cit/aet/artemis/buildagent/service/runner/BuildJobRunner.java | Defines runner abstraction API (execute/cancel/isActive/cleanupOrphans). |
| src/main/java/de/tum/cit/aet/artemis/buildagent/service/BuildJobManagementService.java | Refactors execution/cancellation to use runner abstraction and a dedicated result-wait executor. |
| src/main/java/de/tum/cit/aet/artemis/buildagent/service/BuildJobContainerService.java | Makes Docker container service conditional on Docker runner selection. |
| src/main/java/de/tum/cit/aet/artemis/buildagent/service/BuildAgentInformationService.java | Publishes runner type/version and generalizes runner availability polling. |
| src/main/java/de/tum/cit/aet/artemis/buildagent/service/BuildAgentDockerService.java | Makes Docker service conditional on Docker runner selection. |
| src/main/java/de/tum/cit/aet/artemis/buildagent/dto/BuildAgentDetailsDTO.java | Extends DTO with runner metadata and keeps a compatibility constructor. |
| src/main/java/de/tum/cit/aet/artemis/buildagent/config/KubernetesBuildRunnerProperties.java | Adds configuration properties for Kubernetes runner. |
| src/main/java/de/tum/cit/aet/artemis/buildagent/config/KubernetesBuildRunnerConfiguration.java | Creates Fabric8 Kubernetes client only when Kubernetes runner is selected. |
| src/main/java/de/tum/cit/aet/artemis/buildagent/BuildAgentConfiguration.java | Adds dedicated build-result executor and runner selection for Docker service initialization. |
| run-localci-kubernetes.sh | Adds end-to-end Docker Desktop Kubernetes workflow (build/deploy/test/status/logs/down). |
| helm/artemis/values.yaml | Introduces MVP Helm chart values for multi-node core + build agents + dependencies. |
| helm/artemis/values-docker-desktop.yaml | Adds Docker Desktop-specific values for the managed 3-node kind cluster. |
| helm/artemis/templates/serviceaccount.yaml | Adds base release ServiceAccount template. |
| helm/artemis/templates/secret-ssh-hostkey.yaml | Generates/persists shared LocalVC SSH host key secret for multi-core SSH LB. |
| helm/artemis/templates/secret-artemis.yaml | Adds secret wiring for DB/JWT/admin/registry/broker/build-agent git password. |
| helm/artemis/templates/registry/service.yaml | Adds ClusterIP service for JHipster Registry. |
| helm/artemis/templates/registry/deployment.yaml | Adds deployment for JHipster Registry (Eureka). |
| helm/artemis/templates/postgres/statefulset.yaml | Adds bundled PostgreSQL StatefulSet. |
| helm/artemis/templates/postgres/service.yaml | Adds bundled PostgreSQL service. |
| helm/artemis/templates/NOTES.txt | Adds chart post-install notes and reminders. |
| helm/artemis/templates/gateway/tcproute.yaml | Adds TCPRoute option for git-over-SSH via Gateway API experimental channel. |
| helm/artemis/templates/gateway/service-ssh-lb.yaml | Adds fallback Service exposure for SSH when TCPRoute is unavailable. |
| helm/artemis/templates/gateway/httproute.yaml | Adds HTTPRoute for Artemis HTTP traffic via Gateway API. |
| helm/artemis/templates/gateway/gateway.yaml | Adds Gateway resource with HTTPS listener (and optional TCP listener). |
| helm/artemis/templates/configmap-artemis.yaml | Adds core configuration wiring (URLs, storage paths, Eureka, broker, cookie secure). |
| helm/artemis/templates/build-agents/serviceaccounts-rbac.yaml | Adds controller/workload service accounts and namespaced RBAC for Jobs/Pods/log/exec. |
| helm/artemis/templates/build-agents/namespace.yaml | Optionally creates build namespace. |
| helm/artemis/templates/build-agents/deployment.yaml | Deploys build-agent controllers configured for Kubernetes runner and helper settings. |
| helm/artemis/templates/broker/service.yaml | Adds ActiveMQ STOMP service. |
| helm/artemis/templates/broker/deployment.yaml | Adds ActiveMQ deployment used for STOMP relay. |
| helm/artemis/templates/artemis/statefulset-member.yaml | Adds member core StatefulSet (non-scheduling) with leader readiness wait. |
| helm/artemis/templates/artemis/statefulset-leader.yaml | Adds leader core StatefulSet (scheduling enabled). |
| helm/artemis/templates/artemis/service-ssh.yaml | Adds load-balanced SSH service across core pods. |
| helm/artemis/templates/artemis/service-http.yaml | Adds load-balanced HTTP service across core pods. |
| helm/artemis/templates/artemis/service-headless.yaml | Adds headless service for stable pod DNS + Hazelcast port exposure. |
| helm/artemis/templates/artemis/pvc-data.yaml | Adds shared PVC for /opt/artemis/data (skipped when existingClaim is used). |
| helm/artemis/templates/_helpers.tpl | Adds chart helpers and shared pod spec (init containers, probes, mounts). |
| helm/artemis/README.md | Documents chart purpose, execution flow, required values, and Docker Desktop workflow. |
| helm/artemis/cluster-setup/README.md | Documents optional cluster-level prerequisite manifests and how to apply them. |
| helm/artemis/cluster-setup/metallb-dualstack/gatewayclass.yaml | Provides dual-stack GatewayClass referencing EnvoyProxy parameters. |
| helm/artemis/cluster-setup/metallb-dualstack/envoyproxy.yaml | Provides EnvoyProxy customization for MetalLB pool + dual-stack LB service patch. |
| helm/artemis/cluster-setup/gatewayclass.yaml | Provides plain Envoy GatewayClass manifest. |
| helm/artemis/cluster-setup/clusterissuer-letsencrypt.yaml | Provides optional cert-manager ClusterIssuer manifest for Gateway API HTTP-01. |
| helm/artemis/CLUSTER-SETUP.md | Documents end-to-end cluster setup and validation steps. |
| helm/artemis/Chart.yaml | Adds chart metadata for the MVP Helm chart. |
| helm/artemis/.helmignore | Adds Helm packaging ignore patterns. |
| gradle.properties | Adds Fabric8 Kubernetes client version property. |
| docker/localci-kubernetes-helper/Dockerfile | Adds trusted helper sidecar image for Kubernetes runner. |
| build.gradle | Adds Fabric8 Kubernetes client dependencies (OkHttp transport). |
| .gitignore | Ignores local Kubernetes acceptance-state directory. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
krusche
temporarily deployed
to
playwright-e2e-tests
August 1, 2026 19:20 — with
GitHub Actions
Inactive
GitHub Actions
Inactive
Claudia-Anthropica
requested changes
Aug 1, 2026
Claudia-Anthropica
left a comment
Claudia-Anthropica
left a comment
Contributor
There was a problem hiding this comment.
@krusche The Kubernetes execution path has blocking lifecycle and failure-classification issues, and the multi-node chart currently exposes a credential and breaks rolling-version serialization. The exact head and snapshot match; no prior review threads or completed CI failures duplicate or supersede these findings.
github-project-automation
Bot
moved this from Work In Progress
to Ready For Review
in Artemis Development
krusche
had a problem deploying
to
playwright-e2e-tests
August 11, 2026 14:54 — with
GitHub Actions
Failure
GitHub Actions
Failure
11 tasks
The authorization check in run-localci-kubernetes.sh could not report what was wrong. kubectl auth can-i answers "no" with exit status 1, so under set -Eeuo pipefail the assignment ended the script and the caller saw a bare kubectl exit instead of the message naming the missing permission. Two of the three checks were missing the `|| true` the third already had. DockerBuildJobRunner took its container prefix through field injection. Make it final and pass it into the constructor, so the test can supply it directly instead of writing the field by reflection. The cancellation test parked a build callable on a latch that only the test itself released. An assertion failing before that point left the executor thread blocked, because the callable deliberately swallows the first interrupt and shutdownNow was therefore not a reliable release. Release the latch in tearDown as well. Compare the ZonedDateTime values in the Kubernetes runner test through toInstant(), as the project guidelines require.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Claudia-Anthropica
previously approved these changes
Aug 11, 2026
Claudia-Anthropica
left a comment
Claudia-Anthropica
left a comment
Contributor
There was a problem hiding this comment.
@krusche The exact head resolves the remaining cleanup blocker by releasing every registered callable latch before shutting down the executors. The other new review fixes are consistent with the surrounding implementation, and the authoritative diff shows no unresolved medium-or-higher defect. CI acquisition is complete with no required or failing checks, although optional checks remain in progress.
krusche
temporarily deployed
to
playwright-e2e-tests
August 11, 2026 18:03 — with
GitHub Actions
Inactive
GitHub Actions
Inactive
Two conflicts and one break that merged cleanly but did not compile. BuildJobManagementService: develop added a BuildAgentDockerService parameter where this branch had replaced BuildJobContainerService with BuildJobRunner. Kept the runner and, rather than carry a Docker dependency into a runner-agnostic class, put the question on the abstraction: BuildJobRunner.isFetchingImage, answered by the Docker runner from BuildAgentDockerService and defaulting to false elsewhere, since a Kubernetes job has no local pull to wait for. SharedQueueProcessingService called buildAgentDockerService directly on a field this branch no longer has, so the merge compiled nowhere. It now asks the runner the same question. BuildJobManagementServiceTest existed on both sides with the same name and different contents: develop's image-pull timeout tests and this branch's cancellation-ownership tests. Both still apply, so they are one class now, with develop's tests driving the runner instead of the Docker service.
krusche
dismissed stale reviews from Claudia-Anthropica and coderabbitai[bot]
via
August 12, 2026 09:00
bd3fc74
Member
Author
|
Merged the latest
On the four medium findings in the latest review: all four are already implemented at this head. The review was submitted 2026-08-01T19:21Z, and c65e68f ("Address Kubernetes LocalCI review feedback", 2026-08-03) fixed them two days later:
So I believe the changes-requested state is stale rather than pointing at anything open — happy to look again if any of these reads differently to you. |
Claudia-Anthropica
requested changes
Aug 12, 2026
Claudia-Anthropica
left a comment
Claudia-Anthropica
left a comment
Contributor
There was a problem hiding this comment.
@krusche The develop-merge conflict resolution correctly routes image-fetch state through the runner abstraction, and I found no new medium-or-higher code defect. Changes remain required because the exact-head snapshot records Codacy Static Code Analysis as a completed ACTION_REQUIRED failure, with no repository evidence establishing that failure as unrelated.
krusche
temporarily deployed
to
playwright-e2e-tests
August 12, 2026 09:17 — with
GitHub Actions
Inactive
GitHub Actions
Inactive
krusche
temporarily deployed
to
playwright-e2e-tests
August 12, 2026 17:10 — with
GitHub Actions
Inactive
GitHub Actions
Inactive
14 tasks
Claudia-Anthropica
requested changes
Aug 18, 2026
Claudia-Anthropica
left a comment
Claudia-Anthropica
left a comment
Contributor
There was a problem hiding this comment.
@krusche Changes are required because Kubernetes result collection resolves standard relative result paths from the wrong directory, silently producing empty result archives. The current Codacy ACTION_REQUIRED is also attributable to the helper image's missing non-root default USER; restore a non-root image default while retaining explicit runAsUser overrides for containers that require root. The PostgreSQL server-test check is red, but its capture contains no test report or failing-test diagnostic beyond exit code 1, so I could not attribute that failure.
krusche
temporarily deployed
to
playwright-e2e-tests
August 18, 2026 23:23 — with
GitHub Actions
Inactive
GitHub Actions
Inactive
…tory Review finding, and the effect was silent. KubernetesBuildJobFactory starts the builder with `cd /var/tmp/testing-dir`, so the standard result paths - test-reports/*.xml, **/target/surefire-reports/*.xml - are written relative to that directory. The result collection helper expanded them from its own working directory one level up in /var/tmp, where they matched nothing. Nothing failed as a result: nullglob makes a non-matching pattern iterate zero times, so the helper exited 0 with an empty results archive and the build was reported without any test feedback. That is the worst shape for this bug, because a missing report looks like a test suite that produced nothing. The helper now cds into the build directory before expanding, which also matches the Docker runner, whose mv runs in the build working directory. Replaced `|| true` with `continue` on a missing source. nullglob already covers a non-matching pattern, so the only thing `|| true` still hid was a genuine mv failure. Overlapping patterns stay safe: each glob is expanded when its own loop starts, by which point an earlier loop has already moved those files away. The test pins the cd, that it precedes the loops, and that a real failure is no longer swallowed.
Claudia-Anthropica
requested changes
Aug 19, 2026
Claudia-Anthropica
left a comment
Claudia-Anthropica
left a comment
Contributor
There was a problem hiding this comment.
@krusche The latest head correctly resolves relative result paths from /var/tmp/testing-dir, and all retained review threads are resolved. Changes are still required because the standalone teardown can delete the artemis and artemis-builds namespaces from whichever cluster is current; the required guard is documented inline. The captured E2E report is not tied to this head or a current failed run, and the exact-head snapshot has no failed or required checks.
krusche
temporarily deployed
to
playwright-e2e-tests
August 19, 2026 08:31 — with
GitHub Actions
Inactive
GitHub Actions
Inactive
Merge develop, keeping QueueItemListener from develop and the two image pull exceptions from this branch; LocalCIException is no longer referenced and its import is dropped. tear_down deleted the artemis and artemis-builds namespaces before the only docker-desktop check, and the `down` command reaches it without going through check_cluster_topology at all. Against a non-local context that cleanup would have deleted a real cluster's namespaces. Add is_local_kubectl_context and require_local_kubectl_context; `down` now aborts with the offending context name, and tear_down itself skips every cluster mutation unless the context is docker-desktop, which also covers its use as an EXIT trap. A missing or unreadable context counts as not local.
krusche
temporarily deployed
to
playwright-e2e-tests
August 24, 2026 00:02 — with
GitHub Actions
Inactive
GitHub Actions
Inactive
krusche
temporarily deployed
to
playwright-e2e-tests
August 24, 2026 21:01 — with
GitHub Actions
Inactive
GitHub Actions
Inactive
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This change adds an MVP Kubernetes-native execution path for LocalCI while keeping Docker execution as the default. A multi-node Artemis installation can run two core nodes and multiple build-agent controllers, with each submitted build executed directly as a Kubernetes Job. The existing LocalCI queue, retry, cancellation, result processing, statistics, and time-estimation behavior remains in use.
Checklist
General
Server
Client
Changes affecting Programming Exercises
Motivation and Context
LocalCI build agents currently execute exercise builds through a local Docker daemon. This prevents a multi-node Kubernetes installation from scheduling isolated build workloads natively and would require exposing a Docker socket to the build-agent controllers.
The new execution path lets the existing distributed LocalCI queue assign work to multiple controllers while Kubernetes schedules each build as a namespaced Job. It is intentionally an MVP: it preserves all LocalCI features required for normal programming-exercise builds, but leaves production hardening such as runtime sandboxes, autoscaling, external secret management, and highly available data services to deployment-specific follow-up work.
Description
BuildJobRunnerabstraction and keeps Docker execution as the default.emptyDirworkspace; workload pods receive neither a Docker socket nor Kubernetes API credentials.Steps for Testing
Prerequisites:
kindprovisioner, exactly three nodes, and the containerd image storedocker,kubectl,helm,jq, Java 25, Node.js, andpnpmkubectl config use-context docker-desktop../run-localci-kubernetes.sh allfrom the repository root.Individual lifecycle commands and diagnostics are documented in
helm/artemis/CLUSTER-SETUP.md.Local Verification
./run-localci-kubernetes.sh all: passed on one Docker Desktop control-plane and two worker nodes.bash -n, strict Helm lint/template rendering, andgit diff --check: passed.Test Coverage
Note: Some tests in the Test job did not pass (
failure). Coverage below may be partial.Client
Server
Last updated: 2026-08-24 21:01:47 UTC