Skip to content

Commit 0654c7d

Browse files
efiacorliamfallon
andauthored
Remove hardcoded MetalLB IPs from local dev scripts and tests (#978)
* Remove hardcoded MetalLB IPs from local dev scripts and tests Replace hardcoded 172.18.255.x IP addresses with dynamic discovery from the kind Docker network. The kind network subnet varies between machines, so hardcoded IPs break when Docker assigns a different subnet (e.g. 172.19.0.0/16 instead of 172.18.0.0/16). Changes: - Add scripts/get-kind-metallb-subnet.sh helper for dynamic IP range discovery from the kind Docker network - setup-dev-env.sh generates MetalLB config at runtime instead of applying a static YAML file - install-dev-gitea-setup.sh no longer pins a specific IP; detects and fixes stale annotations on re-runs - remove-porch-server/controller scripts discover function-runner IP from the running service or derive from MetalLB range - E2E tests (CRD + legacy) discover gitea LB IP from the cluster via k8sClient, with env var and hardcoded fallbacks - Performance tests use GITEA_LB_IP env var with fallback - VS Code launch.json uses env var references for DB_HOST and FUNCTION_RUNNER_IP - Remove hardcoded IP annotation from gitea-dev service-lb.yaml - Update docs and tutorials with dynamic discovery instructions Signed-off-by: Fiachra Corcoran <fiachra.corcoran@est.tech> * Address copilot comments Signed-off-by: Fiachra Corcoran <fiachra.corcoran@est.tech> --------- Signed-off-by: Fiachra Corcoran <fiachra.corcoran@est.tech> Co-authored-by: Liam Fallon <35595825+liamfallon@users.noreply.github.com>
1 parent 5e3ab70 commit 0654c7d

17 files changed

Lines changed: 378 additions & 51 deletions

File tree

.vscode/launch.json

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -60,7 +60,7 @@
6060
"WEBHOOK_HOST": "localhost",
6161
"GOOGLE_API_GO_EXPERIMENTAL_DISABLE_NEW_AUTH_LIB": "true",
6262
"DB_DRIVER": "pgx",
63-
"DB_HOST": "172.18.255.202",
63+
"DB_HOST": "${env:DB_HOST}",
6464
"DB_PORT": "5432",
6565
"DB_NAME": "porch",
6666
"DB_USER": "porch",
@@ -88,7 +88,7 @@
8888
"cwd": "${workspaceFolder}",
8989
"env": {
9090
"GIT_CACHE_DIR": "${workspaceFolder}/.cache-controller-repo",
91-
"DB_HOST": "172.18.255.201",
91+
"DB_HOST": "${env:DB_HOST}",
9292
"DB_PORT": "5432",
9393
"DB_NAME": "porch",
9494
"DB_USER": "porch",
@@ -117,13 +117,13 @@
117117
"cwd": "${workspaceFolder}",
118118
"env": {
119119
"GIT_CACHE_DIR": "${workspaceFolder}/.cache-controller-v1alpha2",
120-
"DB_HOST": "172.18.255.201",
120+
"DB_HOST": "${env:DB_HOST}",
121121
"DB_PORT": "5432",
122122
"DB_NAME": "porch",
123123
"DB_USER": "porch",
124124
"DB_PASSWORD": "porch",
125125
"DB_DRIVER": "pgx",
126-
"FUNCTION_RUNNER_ADDRESS": "172.18.255.202:9445"
126+
"FUNCTION_RUNNER_ADDRESS": "${env:FUNCTION_RUNNER_IP}:9445"
127127
}
128128
},
129129
// A configuration for running a porchctl command using the VS Code debugger.
@@ -180,7 +180,7 @@
180180
"mode": "auto",
181181
"program": "${workspaceFolder}/func/client/main.go",
182182
"args": [
183-
"--address=172.18.255.201:9445",
183+
"--address=${env:FUNCTION_RUNNER_IP}:9445",
184184
"--package=${workspaceFolder}/func/config/",
185185
"--image=gcr.io/kpt-fn/set-namespace:v0.2.0",
186186
"--",

deployments/local/metallb-conf.yaml

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,11 +5,13 @@ metadata:
55
namespace: metallb-system
66
spec:
77
addresses:
8+
# This static config assumes the kind Docker network uses 172.18.0.0/16.
9+
# For dynamic generation based on the actual kind network, use:
10+
# source scripts/get-kind-metallb-subnet.sh && generate_metallb_config | kubectl apply -f -
811
- 172.18.255.200-172.18.255.250
912
---
1013
apiVersion: metallb.io/v1beta1
1114
kind: L2Advertisement
1215
metadata:
1316
name: empty
1417
namespace: metallb-system
15-

deployments/local/porch-api-endpoints.yaml

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,9 @@ metadata:
77
namespace: porch-system
88
subsets:
99
- addresses:
10-
- ip: 172.17.0.1
10+
# This placeholder IP is overwritten dynamically by remove-porch-server-from-deployment-config.sh
11+
# using: docker network inspect bridge --format='{{(index .IPAM.Config 0).Gateway}}'
12+
- ip: 0.0.0.0
1113
ports:
1214
- appProtocol: https
1315
name: api

docs/content/en/docs/12_contributing/code-contribution/development-environment.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -139,7 +139,7 @@ This deploys all Porch components including PostgreSQL, except the server which
139139
**Default database connection details:**
140140
```bash
141141
DB_DRIVER=pgx
142-
DB_HOST=172.18.255.202
142+
DB_HOST=<postgres-lb-ip> # Discover with: kubectl get svc -n porch-system porch-postgresql-lb -o jsonpath='{.status.loadBalancer.ingress[0].ip}'
143143
DB_PORT=5432
144144
DB_NAME=porch
145145
DB_USER=porch
@@ -359,8 +359,8 @@ Set the correct function runner IP in your launch configuration:
359359
# Get the function runner IP
360360
kubectl get svc -n porch-system function-runner -o jsonpath='{.status.loadBalancer.ingress[0].ip}'
361361

362-
# Update launch.json
363-
"--function-runner=172.18.255.201:9445"
362+
# Set the FUNCTION_RUNNER_IP environment variable (used by launch.json)
363+
export FUNCTION_RUNNER_IP=$(kubectl get svc -n porch-system function-runner -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
364364
```
365365

366366
### E2E Tests Failing

examples/tutorials/starting-with-porch/metallb-conf.yaml

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,11 +5,13 @@ metadata:
55
namespace: metallb-system
66
spec:
77
addresses:
8-
- 172.18.255.200-172.18.255.250
8+
# Replace with a range from your kind Docker network subnet.
9+
# Discover it with: docker network inspect kind --format='{{(index .IPAM.Config 0).Subnet}}'
10+
# For a subnet of 172.18.0.0/16, use 172.18.255.200-172.18.255.250
11+
- METALLB_IP_RANGE_START-METALLB_IP_RANGE_END
912
---
1013
apiVersion: metallb.io/v1beta1
1114
kind: L2Advertisement
1215
metadata:
1316
name: empty
1417
namespace: metallb-system
15-

examples/tutorials/starting-with-porch/porch-repositories.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ spec:
1111
deployment: true
1212
type: git
1313
git:
14-
repo: http://172.18.255.200:3000/porch/edge1.git
14+
repo: http://GITEA_IP:3000/porch/edge1.git
1515
directory: /
1616
branch: main
1717
createBranch: true
@@ -33,7 +33,7 @@ spec:
3333
deployment: false
3434
type: git
3535
git:
36-
repo: http://172.18.255.200:3000/porch/management.git
36+
repo: http://GITEA_IP:3000/porch/management.git
3737
directory: /
3838
branch: main
3939
secretRef:

scripts/get-kind-metallb-subnet.sh

Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
#!/usr/bin/env bash
2+
# Copyright 2026 The kpt and Nephio Authors
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
16+
# Utility script to dynamically determine the MetalLB IP range from the kind Docker network.
17+
# This removes the need to hardcode 172.18.255.x addresses in scripts and configs.
18+
#
19+
# Usage:
20+
# source scripts/get-kind-metallb-subnet.sh
21+
# # Now METALLB_IP_RANGE_START and METALLB_IP_RANGE_END are set
22+
#
23+
# Or call individual functions:
24+
# get_metallb_ip_range -> sets METALLB_IP_RANGE_START and METALLB_IP_RANGE_END
25+
# get_service_lb_ip <svc> <ns> -> prints the LoadBalancer IP of a service
26+
# wait_for_service_lb_ip <svc> <ns> [timeout] -> waits for and prints the LB IP
27+
28+
# Only set strict mode when executed directly, not when sourced.
29+
# This avoids changing the caller's shell options unexpectedly.
30+
if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
31+
set -euo pipefail
32+
fi
33+
34+
# Get the subnet of the kind Docker network and derive a MetalLB-suitable IP range.
35+
# The range is placed at the top of the subnet's last octet range (x.x.255.200 - x.x.255.250)
36+
# to avoid conflicts with container IPs assigned by Docker.
37+
#
38+
# Sets: METALLB_IP_RANGE_START, METALLB_IP_RANGE_END
39+
get_metallb_ip_range() {
40+
local kind_network="${KIND_DOCKER_NETWORK:-kind}"
41+
42+
# Get the IPv4 subnet from the Docker network (e.g. "172.18.0.0/16")
43+
# Docker networks can have both IPv4 and IPv6 configs, so we iterate over all
44+
# IPAM configs and pick the one that looks like an IPv4 CIDR.
45+
local subnet
46+
subnet="$(docker network inspect "$kind_network" \
47+
--format='{{range .IPAM.Config}}{{.Subnet}} {{end}}' \
48+
| tr ' ' '\n' \
49+
| grep -E '^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+/' \
50+
| head -1)" \
51+
|| { echo "ERROR: Could not inspect Docker network '$kind_network'. Is kind running?" >&2; return 1; }
52+
53+
if [[ -z "$subnet" ]]; then
54+
echo "ERROR: No IPv4 subnet found on Docker network '$kind_network'." >&2
55+
return 1
56+
fi
57+
58+
# Parse the subnet base address and CIDR mask to compute a valid range.
59+
# We place the MetalLB pool at the high end of the subnet to avoid conflicts
60+
# with container IPs assigned by Docker.
61+
local base_ip cidr_mask
62+
base_ip="${subnet%%/*}"
63+
cidr_mask="${subnet##*/}"
64+
65+
# Convert base IP to integer for arithmetic
66+
local IFS='.'
67+
# shellcheck disable=SC2086
68+
set -- $base_ip
69+
local ip_int=$(( ($1 << 24) + ($2 << 16) + ($3 << 8) + $4 ))
70+
IFS=' '
71+
72+
# Calculate the number of host addresses in the subnet
73+
local host_bits=$(( 32 - cidr_mask ))
74+
local subnet_size=$(( 1 << host_bits ))
75+
76+
# Place the pool near the top of the subnet: last 51 addresses before broadcast
77+
# (broadcast = base + subnet_size - 1, so we use base + subnet_size - 52 to base + subnet_size - 2)
78+
local start_int=$(( ip_int + subnet_size - 52 ))
79+
local end_int=$(( ip_int + subnet_size - 2 ))
80+
81+
# Convert integers back to dotted-quad
82+
METALLB_IP_RANGE_START="$(( (start_int >> 24) & 255 )).$(( (start_int >> 16) & 255 )).$(( (start_int >> 8) & 255 )).$(( start_int & 255 ))"
83+
METALLB_IP_RANGE_END="$(( (end_int >> 24) & 255 )).$(( (end_int >> 16) & 255 )).$(( (end_int >> 8) & 255 )).$(( end_int & 255 ))"
84+
85+
export METALLB_IP_RANGE_START
86+
export METALLB_IP_RANGE_END
87+
}
88+
89+
# Generate a MetalLB IPAddressPool + L2Advertisement YAML config using the dynamic IP range.
90+
# Prints the YAML to stdout.
91+
generate_metallb_config() {
92+
get_metallb_ip_range
93+
94+
cat <<EOF
95+
apiVersion: metallb.io/v1beta1
96+
kind: IPAddressPool
97+
metadata:
98+
name: example
99+
namespace: metallb-system
100+
spec:
101+
addresses:
102+
- ${METALLB_IP_RANGE_START}-${METALLB_IP_RANGE_END}
103+
---
104+
apiVersion: metallb.io/v1beta1
105+
kind: L2Advertisement
106+
metadata:
107+
name: empty
108+
namespace: metallb-system
109+
EOF
110+
}
111+
112+
# Get the LoadBalancer IP of a Kubernetes service.
113+
# Args: <service-name> <namespace>
114+
# Prints the IP to stdout. Returns 1 if not available.
115+
get_service_lb_ip() {
116+
local svc_name="$1"
117+
local namespace="$2"
118+
119+
local ip
120+
ip="$(kubectl get svc "$svc_name" -n "$namespace" -o jsonpath='{.status.loadBalancer.ingress[0].ip}' 2>/dev/null)"
121+
122+
if [[ -z "$ip" || "$ip" == "null" ]]; then
123+
return 1
124+
fi
125+
126+
echo "$ip"
127+
}
128+
129+
# Wait for a LoadBalancer service to get an external IP assigned.
130+
# Args: <service-name> <namespace> [timeout-seconds]
131+
# Prints the IP to stdout once available.
132+
wait_for_service_lb_ip() {
133+
local svc_name="$1"
134+
local namespace="$2"
135+
local timeout="${3:-60}"
136+
137+
local elapsed=0
138+
local ip=""
139+
140+
while [[ $elapsed -lt $timeout ]]; do
141+
ip="$(get_service_lb_ip "$svc_name" "$namespace" 2>/dev/null)" && break
142+
sleep 2
143+
elapsed=$((elapsed + 2))
144+
done
145+
146+
if [[ -z "$ip" ]]; then
147+
echo "ERROR: Timed out waiting for LoadBalancer IP on $namespace/$svc_name after ${timeout}s" >&2
148+
return 1
149+
fi
150+
151+
echo "$ip"
152+
}
153+
154+
# Get the function-runner LoadBalancer IP.
155+
# Falls back to waiting if not yet assigned.
156+
get_function_runner_ip() {
157+
wait_for_service_lb_ip "function-runner" "porch-system" "${1:-60}"
158+
}
159+
160+
# Get the Gitea LoadBalancer IP.
161+
# Falls back to waiting if not yet assigned.
162+
get_gitea_ip() {
163+
wait_for_service_lb_ip "gitea-lb" "gitea" "${1:-60}"
164+
}

scripts/install-dev-gitea-setup.sh

Lines changed: 22 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ set -o pipefail # Check errors in piped commands
2020

2121
self_dir="$(dirname "$(readlink -f "$0")")"
2222
git_repo_name=${1:-porch-test}
23-
gitea_ip=${2:-172.18.255.200} # should be from the address range in deployments/local/metallb-conf.yaml
23+
gitea_ip=${2:-} # If provided, pins the Gitea LoadBalancer to this IP; otherwise MetalLB assigns from pool
2424

2525
git_root="$(readlink -f "${self_dir}/..")"
2626
TEST_BLUEPRINTS_PATH="${git_root}/test/pkgs/test-pkgs/test-blueprints.bundle"
@@ -107,16 +107,29 @@ cd "${git_root}/.build/gitea"
107107

108108
# Check if the gitea service of type LoadBalancer exists in the 'gitea' namespace
109109
if kubectl get svc gitea-lb -n gitea --no-headers 2>/dev/null | grep -q LoadBalancer; then
110-
h1 Gitea LoadBalancer service exists. Skipping mutations
110+
h1 Gitea LoadBalancer service exists. Checking IP allocation...
111+
112+
# If the service has a stale pinned IP that MetalLB can't allocate, remove the annotation
113+
# so MetalLB can assign a valid IP from the current pool.
114+
lb_ip="$(kubectl get svc gitea-lb -n gitea -o jsonpath='{.status.loadBalancer.ingress[0].ip}' 2>/dev/null)"
115+
if [[ -z "$lb_ip" || "$lb_ip" == "null" ]]; then
116+
echo "WARNING: gitea-lb has no allocated IP. Removing stale IP annotation to allow MetalLB reassignment..."
117+
kubectl annotate svc gitea-lb -n gitea metallb.universe.tf/loadBalancerIPs- 2>/dev/null || true
118+
# Also strip the stale annotation from the local kpt package so future applies don't re-pin it
119+
sed -i '/metallb.universe.tf\/loadBalancerIPs/d' "${git_root}/.build/gitea/service-lb.yaml" 2>/dev/null || true
120+
fi
111121
else
112122
h1 Gitea LoadBalancer service does not exist. Mutating pkg...
113-
114-
kpt fn eval \
115-
--image ghcr.io/kptdev/krm-functions-catalog/set-annotations:v0.1.7 \
116-
--match-kind Service \
117-
--match-name gitea-lb \
118-
--match-namespace gitea \
119-
-- "metallb.universe.tf/loadBalancerIPs=${gitea_ip}"
123+
124+
# Only pin a specific IP if one was explicitly provided; otherwise let MetalLB assign from pool
125+
if [[ -n "${gitea_ip}" ]]; then
126+
kpt fn eval \
127+
--image ghcr.io/kptdev/krm-functions-catalog/set-annotations:v0.1.7 \
128+
--match-kind Service \
129+
--match-name gitea-lb \
130+
--match-namespace gitea \
131+
-- "metallb.universe.tf/loadBalancerIPs=${gitea_ip}"
132+
fi
120133
fi
121134

122135
kpt fn render

scripts/remove-controller-from-deployment-config.sh

Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,8 +18,25 @@ set -e # Exit on error
1818
set -u # Must predefine variables
1919
set -o pipefail # Check errors in piped commands
2020

21-
function_runner_ip="${1:-172.18.255.202}"
2221
self_dir="$(dirname "$(readlink -f "$0")")"
22+
git_root="$(readlink -f "${self_dir}/..")"
23+
source "${git_root}/scripts/get-kind-metallb-subnet.sh"
24+
25+
# Discover function-runner IP dynamically. Accept as argument or env var for backward compat.
26+
if [[ -n "${1:-}" ]]; then
27+
function_runner_ip="$1"
28+
elif [[ -n "${FUNCTION_RUNNER_IP:-}" ]]; then
29+
function_runner_ip="$FUNCTION_RUNNER_IP"
30+
else
31+
# Try to get it from the running service first; fall back to deriving from MetalLB range
32+
function_runner_ip="$(get_service_lb_ip function-runner porch-system 2>/dev/null)" || {
33+
get_metallb_ip_range
34+
# Use third IP in the MetalLB range for function-runner when controller runs locally
35+
# (first is gitea, second is used when server runs locally)
36+
function_runner_ip="${METALLB_IP_RANGE_START%.*}.$((${METALLB_IP_RANGE_START##*.} + 2))"
37+
}
38+
fi
39+
2340
deployment_config_dir="${DEPLOYPORCHCONFIGDIR:-$(readlink -f "${self_dir}/../.build/deploy")}"
2441

2542
cd "${deployment_config_dir}"

scripts/remove-porch-server-from-deployment-config.sh

Lines changed: 16 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -19,11 +19,23 @@ set -u # Must predefine variables
1919
set -o pipefail # Check errors in piped commands
2020
self_dir="$(dirname "$(readlink -f "$0")")"
2121

22-
# function_runner_ip should match the --function-runner argument given to porch-server
23-
# (also, the IP should be from the address range specified in deployments/local/metallb-conf.yaml)
24-
function_runner_ip="172.18.255.201"
2522
git_root="$(readlink -f "${self_dir}/..")"
26-
deployment_config_dir="${DEPLOYPORCHCONFIGDIR:-${git_root}/.build/deploy)}"
23+
source "${git_root}/scripts/get-kind-metallb-subnet.sh"
24+
25+
# function_runner_ip should match the --function-runner argument given to porch-server.
26+
# Discover dynamically from the MetalLB pool if not set via environment variable.
27+
if [[ -n "${FUNCTION_RUNNER_IP:-}" ]]; then
28+
function_runner_ip="$FUNCTION_RUNNER_IP"
29+
else
30+
# Try to get it from the running service first; fall back to deriving from MetalLB range
31+
function_runner_ip="$(get_service_lb_ip function-runner porch-system 2>/dev/null)" || {
32+
get_metallb_ip_range
33+
# Use second IP in the MetalLB range for function-runner (first is typically gitea)
34+
function_runner_ip="${METALLB_IP_RANGE_START%.*}.$((${METALLB_IP_RANGE_START##*.} + 1))"
35+
}
36+
fi
37+
38+
deployment_config_dir="${DEPLOYPORCHCONFIGDIR:-${git_root}/.build/deploy}"
2739
cd "${deployment_config_dir}"
2840

2941
# expose function-runner to local processes

0 commit comments

Comments
 (0)