Skip to content

Commit 100b9f8

Browse files
committed
ci(deploy): speed up ASG refresh (120s warmup) and allow 60 min
The aws-dev portal ASG runs a larger fleet (8 in-service + warm pool = 12 instances). A full instance refresh at 300s InstanceWarmup ran ~60 min and exceeded the poll cap. Cut InstanceWarmup to 120s (bootstrap completes the launch hook in ~90s and the ALB target-group health check gates real traffic, so availability is preserved by MinHealthyPercentage 50) and raise the poll cap to 60 min for the full in-service + warm-pool set.
1 parent 1de289c commit 100b9f8

1 file changed

Lines changed: 12 additions & 7 deletions

File tree

.github/workflows/_shifter-platform.yml

Lines changed: 12 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1106,10 +1106,15 @@ jobs:
11061106
else
11071107
# Start instance refresh
11081108
# New instances read the updated image digest from Parameter Store via user_data.
1109-
# Lifecycle hook ensures instance is ready before joining ALB
1109+
# Lifecycle hook ensures instance is ready before joining ALB.
1110+
# InstanceWarmup is 120s: bootstrap completes the launch hook in ~90s
1111+
# and the ALB target-group health check gates real traffic, so a
1112+
# shorter warmup keeps a full (in-service + warm-pool) refresh inside
1113+
# the poll window without cutting availability (MinHealthyPercentage
1114+
# still holds 50% in service).
11101115
REFRESH_ID=$(aws autoscaling start-instance-refresh \
11111116
--auto-scaling-group-name "$ASG_NAME" \
1112-
--preferences '{"MinHealthyPercentage": 50, "InstanceWarmup": 300}' \
1117+
--preferences '{"MinHealthyPercentage": 50, "InstanceWarmup": 120}' \
11131118
--query "InstanceRefreshId" --output text)
11141119
11151120
echo "Instance refresh started: $REFRESH_ID"
@@ -1119,11 +1124,11 @@ jobs:
11191124
# Wait for instance refresh to complete
11201125
echo "Waiting for instance refresh to complete..."
11211126
RETRIES=0
1122-
# A full 8-instance refresh (MinHealthyPercentage 50, 300s warmup per
1123-
# batch, plus per-instance termination drain) runs ~34 min in dev, so
1124-
# a 30-minute cap timed the step out right at the finish line even
1125-
# after the ASG launch hook was fixed. 45 min gives real headroom.
1126-
MAX_RETRIES=90 # 45 minutes max wait
1127+
# A full refresh covers in-service + warm-pool instances (12 in dev),
1128+
# each with warmup plus per-instance termination drain, so it runs
1129+
# well beyond the original 30-minute cap. Combined with the reduced
1130+
# InstanceWarmup above, 60 minutes gives headroom for the full set.
1131+
MAX_RETRIES=120 # 60 minutes max wait
11271132
while true; do
11281133
STATUS=$(aws autoscaling describe-instance-refreshes \
11291134
--auto-scaling-group-name "$ASG_NAME" \

0 commit comments

Comments
 (0)