Skip to content

[awf] cli-proxy: Harden DIFC proxy liveness probe with retry/backoff #4674

Description

@lpcox

Problem

The awf-cli-proxy sidecar fails fast after only 2 liveness probes (1s apart) when the external DIFC proxy at localhost:18443 is slow to accept connections. On 2026-06-10 (07:46–07:53 UTC), this caused two scheduled workflows (Auto-Triage, Sub-Issue Closer) to abort at startup — 0 agent turns executed across both runs.

Context

Originally reported in github/gh-aw#38309. Affected workflow runs: 27261698585, 27261373050. Both show the same signature: connect: connection refused on [::1]:18443. Audit confirmed no firewall-policy regression — the agent simply never started.

Root Cause

The cli-proxy liveness probe in containers/api-proxy/ (or equivalent startup logic) retries only 2 times with a 1s delay. Under concurrent scheduled job load, the external DIFC proxy takes longer than 2s to accept connections, triggering a fail-fast abort that kills the entire firewall startup.

Proposed Solution

  1. Increase retry budget: replace the 2-attempt/1s fail-fast with exponential backoff (~5 attempts over 15–30s total) in the cli-proxy probe logic.
  2. Bind parity: confirm the localhost:18443 tunnel listener binds 127.0.0.1 to match the probe's [::1] resolution, or pin both to IPv4.
  3. Optional (P2): mirror node:lts-alpine from Docker Hub to GHCR to remove Docker Hub from the critical image-pull path.

Generated by Firewall Issue Dispatcher · 298 AIC · ⊞ 27.8K ·

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions