Weekly agent sandbox: compiled CLI can't traverse TLS-intercepting proxy — curl fallback shipped, CLI/MCP follow-ups

[zachdunn]

Context

The weekly scheduled maintenance routine (a sandboxed cloud agent) runs behind a TLS-intercepting egress proxy. The compiled releases CLI (a bun build --compile binary) cannot complete API requests there:

• Environment verified: CLI 0.73.0 (bun 1.3.13, linux-x64); proxy CA readable at /root/.ccr/ca-bundle.crt.
• Setting NODE_EXTRA_CA_CERTS / SSL_CERT_FILE / NODE_USE_SYSTEM_CA=1 does not fix it. The error is socket connection was closed unexpectedly, not a cert-verification code — so Bun's native HTTP client is failing before the CA env vars are consulted, and the CLI's cert-error hint never fires.
• curl works in the same sandbox (uses the system CA store / honors the proxy), and the agent runtime's own HTTP stack works (which is why MCP works there).
• The npm launcher (npm/releases/bin/releases) is a clean env pass-through (execFileSync(bin, args, { stdio: "inherit" }), no env: override), so env-var inheritance is not the cause — the failure is inside Bun's linux-x64 compiled fetch.

Shipped (unblocks the weekly routine now)

A curl-to-REST fallback was added to the maintenance skills, since every CLI command is a thin wrapper over an existing admin REST route:

• regenerating-overviews → new "Running without the CLI (curl fallback)" section: overview inputs (GET /v1/orgs/:slug/overview/inputs), generate, overview update (POST /v1/orgs/:slug/overview), plus the plan manifest (GET /v1/admin/overviews?format=plan). Handles the --max-content-chars client-side-clipping nuance via a file + jq.
• maintaining-orgs → pointer to the fallback; documents that admin source fetch has no curl equivalent yet.

Remaining

• Diagnose A vs B (in the scheduled agent's sandbox): run env | grep -iE 'proxy|ccr|cert|ssl' and curl -v https://api.releases.sh/v1/health. If an HTTP(S)_PROXY var / CONNECT tunnel is present → it's a proxy-routing issue; otherwise a TLS-handshake issue with the intercept cert.
• CLI Bun fix (repo: buildinternet/releases-cli): depending on A/B — bump the release-CI Bun pin (1.3.13 in release.yml/test.yml) and rebuild, and/or have apiFetch explicitly read HTTPS_PROXY/HTTP_PROXY and pass Bun's { proxy } option to fetch (Bun's fetch supports both the option and the env vars). Low-risk belt-and-suspenders regardless of A/B.
• MCP admin-surface decision — the remote MCP server is a read-only public catalog and does not expose overview plan/inputs/update as tools. Deciding whether to add write-scoped MCP tools for orchestrator maintenance (running into USER_API_KEY_MAX_SCOPE = read — needs a relk_/root principal, not a relu_ user key) is a deliberate product call, not a reflexive fix. Track the yes/no here before building.
• admin source fetch curl fallback for maintaining-orgs — it dispatches a managed update run and polls (--wait), not a single REST call, so the sandbox currently skips the fetch and regenerates from indexed data. Document the fetch-trigger REST path (workflow dispatch + status poll) so the sandbox can pre-fetch too.
• Confirm the scheduled agent's principal — the overview write needs write scope under any path (CLI-fixed or MCP). Verify the sandbox holds the root RELEASES_API_KEY or a relk_ write/admin token.

Decision recorded

For the weekly routine, MCP + curl-to-REST is the correct channel given the sandbox constraint (the agent runtime trusts the intercept CA; the standalone Bun binary doesn't). The CLI Bun fix is the durable improvement; the MCP admin tools are an optional, separate expansion.

[zachdunn]

Root cause found: stale proxy-port override, not a Bun/CA issue

Diagnostic run in the scheduled agent's sandbox settles this as hypothesis (A) proxy routing — and more specifically a stale proxy port, not a Bun TLS or CA-trust problem.

• curl -v through the proxy: clean CONNECT tunnel (HTTP 200), full TLS 1.3 handshake, cert verified against /root/.ccr/ca-bundle.crt (SSL certificate verify ok). No interception problem.
• env | grep -i proxy: the ambient env already carries the correct live port in both cases — HTTPS_PROXY=http://127.0.0.1:44959 and https_proxy=…:44959. (no_proxy does not list releases.sh, so API traffic correctly routes through the proxy.)
• The earlier "socket connection was closed unexpectedly" failures were self-inflicted: the agent was manually overriding HTTPS_PROXY with …:42041, a port captured from a prior sandbox session. The proxy port rotates per session. Bun's fetch honors HTTPS_PROXY natively, so it faithfully dialed the dead port → socket close.

Fix

Do not hardcode the port (HTTPS_PROXY=…:44959 releases … reintroduces the bug next session). Trust the ambient HTTPS_PROXY and run the CLI bare. Pending verification:

releases --version && RELEASES_API_KEY="$RELEASES_API_KEY" releases admin overview plan --json | head -c 300

If the bare run works, the CLI is viable in-sandbox as-is — no CA patch, no Bun upgrade, no curl fallback, no MCP-primary needed.

Impact on the checklist above

• CLI Bun fix (pin bump / explicit {proxy}) — moot. Bun already honors ambient HTTPS_PROXY; there was never a Bun bug. An explicit {proxy} read in apiFetch is unnecessary.
• Curl-to-REST fallback — still functional, but demoted to an optional backup. Its "why" (Bun surfaces the intercept as socket-close) is a misdiagnosis and will be corrected in the skills once the bare run is confirmed.
• MCP admin-tool decision — still an independent, deliberate call; unaffected by this finding.
• admin source fetch sandbox path — with the CLI working, the sandbox can run it directly; no curl equivalent needed.
• Scheduled-agent principal — still must hold a write-capable key (relk_/root) for the overview write.

Net: the weekly routine's fix is a one-liner — never override the proxy port; use the ambient HTTPS_PROXY.

[zachdunn]

Correction: not the proxy port — it's the compiled Bun binary's networking

The stale-port theory (prior comment) was wrong. Follow-up testing in the sandbox:

• The bare command (ambient HTTPS_PROXY, no override) still fails with socket connection was closed unexpectedly.
• The proxy port rotates per shell invocation (42041 → 44959 → 37839), and Bun reads the current ambient value correctly each time — the error is independent of the port value.
• Adding api.releases.sh to NO_PROXY (forcing a direct connection) also fails — but that's expected: direct egress is blocked in the sandbox (the proxy is mandatory), so it exonerates nothing.
• curl through the same proxy works cleanly: CONNECT tunnel, TLS 1.3, cert verified against /root/.ccr/ca-bundle.crt.

Conclusion: the fault is inside the compiled Bun binary's proxy-tunnel networking (post-CONNECT TLS to the intercepted origin), not any environment configuration. Every config lever is exhausted: correct ambient proxy, all CA env vars (NODE_EXTRA_CA_CERTS/SSL_CERT_FILE/NODE_USE_SYSTEM_CA), and NO_PROXY. Likely mechanism: Bun's tunneled TLS doesn't apply the ccr CA, or an HTTP/2-over-CONNECT bug on this kernel/libc.

Decision

CLI-in-sandbox is not fixable by configuration. The curl-to-REST fallback in regenerating-overviews/maintaining-orgs is the sandbox path (not a backup); MCP for the reads it exposes.

Remaining CLI-repo work (buildinternet/releases-cli) — does not block the scheduled agent

• Diagnostic: NODE_TLS_REJECT_UNAUTHORIZED=0 releases admin overview plan --json (diagnostic ONLY, disables TLS verification). Works ⇒ Bun isn't trusting the ccr CA on the tunneled connection (targeted, fixable). Fails ⇒ deeper Bun proxy/HTTP-2 bug.
• Bump the Bun pin (1.3.13 in release.yml/test.yml), rebuild linux-x64, re-test — most likely real fix if it's a Bun networking bug.
• Fallback build option: a Node-runtime CLI build (Node's fetch was confirmed working in this env, and the npm launcher is already Node) — larger distribution change, only if the Bun path stays broken.

No longer relevant

• explicit {proxy} in apiFetch — Bun already reads HTTPS_PROXY correctly; not the issue.

[zachdunn]

Diagnostic closed: genuine Bun networking bug, no env-var fix

NODE_TLS_REJECT_UNAUTHORIZED=0 releases admin overview plan --json → same socket connection was closed unexpectedly. Disabling cert verification changed nothing, so it is not a CA-trust-on-tunnel issue — it's a genuine Bun proxy/HTTP-2 bug in the compiled binary on this kernel/libc.

No targeted env-var fix exists. Config path is fully exhausted (proxy port, all CA vars, NO_PROXY, TLS-reject-off).

Resolution paths (CLI repo, non-blocking for the scheduled agent, which uses the curl-to-REST skill fallback):

1. Bun upgrade + rebuild the linux-x64 binary (bump the 1.3.13 pin in release.yml/test.yml), then re-test in-sandbox.
2. Node-runtime CLI build if the Bun path stays broken — Node's fetch works in this environment and the npm launcher is already Node.

Sandbox path stays: curl-to-REST fallback (shipped in regenerating-overviews + maintaining-orgs), MCP for reads it exposes.