Skip to content

feat: Productionize --container-runtime sbx (Docker sbx microVM support) #6117

Description

@lpcox

Summary

Productionize the Docker sbx microVM runtime support in AWF. The prototype is implemented in #6101 (branch feat/docker-sbx-runtime). This issue tracks the remaining work to make --container-runtime sbx production-ready: config spec alignment with the gh-aw compiler (github/gh-aw#44978), schema updates, documentation, and implementation hardening.

Context

When container.containerRuntime: "sbx" is set in the AWF stdin config (or --container-runtime sbx CLI flag), AWF:

  1. Starts infrastructure containers (Squid, api-proxy) in Docker Compose as normal
  2. Omits the agent from docker-compose.yml (no Compose agent service)
  3. Creates a Docker sbx microVM via the sbx CLI
  4. Runs the user command inside the microVM
  5. Routes all microVM egress through Squid (via proxy env vars + Docker port publishing)
  6. Routes API calls through the api-proxy sidecar (via host.docker.internal)

The microVM cannot reach Docker-internal IPs (172.30.0.x). All communication uses:

  • 172.17.0.0:3128 — Squid proxy (port-published)
  • host.docker.internal:10000-10004 — api-proxy (reachable via docker0 bridge + awf-ext network)

Config & Schema Changes

AWF Stdin Config (awf-config-schema.json)

The container.containerRuntime enum already includes "sbx". Verify the schema description is accurate and add any sbx-specific fields if needed.

Current schema:

"containerRuntime": {
  "type": "string",
  "enum": ["gvisor", "sbx"],
  "description": "Container runtime for the agent container. \"gvisor\" runs the agent under gVisor's runsc runtime (OCI runtime, compose-based). \"sbx\" runs the agent inside a Docker sbx microVM with hypervisor isolation; infrastructure containers (squid-proxy, api-proxy) stay in Docker Compose on the host and the sbx proxy chains upstream through AWF's Squid for domain filtering. Only the agent uses the custom runtime; infrastructure containers always use the default runc runtime."
}

Alignment with gh-aw compiler (github/gh-aw#44978):

  • gh-aw frontmatter: sandbox.agent.runtime: docker-sbx
  • AWF config: container.containerRuntime: "sbx"
  • The compiler maps docker-sbxsbx when generating the AWF config JSON

CLI Flag

Already implemented:

--container-runtime sbx

No changes needed to the CLI option definition.

Implementation Checklist

Already Done (PR #6101)

  • container-runtime.ts: RUNTIME_REGISTRY entry for sbx with executionModel: 'microvm'
  • compose-generator.ts: Omit agent service when runtimeUsesComposeAgent() returns false
  • compose-generator.ts: Publish api-proxy ports for microVM runtimes
  • compose-generator.ts: Attach api-proxy to awf-ext network for bridge reachability
  • main-action.ts: sbx sandbox creation, environment setup, health checks, agent execution
  • sbx-manager.ts: createSandbox(), execInSandbox(), removeSandbox(), isSbxAvailable()
  • Api-proxy accessible via host.docker.internal (not Docker-internal IPs)
  • Credential isolation: agent never sees real auth tokens
  • network.isolation: true enforced (no host iptables)
  • End-to-end CI smoke test passing

Remaining Work

  • Remove diagnostic logging — Strip [sbx-env] and [sbx-diag] verbose log lines from main-action.ts. Keep logger.info for key milestones only (sandbox created, health check passed, agent started, agent exited).

  • Artifact permission repair — Fix "Rootless artifact permission repair failed" warnings for /tmp/gh-aw/sandbox/firewall/logs. Squid logs are owned by the squid user inside the container; AWF's cleanup can't chmod them rootlessly. Options:

    • Skip permission repair for sbx mode (logs are still accessible via docker cp)
    • Run squid with same UID as the host user
    • Use docker cp to extract logs before cleanup
  • AWF_REFLECT_URL for sbx — The copilot harness tries http://api-proxy:10000/reflect which fails from sbx (Docker name not resolvable). Set AWF_REFLECT_URL=http://host.docker.internal:10000/reflect in the sbx environment so the harness can reach it.

  • Health check timeout configurability — Currently hardcoded to 30s. Consider making this configurable via config or defaulting based on runtime type.

  • Cleanup on failure — Ensure sbx stop + sbx rm always runs on agent failure/timeout, even if the process is killed (signal handling).

  • Unit tests — Add/expand tests:

    • buildAgentCredentialEnv() with host.docker.internal for sbx
    • runtimeUsesComposeAgent('sbx') returns false
    • Compose generator omits agent service for sbx
    • Api-proxy gets awf-ext network attachment for sbx
    • Api-proxy ports are published for sbx
  • Documentation updates:

    • README.md: Add sbx runtime to the container runtime section
    • docs/environment.md: Document sbx-specific env vars (SBX_GATEWAY_IP, SBX_HOST_DOCKER_INTERNAL)
    • AGENTS.md: Update architecture section with microVM execution model
    • Schema description: Verify accuracy

Architecture (for documentation)

awf --container-runtime sbx --config <config.json> -- <command>
    ↓
CLI: parseConfig() → containerRuntime = "sbx"
    ↓
runtimeUsesComposeAgent("sbx") → false (microVM execution model)
    ↓
Phase 1: Docker Compose starts infrastructure only:
├── awf-squid (172.30.0.10, ports 3128:3128 on host)
├── awf-api-proxy (172.30.0.30, on awf-ext bridge)
└── awf-net (internal: true) + awf-ext (bridge)
    ↓
Phase 2: sbx microVM creation:
├── sbx create shell --name awf-agent-<id> <workspace>
├── Mount workspace + additional mounts
├── Set proxy env vars (HTTPS_PROXY=http://172.17.0.0:3128)
├── Set credential env vars (COPILOT_API_URL=http://host.docker.internal:10002)
└── Exclude real auth tokens (--exclude-env)
    ↓
Phase 3: Health check (poll host.docker.internal:10000/health from inside sbx)
    ↓
Phase 4: Agent execution (sbx exec awf-agent-<id> <command>)
    ↓
Phase 5: Cleanup (sbx stop + rm, docker compose down)

Network Topology

sbx microVM (KVM)                    Host Docker
┌─────────────────┐                 ┌──────────────────────────┐
│ eth0            │                 │ docker0 bridge (172.17.0.1)│
│ gw: 172.17.0.0 │                 │                            │
│                 │    port 3128    │ awf-squid (awf-ext bridge) │
│ HTTPS_PROXY ───────────────────► │   ├── awf-net (internal)   │
│ =172.17.0.0:3128│                 │   └── awf-ext (published)  │
│                 │                 │                            │
│ COPILOT_API_URL │ host.docker.   │ awf-api-proxy (awf-ext)    │
│ =host.docker.  ─── internal ───► │   ports 10000-10004        │
│  internal:10002 │                 │                            │
│                 │                 │ awmg-mcpg (bridge)         │
│ MCP gateway ──────────────────► │   port 8080 (0.0.0.0)      │
│ =host.docker.  │                 │                            │
│  internal:8080  │                 └──────────────────────────┘
└─────────────────┘
  Direct outbound → BLOCKED (no route except through proxy)

Security Model

  1. Credential isolation: Real API tokens (COPILOT_GITHUB_TOKEN, GITHUB_MCP_SERVER_TOKEN, MCP_GATEWAY_API_KEY) are excluded from the sbx environment via --exclude-env. The agent only has dummy BYOK keys.
  2. Network isolation: network.isolation: true always — no host iptables, no NET_ADMIN. Enforcement via Docker network topology (internal network + Squid as sole egress).
  3. Hypervisor boundary: The sbx microVM runs under KVM — stronger isolation than OCI containers (gVisor) or chroot.
  4. Api-proxy injection: The api-proxy sidecar injects real credentials upstream, invisible to the agent.

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions