Skip to content

Commit e095340

Browse files
committed
proxy: dial the tunnel through an HTTP proxy when the guest requires one
The tunnel client (`proxy run --url`) used the runtime's native WebSocket, which ignores HTTP(S)_PROXY and always dials direct. In a sandbox whose only egress is an explicit proxy gateway (e.g. Docker Sandboxes), the broker was unreachable without an external socat shim. When a proxy applies to the tunnel URL (HTTP_PROXY/HTTPS_PROXY/ALL_PROXY, honoring NO_PROXY), the client now establishes an HTTP CONNECT tunnel through it, optional TLS for wss://, and the WS handshake itself, then speaks RFC 6455 over the socket. This hand-rolled path is runtime-agnostic (net/tls/crypto), so it behaves identically under Node and the compiled Bun binary; the direct dial is unchanged. Adds a Docker Sandboxes guide.
1 parent b76721d commit e095340

7 files changed

Lines changed: 635 additions & 4 deletions

File tree

.bumpy/tunnel-connect-proxy.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
varlock: minor
3+
---
4+
5+
proxy run --url now dials its tunnel through an HTTP proxy (HTTP(S)_PROXY/NO_PROXY), so a sandboxed agent whose only egress is a proxy gateway (e.g. Docker Sandboxes) can reach a broker

packages/varlock-website/src/content/docs/guides/proxy/running.mdx

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,8 @@ node agent.js # now routed through the proxy
4343

4444
The workload does not have to be on the same machine. `proxy start --expose` makes the proxy reachable off-loopback and serves a built-in WebSocket tunnel, gated by a per-session data-plane token, and `proxy run --url wss://<host>` runs a command through that broker from anywhere, self-wiring the placeholder env and CA certs over the tunnel. The token is a credential: pin it with `VARLOCK_PROXY_TOKEN` or read it back with `varlock proxy token`, and prefer passing it to clients as an env var rather than a `--token` argument. This is how cloud sandboxes reach a broker; see the [E2B](/sandboxes/e2b/) and [Fly.io](/sandboxes/flyio/) guides for full recipes and the [proxy CLI reference](/reference/cli/proxy/) for the flags.
4545

46+
When the guest's only egress is an explicit HTTP proxy (a corporate proxy, or a sandbox gateway like [Docker Sandboxes](/sandboxes/docker-sandboxes/)), `proxy run --url` dials the tunnel through it automatically: it honors `HTTP_PROXY` / `HTTPS_PROXY` / `ALL_PROXY` and `NO_PROXY` (a loopback or `NO_PROXY`-matched broker still dials direct). Only HTTP `CONNECT` proxies are supported, not SOCKS. The tunnel carries TLS end to end, so an intermediate proxy that terminates TLS only ever sees the encrypted tunnel.
47+
4648
## Sessions
4749

4850
Every `varlock proxy` command operates on a **session**: one running proxy with its own short id (printed by `proxy start`, listed by `proxy status`). You target a session in one of two ways:
Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
---
2+
title: Docker Sandboxes
3+
description: Using varlock with Docker Sandboxes (sbx), so an agent in a local microVM routes through the varlock credential proxy and holds only placeholders.
4+
---
5+
6+
[Docker Sandboxes](https://docs.docker.com/ai/sandboxes/) (the `sbx` CLI) runs agents in local microVMs. Each sandbox gets its own Docker daemon, filesystem, and network, and all egress is forced through a host-side gateway with a deny-by-default domain allowlist (`sbx policy`). That gateway is the seam varlock plugs into: point the agent at a varlock broker, allow the broker in policy, and the agent holds only [placeholders](/guides/proxy/rules/#placeholders) while varlock injects real secrets at the wire.
7+
8+
The recommended shape for local development is a **broker on your own machine**: secrets, resolver [plugins](/guides/plugins/), biometric unlock, and the interactive request log all stay on the host, and the sandbox reaches the broker through the sbx gateway. To share one broker across a fleet, run it on [infrastructure you operate](#remote-broker) and reach it at a public URL instead.
9+
10+
## How sbx egress works
11+
12+
Two facts about the gateway shape the setup:
13+
14+
- **Everything goes through the gateway.** Inside a sandbox, `HTTP_PROXY` / `HTTPS_PROXY` point at `gateway.docker.internal:3128`, the sbx CA is in the trust store, and non-allowed domains do not even resolve. `varlock proxy run --url` [honors those proxy env vars](/guides/proxy/running/#remote-proxy-start---expose--proxy-run---url) and dials its tunnel through the gateway automatically, so no shim is needed.
15+
- **The gateway can reach a host-local service** when you allow it in policy. This is what makes a host broker reachable from the microVM.
16+
17+
## Broker on your machine
18+
19+
```
20+
[sbx microVM] [your host]
21+
varlock proxy run --url varlock proxy start --expose
22+
ws://host.docker.internal:PORT (real secrets, plugins, request log)
23+
│ ▲
24+
└──▶ gateway.docker.internal:3128 ────┘ (allowed by sbx policy)
25+
```
26+
27+
### 1. Schema
28+
29+
Mark the secrets your agent uses with [`@proxy(domain=...)`](/reference/item-decorators/#proxy) and give each an explicit [`@placeholder`](/reference/item-decorators/#placeholder):
30+
31+
```env-spec title=".env.schema"
32+
# @proxy(domain="api.anthropic.com")
33+
# @placeholder=sk-ant-api03-000000000000000000000000
34+
ANTHROPIC_API_KEY=
35+
```
36+
37+
Egress is permissive by default (unmatched hosts pass through untouched, which is fine because the agent holds only placeholders). To make the broker refuse anything without a rule, set [`@proxyConfig={egress="strict"}`](/guides/proxy/rules/#egress-modes) in the schema header.
38+
39+
### 2. Start the broker on the host
40+
41+
`--expose` binds off-loopback and serves the [tunnel](/guides/proxy/running/#remote-proxy-start---expose--proxy-run---url), minting a data-plane token. Pin the token so you can hand the same value to the agent:
42+
43+
```bash
44+
export VARLOCK_PROXY_TOKEN=$(uuidgen)
45+
varlock proxy start --expose --port 8080
46+
```
47+
48+
### 3. Allow the broker in sbx policy
49+
50+
The gateway rewrites `host.docker.internal` to `localhost` before evaluating policy, so the rule that matches is for **`localhost`**, even though the agent connects to `host.docker.internal`:
51+
52+
```bash
53+
sbx policy allow network "localhost:8080"
54+
```
55+
56+
:::caution[sbx quirk]
57+
Allowing `host.docker.internal:8080` does **not** work (and `sbx policy check` will misleadingly say it is allowed). Allow `localhost:8080`. This is a Docker Sandboxes behavior, not a varlock one.
58+
:::
59+
60+
### 4. Run the agent through the broker
61+
62+
Install varlock in the sandbox, then wrap the agent command with `proxy run --url`. The agent connects to the broker at `host.docker.internal`, and varlock self-wires its placeholder env and CA certs over the tunnel:
63+
64+
```bash
65+
# in a shell sandbox (sbx create shell . && sbx exec <name> -- ...):
66+
curl -sSfL https://varlock.dev/install.sh | sh -s
67+
68+
VARLOCK_PROXY_TOKEN=$YOUR_TOKEN \
69+
varlock proxy run --url ws://host.docker.internal:8080 -- your-agent-command
70+
```
71+
72+
That is the whole path: the agent holds placeholders, the broker on your host injects real values only on verified TLS connections to hosts your schema allows, and every request is checked against your [`@proxy` rules](/guides/proxy/rules/#routing-rules) and recorded in the [audit log](/guides/proxy/running/#auditing).
73+
74+
:::note[Installing varlock in the sandbox]
75+
The install script drops a self-contained binary and works on any template. Templates on Node 22.3+ (the `shell` template is) can instead `npm i -g varlock`, a lighter install. Baking varlock into a [custom template](https://docs.docker.com/ai/sandboxes/customize/) skips the per-sandbox install, which matters for a fleet. Pass the token via the environment (`sbx exec -e` or a template default), not on the command line, so it stays out of process listings.
76+
:::
77+
78+
## Remote broker
79+
80+
To share one broker across machines or a fleet, run it on infrastructure you operate and expose it at a URL that carries WebSockets. Because sbx allows direct TLS to allowlisted domains, the agent reaches a public `wss://` broker directly (no gateway asymmetry), so you only allow the broker's domain:
81+
82+
```bash
83+
sbx policy allow network "broker.example.com"
84+
# in the sandbox:
85+
VARLOCK_PROXY_TOKEN=$YOUR_TOKEN \
86+
varlock proxy run --url wss://broker.example.com -- your-agent-command
87+
```
88+
89+
The data-plane token gates the tunnel and the tunnel carries TLS end to end, so a public URL is safe and any intermediary only sees ciphertext. See the [topologies overview](/sandboxes/overview/#topologies) and the [E2B](/sandboxes/e2b/) / [Fly.io](/sandboxes/flyio/) guides for the same broker shape on cloud providers.
90+
91+
## varlock proxy vs sbx secrets
92+
93+
Docker Sandboxes ships its own credential injection (`sbx secret`): the gateway substitutes a stored keychain value into request headers for a matching host. It covers the basic case. Route through a varlock broker when you want:
94+
95+
- **Custody in your secret manager.** Secrets come from wherever you already keep them through [plugins](/guides/plugins/) (1Password, Vault, AWS, Doppler, ...) and stay in your custody, instead of being copied into another store.
96+
- **One schema.** Your `.env.schema` describes every value, its type, and its routing in one declarative layer, legible to people and agents alike.
97+
- **Response scrubbing.** varlock scans response bodies and redacts injected secret values, so an allowlisted endpoint that echoes a request header cannot hand the real secret back to the agent.
98+
- **Policy and audit.** Match on path and method, [hot-reload](/guides/proxy/running/#editing-the-schema-while-a-session-is-running) the schema, and keep your own [audit log](/guides/proxy/running/#auditing).
99+
100+
The two compose: sbx provides microVM isolation and deny-by-default egress; the varlock broker provides custody, injection, and scrubbing.
101+
102+
## Trust model
103+
104+
The broker holds real secrets on whatever machine runs it. For a host broker in local development, that is your own machine, the same place the secrets already live. The agent's microVM never holds them: a compromised or prompt-injected agent yields placeholders and only the requests your rules and egress mode allow. Keep sbx policy tight (allow just the broker, plus whatever hosts the agent legitimately needs), and treat the data-plane token like any shared secret (rotate by restarting the broker with a new one).

packages/varlock-website/src/content/docs/sandboxes/overview.mdx

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,7 @@ Cloud sandbox providers run the agent in a remote VM, so the proxy is reached ov
2828
<CardGrid>
2929
<LinkCard title="E2B" href="/sandboxes/e2b/" description="Broker sandbox or tunnel to your machine; egress lockdown via network rules" />
3030
<LinkCard title="Fly.io" href="/sandboxes/flyio/" description="Broker sprite or tunnel to your machine; egress lockdown via network policy" />
31+
<LinkCard title="Docker Sandboxes" href="/sandboxes/docker-sandboxes/" description="Local microVMs (sbx); reach a host or remote broker through the gateway" />
3132
</CardGrid>
3233

3334
## Local tools
@@ -41,4 +42,4 @@ Cloud sandbox providers run the agent in a remote VM, so the proxy is reached ov
4142
<LinkCard title="MXC" href="/sandboxes/mxc/" description="Windows AppContainer (processcontainer). Windows 11" />
4243
</CardGrid>
4344

44-
Tools that already broker credentials at the network boundary (Docker Sandboxes, microsandbox, Anthropic `srt` credential `mask`, and similar) are out of scope here. Use those on their own, or wait for varlock host bridging if you want varlock as the broker inside a microVM that cannot reach host loopback.
45+
Some tools already broker credentials at their own network boundary (microsandbox, Anthropic `srt` credential `mask`, and similar). You can use those on their own; varlock adds schema-driven policy, response scrubbing, your own audit log, and custody in your secret manager on top. [Docker Sandboxes](/sandboxes/docker-sandboxes/) has such a gateway but also lets its policy reach a varlock broker, so it gets its own guide above (host broker or remote broker, with varlock as the injector).

packages/varlock-website/src/sidebar.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -259,6 +259,7 @@ export const sidebar: StarlightUserConfig['sidebar'] = [
259259
items: [
260260
{ label: 'E2B', slug: 'sandboxes/e2b' },
261261
{ label: 'Fly.io', slug: 'sandboxes/flyio' },
262+
{ label: 'Docker Sandboxes', slug: 'sandboxes/docker-sandboxes' },
262263
],
263264
},
264265
{

packages/varlock/src/proxy/tunnel.test.ts

Lines changed: 160 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,12 +5,63 @@ import { randomBytes } from 'node:crypto';
55
import { describe, expect, test } from 'vitest';
66

77
import {
8-
attachTunnelServer, fetchTunnelBootstrap, startTunnelClientListener, TUNNEL_TOKEN_HEADER, TUNNEL_PATH,
8+
attachTunnelServer, fetchTunnelBootstrap, startTunnelClientListener, proxyForTunnelUrl,
9+
TUNNEL_TOKEN_HEADER, TUNNEL_PATH,
910
type TunnelBootstrap,
1011
} from './tunnel';
1112

1213
const TOKEN = 'tunnel-test-token';
1314

15+
/** Set env vars for a test, returning a restore fn that puts the prior values back. */
16+
function withEnvVars(vars: Record<string, string>): () => void {
17+
const prev: Record<string, string | undefined> = {};
18+
for (const [key, value] of Object.entries(vars)) {
19+
prev[key] = process.env[key];
20+
process.env[key] = value;
21+
}
22+
return () => {
23+
for (const [key, was] of Object.entries(prev)) {
24+
if (was === undefined) {
25+
delete process.env[key];
26+
} else {
27+
process.env[key] = was;
28+
}
29+
}
30+
};
31+
}
32+
33+
/** A minimal HTTP CONNECT proxy standing in for a sandbox egress gateway. Counts
34+
* how many CONNECTs it served so tests can assert traffic actually went through it. */
35+
function startConnectProxy(): Promise<{ port: number; connects: () => number; close: () => void }> {
36+
let connects = 0;
37+
return new Promise((resolve) => {
38+
const srv = http.createServer();
39+
srv.on('connect', (req, clientSock, head) => {
40+
connects += 1;
41+
const [host, port] = (req.url ?? '').split(':');
42+
const upstream = net.connect(Number(port), host, () => {
43+
clientSock.write('HTTP/1.1 200 Connection Established\r\n\r\n');
44+
if (head?.length) upstream.write(head);
45+
upstream.pipe(clientSock);
46+
clientSock.pipe(upstream);
47+
});
48+
const kill = () => {
49+
upstream.destroy();
50+
clientSock.destroy();
51+
};
52+
upstream.on('error', kill);
53+
clientSock.on('error', kill);
54+
});
55+
srv.listen(0, '127.0.0.1', () => {
56+
resolve({
57+
port: (srv.address() as net.AddressInfo).port,
58+
connects: () => connects,
59+
close: () => srv.close(),
60+
});
61+
});
62+
});
63+
}
64+
1465
/** An echo TCP server standing in for the broker's proxy loopback port. */
1566
function startEcho(): Promise<{ port: number; close: () => void }> {
1667
return new Promise((resolve) => {
@@ -232,6 +283,114 @@ describe('server frame codec (raw client)', () => {
232283
});
233284
});
234285

286+
describe('proxy selection', () => {
287+
const base = 'ws://broker.internal:8080';
288+
test('picks HTTP_PROXY for ws:// and honors NO_PROXY', () => {
289+
expect(proxyForTunnelUrl(base, { HTTP_PROXY: 'http://gw:3128' })).toBe('http://gw:3128');
290+
expect(proxyForTunnelUrl(base, { HTTP_PROXY: 'http://gw:3128', NO_PROXY: 'broker.internal' })).toBeUndefined();
291+
expect(proxyForTunnelUrl(base, { HTTP_PROXY: 'http://gw:3128', NO_PROXY: '*' })).toBeUndefined();
292+
});
293+
test('picks HTTPS_PROXY for wss:// and falls back to ALL_PROXY', () => {
294+
expect(proxyForTunnelUrl('wss://broker.example.com', { HTTPS_PROXY: 'http://gw:3128' })).toBe('http://gw:3128');
295+
expect(proxyForTunnelUrl(base, { ALL_PROXY: 'http://gw:3128' })).toBe('http://gw:3128');
296+
});
297+
test('ignores socks proxies and a missing proxy (direct dial)', () => {
298+
expect(proxyForTunnelUrl(base, {})).toBeUndefined();
299+
expect(proxyForTunnelUrl(base, { ALL_PROXY: 'socks5://gw:1080' })).toBeUndefined();
300+
});
301+
test('dot-suffix NO_PROXY matches subdomains', () => {
302+
expect(proxyForTunnelUrl('ws://a.corp.example:8080', { HTTP_PROXY: 'http://gw:3128', NO_PROXY: '.example' })).toBeUndefined();
303+
expect(proxyForTunnelUrl('ws://a.corp.example:8080', { HTTP_PROXY: 'http://gw:3128', NO_PROXY: 'other.test' })).toBe('http://gw:3128');
304+
});
305+
});
306+
307+
describe('tunnel through a CONNECT proxy', () => {
308+
test('fetches the bootstrap via the proxy when HTTP_PROXY is set', async () => {
309+
const echo = await startEcho();
310+
const broker = await startBroker(echo.port, { payloadJson: '{"env":{"VIA":"proxy"},"omittedKeys":[],"serializedGraph":{"config":{}}}', certs: { 'ca-cert.pem': 'PX' } });
311+
const proxy = await startConnectProxy();
312+
const restore = withEnvVars({ HTTP_PROXY: `http://127.0.0.1:${proxy.port}` });
313+
try {
314+
const boot = await fetchTunnelBootstrap(broker.url, TOKEN);
315+
expect(JSON.parse(boot.payloadJson).env.VIA).toBe('proxy');
316+
expect(boot.certs['ca-cert.pem']).toBe('PX');
317+
expect(proxy.connects()).toBeGreaterThan(0);
318+
} finally {
319+
restore();
320+
}
321+
proxy.close();
322+
broker.close();
323+
echo.close();
324+
});
325+
326+
test('bridges the data path through the proxy (large, both directions)', async () => {
327+
const echo = await startEcho();
328+
const broker = await startBroker(echo.port);
329+
const proxy = await startConnectProxy();
330+
const restore = withEnvVars({ HTTP_PROXY: `http://127.0.0.1:${proxy.port}` });
331+
let listener: Awaited<ReturnType<typeof startTunnelClientListener>> | undefined;
332+
try {
333+
listener = await startTunnelClientListener({ url: broker.url, token: TOKEN });
334+
const payload = randomBytes(100_000);
335+
const received = await new Promise<Buffer>((resolve, reject) => {
336+
const chunks: Array<Buffer> = [];
337+
let total = 0;
338+
const c = net.connect(listener!.port, '127.0.0.1', () => c.write(payload));
339+
c.on('data', (d: Buffer) => {
340+
chunks.push(d);
341+
total += d.length;
342+
if (total >= payload.length) {
343+
c.destroy();
344+
resolve(Buffer.concat(chunks));
345+
}
346+
});
347+
c.on('error', reject);
348+
setTimeout(() => reject(new Error(`timed out after ${total}`)), 5000);
349+
});
350+
expect(received.equals(payload)).toBe(true);
351+
expect(proxy.connects()).toBeGreaterThan(0);
352+
} finally {
353+
restore();
354+
}
355+
listener?.close();
356+
proxy.close();
357+
broker.close();
358+
echo.close();
359+
});
360+
361+
test('surfaces a bad token through the proxy as an error', async () => {
362+
const echo = await startEcho();
363+
const broker = await startBroker(echo.port);
364+
const proxy = await startConnectProxy();
365+
const restore = withEnvVars({ HTTP_PROXY: `http://127.0.0.1:${proxy.port}` });
366+
try {
367+
await expect(fetchTunnelBootstrap(broker.url, 'wrong-token', 3000)).rejects.toThrow();
368+
} finally {
369+
restore();
370+
}
371+
proxy.close();
372+
broker.close();
373+
echo.close();
374+
});
375+
376+
test('NO_PROXY makes a loopback broker dial direct (proxy unused)', async () => {
377+
const echo = await startEcho();
378+
const broker = await startBroker(echo.port);
379+
const proxy = await startConnectProxy();
380+
const restore = withEnvVars({ HTTP_PROXY: `http://127.0.0.1:${proxy.port}`, NO_PROXY: '127.0.0.1' });
381+
try {
382+
const boot = await fetchTunnelBootstrap(broker.url, TOKEN);
383+
expect(boot.certs['ca-cert.pem']).toBe('CA');
384+
expect(proxy.connects()).toBe(0);
385+
} finally {
386+
restore();
387+
}
388+
proxy.close();
389+
broker.close();
390+
echo.close();
391+
});
392+
});
393+
235394
describe('tunnel constants', () => {
236395
test('token header and path are stable', () => {
237396
expect(TUNNEL_TOKEN_HEADER).toBe('x-varlock-tunnel-token');

0 commit comments

Comments
 (0)