Skip to content

Commit f4122a6

Browse files
committed
Add VMVM provider and new test suites to README
1 parent 1f6adb8 commit f4122a6

1 file changed

Lines changed: 16 additions & 0 deletions

File tree

README.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ AI agents are the new power users. They don't read your UI — they parse your d
1919
| [Modal](https://modal.com) | Container | Supported |
2020
| [CodeSandbox](https://codesandbox.io) | Docker | Supported |
2121
| [Fly.io Machines](https://fly.io) | Firecracker | Supported |
22+
| VMVM | microVM (Meta internal) | Supported |
2223
| Docker Image | Local container | Supported |
2324
| MicroVM | Local microVM | Supported |
2425

@@ -33,6 +34,10 @@ sandbox-bench runs modular test suites inside a single sandbox lifecycle. Pick w
3334
| **swe** | SWE-bench-style: package install, git, pytest, network | `network_access`, `pip_install`, `git_clone`, `pytest` |
3435
| **environment** | Complex onramp: Node.js, npm, venv, multi-step builds | `nodejs`, `npm`, `project_clone`, `multi_step_build` |
3536
| **performance** | Agent spawn latency, warm start, file I/O throughput | `agent_spawn`, `warm_start`, `rapid_exec`, `file_io_10mb` |
37+
| **networking** | Protocol support: TCP, UDP, HTTPS, WebSocket, SSH, inbound | `tcp_outbound`, `udp_outbound`, `websocket`, `inbound_listen` |
38+
| **mcp** | MCP tool-use integration | `tool_discovery`, `tool_invocation` |
39+
| **training_batch** | Concurrent sandbox scaling (256–4096) | `tier1_256`, `tier2_1024`, `tier3_4096` |
40+
| **agentic_session** | Snapshot/restore for long-running agents | `snapshot`, `destroy_and_restore`, `verify_restore` |
3641
| **full** | All of the above ||
3742

3843
Default is `basic` for fast iteration. Use `--suite full` for comprehensive benchmarking.
@@ -119,6 +124,17 @@ Or use a `.env` file:
119124
sandbox-bench run --all --env-file .env
120125
```
121126

127+
### VMVM (Meta internal)
128+
129+
VMVM requires `vacli` on PATH, valid Meta TLS credentials, and a FaaS tenant ID. Set the tenant ID via environment variable:
130+
131+
```bash
132+
export VMVM_TENANT_ID="your-tenant-id"
133+
sandbox-bench run -p vmvm
134+
```
135+
136+
For multi-suite runs with different tenants per suite, copy `.env.meta.example` to `.env.meta` and fill in your values. The helper scripts (`run-vmvm-bench.sh`, `run-vmvm-tier1.sh`) source this file automatically.
137+
122138
## Adding a Provider
123139

124140
Implement the `SandboxProvider` interface:

0 commit comments

Comments
 (0)