newton-cli: Agent-native CLI wrapper — drive Newton simulations with JSON

[Terry-cyx]

Hi Newton team and community 👋

I built newton-cli — a CLI wrapper that lets AI agents (Claude, GPT, etc.) drive Newton physics simulations through typed commands and JSON envelopes, without writing Python.

Why

LLM agents are good at generating structured data (JSON) but struggle with complex Python involving Warp arrays, CUDA device management, and Newton's builder pattern. newton-cli bridges that gap:

• Route A (declarative): Agent emits a recipe.json describing ModelBuilder calls → CLI builds the model, runs the solver, outputs state.npz + render.png. No Python generated. 25 Newton examples work this way.
• Route B (run-script): For examples needing per-step Python (custom @wp.kernel, torch policies, autograd), the agent writes a script and the CLI runs it in a subprocess with structured exit envelopes. 33 more examples work this way.

58 / 65 Newton examples driven end-to-end through the CLI.

Quick demo

# Build a scene from a JSON recipe
newton-cli model build --recipe pendulum.json --out model.json --json

# Simulate 100 frames with MuJoCo solver
newton-cli sim run --model model.json --solver SolverMuJoCo \
    --num-frames 100 --fps 60 --out final.npz --json

# Render the final state (headless OpenGL, same as Newton's viewer)
newton-cli viewer render --model model.json --state final.npz --out render.png --json

Every command outputs {"schema":"newton-cli/v1","data":...} with --json. Exit codes are deterministic (0/2/3/4/5).

Gallery

All renders produced by newton-cli viewer render:

Unitree G1 (MJCF)	Cloth (XPBD)	Granular (MPM)
40 cables (VBD)	Pyramid (contacts)	ANYmal (URDF)

Solvers covered

SolverXPBD, SolverVBD, SolverMuJoCo, SolverImplicitMPM, SolverStyle3D, SolverSemiImplicit — plus custom @wp.kernel via Route B.

Links

• Repo: https://github.com/Terry-cyx/newton-cli
• README in 4 languages: English, 简体中文, 日本語, 한국어
• License: MIT

Feedback, issues, and PRs welcome. Would love to hear from the Newton team if there are API patterns that could make the CLI integration smoother (e.g., Model serialization, headless viewer improvements).

Thanks for building such a great engine! 🙏