Skip to content

Commit db5a1c9

Browse files
r33drichardsOpenClaw AgentclaudeOpenClaw
authored
Feat/openapi client cli with tests (#120)
* feat: add OpenAPI spec, mcp-v8-client crate, mcp-v8-cli binary, and release workflows ## Server — OpenAPI (utoipa) - Add utoipa, utoipa-axum, utoipa-swagger-ui to server/Cargo.toml - Annotate all api.rs structs with #[derive(ToSchema)] - Annotate all handlers with #[utoipa::path(...)] - Add ApiDoc struct with #[derive(OpenApi)] listing all paths + schemas - Add --print-openapi CLI flag: prints spec to stdout and exits - Add /swagger-ui and /api-doc/openapi.json routes on http/sse transports - Commit initial openapi.json (regenerate with: server --print-openapi > openapi.json) ## mcp-v8-client crate - New crate at mcp-v8-client/ with progenitor build.rs codegen - build.rs reads openapi.json and generates typed Client at compile time - src/lib.rs includes the generated code, re-exports Client + types - src/main.rs: mcp-v8-cli binary with clap subcommands: exec, executions list/get/output/cancel - --url flag (env: MCP_V8_URL, default: http://localhost:3000) - --json flag for raw JSON output ## Workspace - Root Cargo.toml workspace including server + mcp-v8-client - Nix build (server/ standalone) is unaffected ## GitHub Actions - release.yml: extended with build-server -> generate-openapi -> build-cli pipeline - CLI built for linux-x86_64, linux-arm64, macos-x86_64, macos-arm64 - CLI binaries gzipped and attached as release assets alongside server binary - openapi.json regenerated from built server binary on every release - mcp-v8-client published to crates.io on tag push (CARGO_REGISTRY_TOKEN secret) - publish-client.yml: new workflow mirroring ip-allocator publish-rust-sdk.yml ## install.sh - Now installs both mcp-v8 and mcp-v8-cli by default - MCP_V8_INSTALL=cli or MCP_V8_INSTALL=server to install one ## README - New sections: HTTP API & OpenAPI, mcp-v8-cli, mcp-v8-client crate - --print-openapi documented under Command Line Arguments - Updated Installation to mention mcp-v8-cli and crates.io * fix: restore install.sh unchanged; add install-cli.sh for CLI-only installs - Revert install.sh to its original state (server binary only, unchanged) - Add install-cli.sh: mirrors install.sh style but targets mcp-v8-cli release assets (mcp-v8-cli-linux, mcp-v8-cli-linux-arm64, mcp-v8-cli-macos, mcp-v8-cli-macos-arm64) - Update README Installation section to reference install-cli.sh - Update mcp-v8-client/README.md to reference install-cli.sh * test: add CLI e2e tests (bash + Rust) and GHA workflow - .github/workflows/cli-e2e.yml: builds server + CLI via nix develop, starts the server in stateless HTTP mode, runs the bash test script, then runs the Rust integration test suite (each test spawns its own server). - tests/cli-e2e.sh: bash script with 7 end-to-end tests covering: exec, executions list/get/output/cancel, poll-until-complete, and the --json flag. Env vars CLI_BIN and SERVER_URL are configurable. - mcp-v8-client/tests/cli_e2e.rs: Rust integration tests mirroring the same 7 test cases. Each test starts an independent server process on a free port (via CARGO_BIN_EXE_mcp-v8-cli + SERVER_BIN env var), runs CLI subcommands as child processes, and asserts exit codes and output. - mcp-v8-client/Cargo.toml: add reqwest (blocking) as a dev-dependency for server health-polling inside the Rust tests. * fix: add IntoParams derive to OutputQuery for utoipa compatibility * fix: add anyhow dep to mcp-v8-client; use fake hash to get correct Nix vendor hash * fix: drop utoipa-swagger-ui, serve openapi.json via plain axum route * fix: restore correct Nix hash; fix cli-e2e binary paths in workflow * fix: update binary paths and Nix hash for workspace layout The workspace Cargo.toml at the root causes cargo to output binaries to <workspace_root>/target/ instead of server/target/. Updated all references in Dockerfile, workflows, tests, and scripts. Regenerated server/Cargo.lock to include utoipa dependencies and updated the Nix vendor hash. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use console.log in CLI E2E test 5 so output is captured The executions output endpoint returns console output, not expression return values. Changed `2+2` to `console.log(2+2)` so the test can verify the output contains '4'. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: 4 bugs from PR review - ExecutionSummary: add #[serde(rename = "execution_id")] so list table shows IDs instead of '-' (field was named 'id', spec expected 'execution_id') - Move [patch.crates-io] deno_core fork from server/Cargo.toml to workspace root Cargo.toml; Cargo silently ignores [patch] in non-root members, so the heap-leak fix was not being applied in workspace builds - Remove unused openapi_router() fn and utoipa-axum dep from server; the server uses api_router() directly - Regenerate openapi.json from current server binary (utoipa 5 -> 4 downgrade restores OpenAPI 3.0.3 output compatible with progenitor 0.8); fix CLI arg heap_memory_max_mb: Option<i64> -> Option<u64> to match generated client type (progenitor emits u64 when schema has minimum: 0) * ci: add openapi-drift workflow to catch uncommitted spec changes Builds the server binary, regenerates openapi.json via --print-openapi, then fails with a helpful message if either openapi.json or mcp-v8-client/openapi.json differs from what's committed. * fix: update Nix vendor hash for utoipa 4 downgrade - Regenerate server/Cargo.lock resolving utoipa to 4.2.3 (was 5.4.0) The Nix build vendors ./server standalone at a fixed hash; utoipa 5 was not in the vendored store since the PR originally had utoipa 5 at commit time but we downgraded to 4 for progenitor 0.8 compat. - Set flake.nix cargoDeps hash to fake value (sha256-AAAA...) so the Nix build fails with the correct expected hash on first CI run. Update the hash once CI reports the correct value. - Add workspace Cargo.lock (tracks mcp-v8-client deps separately) * fix: update Nix vendor hash for utoipa 4 (sha256-L1IGe3...) * fix: update test_list_executions to use execution_id field The ExecutionSummary rename (id -> execution_id via serde rename) broke the existing server integration test that was still asserting on the old 'id' field name in the list response. * docs: add http-api-and-client.md covering CLI, Rust SDK, and code generation - Full CLI reference and walkthrough with real command output - Rust SDK examples (submit, poll, cancel, paginate output) verified compiling and running against a live server - Step-by-step guide for generating clients in TS, Python, Go, or any OpenAPI-supported language from openapi.json - Documents the openapi-drift CI workflow and how to resolve failures * feat: add /api/cli endpoints to serve CLI downloads for the running version GET /api/cli Returns a JSON index listing download URLs for all supported platforms. Version is taken from CARGO_PKG_VERSION, URLs point to the matching GitHub Release asset so the CLI always matches the server binary. GET /api/cli/{platform} (linux-x86_64 | linux-aarch64 | macos-x86_64 | macos-aarch64) HTTP 307 redirect to the CLI binary on GitHub Releases. Add ?gz=1 for the gzip-compressed variant. Also: - Fix release.yml asset naming: use explicit {bin_name} for CLI artifacts (was the confusing $bin-$dir_trimmed pattern, now mcp-v8-cli-linux-x86_64 etc.) - Regenerate openapi.json to include the new /api/cli paths * feat: embed CLI binaries in server binary for direct download Server now serves the CLI binary directly from memory — no redirect, no GitHub dependency at download time. How it works: - server/build.rs reads MCP_V8_CLI_{PLATFORM} env vars at compile time and copies the binaries into OUT_DIR; include_bytes! embeds them. - In dev/local builds (no env vars) placeholder empty files are written, and /api/cli/{platform} returns 404 with a helpful message. - The release workflow adds a 'rebuild-server' job that runs after 'build-cli', downloads the matching CLI artifact, and rebuilds each server binary with the env var set. New endpoints: GET /api/version — { "version": "0.1.0" } GET /api/cli — index with per-platform URLs + available flag GET /api/cli/{platform} — streams the binary (Content-Disposition: attachment) Usage from any host pointing at your server: curl -L http://my-server:3000/api/cli/linux-x86_64 -o mcp-v8-cli chmod +x mcp-v8-cli * test: add integration tests for /api/version and /api/cli endpoints 9 tests covering: - /api/version returns a non-empty semver string matching CARGO_PKG_VERSION - /api/cli index shape (version, assets array, required fields per asset) - /api/cli covers all 4 expected platforms - /api/cli version matches /api/version - /api/cli asset urls point back to this server's /api/cli/{platform} - /api/cli/{unknown} returns 404 with error listing valid platforms - /api/cli/{platform} returns 200 ELF binary (release build) or 404 with 'not embedded / set MCP_V8_CLI_*' message (dev build) - available flag in index is consistent with actual download status Tests pass in both dev builds (available=false, 404 path) and release builds with MCP_V8_CLI_LINUX_X86_64 set (available=true, 200 + ELF). * test: add E2E test that downloads CLI from API and runs console.log(1+1) Downloads mcp-v8-cli from /api/cli/linux-x86_64, writes it to a temp file, chmod +x, then: 1. Runs: mcp-v8-cli --json exec 'console.log(1+1)' 2. Polls: mcp-v8-cli --json executions get <id> until completed 3. Reads: mcp-v8-cli executions output <id> 4. Asserts output contains '2' The entire flow uses only the binary downloaded from the server — the same one that was embedded at build time. Skips gracefully with a clear message in dev builds where the binary is not embedded. * docs: update http-api-and-client.md with version, CLI download, and embedding guide - Add full API reference table including /api/version and /api/cli - Document /api/version and /api/cli endpoints with real output - Update CLI installation section: server-hosted download is now the recommended method (always matches the running version) - Add one-liner download+exec+output example (console.log(1+1) → 2) - Add 'How the CLI binary embedding works' section explaining the two-pass release build, dev build behaviour, and how to test the full flow locally with MCP_V8_CLI_LINUX_X86_64 --------- Co-authored-by: OpenClaw Agent <agent@openclaw.dev> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: OpenClaw <fixes@openclaw>
1 parent 3d12942 commit db5a1c9

31 files changed

Lines changed: 11235 additions & 727 deletions

.github/workflows/cli-e2e.yml

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
name: CLI E2E Tests
2+
3+
on:
4+
push:
5+
branches: [ main ]
6+
pull_request:
7+
branches: [ main ]
8+
9+
jobs:
10+
cli-e2e:
11+
name: CLI E2E
12+
runs-on: ubuntu-latest
13+
timeout-minutes: 30
14+
15+
steps:
16+
- name: Checkout code
17+
uses: actions/checkout@v4
18+
19+
- name: Install Nix
20+
uses: DeterminateSystems/nix-installer-action@main
21+
with:
22+
extra-conf: |
23+
sandbox = relaxed
24+
25+
- uses: DeterminateSystems/flake-checker-action@main
26+
27+
- name: Build server binary
28+
run: nix develop --command bash -c "cargo build --release -p server && cp target/release/server server-bin"
29+
30+
- name: Build CLI binary
31+
run: nix develop --command bash -c "cargo build --release -p mcp-v8-client --bin mcp-v8-cli && cp target/release/mcp-v8-cli cli-bin"
32+
33+
- name: Run bash CLI E2E tests
34+
run: |
35+
# Start the server in HTTP stateless mode
36+
./server-bin --stateless --http-port 3000 &
37+
SERVER_PID=$!
38+
39+
cleanup() { kill $SERVER_PID 2>/dev/null || true; }
40+
trap cleanup EXIT
41+
42+
# Wait for server to be ready (up to 30s)
43+
echo "Waiting for server to be ready..."
44+
for i in $(seq 1 30); do
45+
HTTP_CODE=$(curl -s --max-time 5 -o /dev/null -w '%{http_code}' \
46+
http://localhost:3000/api/exec \
47+
-H 'Content-Type: application/json' \
48+
-d '{"code": "1+1"}' 2>/dev/null || echo "000")
49+
if [ "$HTTP_CODE" = "202" ] || [ "$HTTP_CODE" = "200" ]; then
50+
echo "Server is ready (HTTP $HTTP_CODE)"
51+
break
52+
fi
53+
echo " attempt $i: HTTP $HTTP_CODE"
54+
sleep 1
55+
done
56+
57+
if [ "$HTTP_CODE" != "202" ] && [ "$HTTP_CODE" != "200" ]; then
58+
echo "ERROR: Server did not become ready after 30 attempts"
59+
exit 1
60+
fi
61+
62+
# Run the CLI e2e test script
63+
CLI_BIN=./cli-bin \
64+
SERVER_URL=http://localhost:3000 \
65+
bash tests/cli-e2e.sh
66+
67+
- name: Run Rust CLI integration tests
68+
# Each test starts its own server, so no external server is needed here.
69+
# SERVER_BIN points to the pre-built release binary built above.
70+
run: |
71+
nix develop --command bash -c "
72+
SERVER_BIN=\$(pwd)/server-bin \
73+
cargo test --test cli_e2e -p mcp-v8-client -- --test-threads=4 --nocapture
74+
"

.github/workflows/load-test.yml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -59,11 +59,11 @@ jobs:
5959
6060
# ── Build ───────────────────────────────────────────────────────
6161
- name: Build MCP-V8 binary
62-
run: nix develop --command bash -c "cd server && cargo build --release"
62+
run: nix develop --command bash -c "cargo build --release -p server"
6363

6464
- name: Verify binary
6565
run: |
66-
ls -lh server/target/release/server
66+
ls -lh target/release/server
6767
6868
# ── Prepare data directories ────────────────────────────────────
6969
- name: Create data directories
@@ -76,7 +76,7 @@ jobs:
7676
# ── Start server(s) ─────────────────────────────────────────────
7777
- name: Start ${{ env.LABEL }} server(s)
7878
run: |
79-
BINARY="$(pwd)/server/target/release/server"
79+
BINARY="$(pwd)/target/release/server"
8080
8181
if [ "${{ matrix.topology }}" = "single" ]; then
8282
# ── Single node ─────────────────────────────────
Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
name: OpenAPI Drift Check
2+
3+
on:
4+
push:
5+
branches: [ main ]
6+
pull_request:
7+
branches: [ main ]
8+
9+
jobs:
10+
openapi-drift:
11+
name: Check openapi.json is up to date
12+
runs-on: ubuntu-latest
13+
timeout-minutes: 20
14+
15+
steps:
16+
- name: Checkout code
17+
uses: actions/checkout@v4
18+
19+
- name: Install Nix
20+
uses: DeterminateSystems/nix-installer-action@main
21+
with:
22+
extra-conf: |
23+
sandbox = relaxed
24+
25+
- uses: DeterminateSystems/flake-checker-action@main
26+
27+
- name: Build server binary
28+
run: nix develop --command bash -c "cargo build --release -p server"
29+
30+
- name: Regenerate openapi.json
31+
run: |
32+
./target/release/server --print-openapi > openapi.json
33+
cp openapi.json mcp-v8-client/openapi.json
34+
35+
- name: Check for drift
36+
run: |
37+
if ! git diff --exit-code openapi.json mcp-v8-client/openapi.json; then
38+
echo ""
39+
echo "❌ openapi.json is out of date."
40+
echo ""
41+
echo "The committed openapi.json does not match what the server binary generates."
42+
echo "Regenerate it with:"
43+
echo ""
44+
echo " cargo build --release -p server"
45+
echo " ./target/release/server --print-openapi > openapi.json"
46+
echo " cp openapi.json mcp-v8-client/openapi.json"
47+
echo ""
48+
exit 1
49+
fi
50+
echo "✅ openapi.json is up to date."
Lines changed: 130 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,130 @@
1+
name: Publish mcp-v8-client
2+
3+
# Mirrors the pattern from ip-allocator-webserver/publish-rust-sdk.yml.
4+
#
5+
# On every push to main that touches the server API or client crate:
6+
# - Verifies the client builds cleanly against the committed openapi.json
7+
# - Publishes to crates.io (continues on error — handles "already published")
8+
#
9+
# On pull_requests: builds and comments with install instructions.
10+
# On workflow_dispatch: optionally publish with a custom version.
11+
12+
on:
13+
push:
14+
branches:
15+
- main
16+
paths:
17+
- 'server/src/api.rs'
18+
- 'mcp-v8-client/**'
19+
- 'openapi.json'
20+
pull_request:
21+
branches:
22+
- main
23+
paths:
24+
- 'server/src/api.rs'
25+
- 'mcp-v8-client/**'
26+
- 'openapi.json'
27+
workflow_dispatch:
28+
inputs:
29+
publish:
30+
description: 'Publish to crates.io (yes/no)'
31+
required: false
32+
default: 'no'
33+
34+
env:
35+
PACKAGE_NAME: "mcp-v8-client"
36+
CARGO_TERM_COLOR: always
37+
38+
jobs:
39+
build-and-publish:
40+
runs-on: ubuntu-latest
41+
permissions:
42+
contents: read
43+
pull-requests: write
44+
45+
steps:
46+
- name: Checkout code
47+
uses: actions/checkout@v4
48+
49+
- name: Install Rust stable
50+
uses: dtolnay/rust-toolchain@stable
51+
52+
- uses: Swatinem/rust-cache@v2
53+
with:
54+
workspaces: "mcp-v8-client -> target"
55+
56+
- name: Determine version
57+
id: version
58+
run: |
59+
BASE_VERSION=$(grep -E '^version = ' mcp-v8-client/Cargo.toml | head -1 | sed 's/version = "\(.*\)"/\1/')
60+
if [ "${{ github.event_name }}" == "pull_request" ]; then
61+
PR_NUMBER="${{ github.event.pull_request.number }}"
62+
SHORT_SHA=$(echo "${{ github.sha }}" | cut -c1-7)
63+
VERSION="${BASE_VERSION}-beta.${PR_NUMBER}.${SHORT_SHA}"
64+
else
65+
VERSION="$BASE_VERSION"
66+
fi
67+
echo "version=$VERSION" >> $GITHUB_OUTPUT
68+
echo "📦 Package version: $VERSION"
69+
70+
- name: Build client
71+
run: |
72+
cd mcp-v8-client
73+
cargo build --release --verbose
74+
75+
- name: Verify package
76+
run: |
77+
cd mcp-v8-client
78+
cargo package --list
79+
80+
- name: Publish to crates.io (main branch)
81+
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
82+
run: |
83+
cd mcp-v8-client
84+
cargo publish --token ${{ secrets.CARGO_REGISTRY_TOKEN }}
85+
continue-on-error: true
86+
87+
- name: Publish to crates.io (manual)
88+
if: github.event_name == 'workflow_dispatch' && github.event.inputs.publish == 'yes'
89+
run: |
90+
cd mcp-v8-client
91+
cargo publish --token ${{ secrets.CARGO_REGISTRY_TOKEN }}
92+
93+
- name: Comment on PR
94+
if: github.event_name == 'pull_request'
95+
uses: actions/github-script@v7
96+
with:
97+
script: |
98+
const version = '${{ steps.version.outputs.version }}';
99+
const packageName = '${{ env.PACKAGE_NAME }}';
100+
const comment = `## 🦀 mcp-v8-client Built Successfully
101+
102+
**Package:** \`${packageName}\`
103+
**Version:** \`${version}\`
104+
105+
### Add as dependency
106+
107+
\`\`\`toml
108+
[dependencies]
109+
${packageName} = { git = "https://github.com/${{ github.repository }}", branch = "${{ github.head_ref }}" }
110+
\`\`\`
111+
112+
### CLI (from this branch)
113+
114+
\`\`\`bash
115+
cargo install --git https://github.com/${{ github.repository }} --branch ${{ github.head_ref }} mcp-v8-client
116+
\`\`\`
117+
`;
118+
github.rest.issues.createComment({
119+
issue_number: context.issue.number,
120+
owner: context.repo.owner,
121+
repo: context.repo.repo,
122+
body: comment
123+
});
124+
125+
- name: Upload artifact
126+
uses: actions/upload-artifact@v4
127+
with:
128+
name: mcp-v8-client-${{ steps.version.outputs.version }}
129+
path: mcp-v8-client/
130+
retention-days: 30

0 commit comments

Comments
 (0)