-
Notifications
You must be signed in to change notification settings - Fork 0
refactor: Switch to cargo-xtask for building the project #77
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 1 commit
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| [alias] | ||
| xtask = "run --package xtask --" |
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,3 @@ | ||
| [workspace] | ||
| resolver = "3" | ||
| members = ["vmm", "cli", "backend", "agent", "backend/virt"] | ||
| members = ["vmm", "cli", "backend", "agent", "backend/virt", "xtask"] |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -5,61 +5,101 @@ Simple end-to-end startup for local development. | |
| ## 0) Prerequisites (one-time) | ||
|
|
||
| - Linux OS | ||
| - Rust toolchain with Musl (rustup target add x86_64-unknown-linux-musl) | ||
| - Rust toolchain with Musl: `rustup target add x86_64-unknown-linux-musl` | ||
| - nftables (to setup network) | ||
| - docker (for initramfs generation) | ||
| - Build tools (curl, tar, make, gcc) | ||
|
|
||
| You also need a VM kernel file at `backend/vmlinux`. | ||
| You need a VM kernel file at `backend/vmlinux`. You must build it separately (instructions to come) | ||
|
|
||
| ## 1) Build everything (one command) | ||
| ## 1) Build everything | ||
|
|
||
| From repository root: | ||
|
|
||
| ```bash | ||
| cargo build -p backend -p cli && \ | ||
| cargo build -p agent --target x86_64-unknown-linux-musl && \ | ||
| cp ./target/x86_64-unknown-linux-musl/debug/agent ./backend/cloude-agentd && \ | ||
| chmod +x ./backend/cloude-agentd | ||
| cargo xtask build | ||
| ``` | ||
|
|
||
| ## 2) Start backend (one command, terminal A) | ||
| This will: | ||
| - Build backend and CLI binaries | ||
| - Build agent with musl target | ||
| - Copy agent binary to `backend/cloude-agentd` | ||
|
|
||
| ## 2) Start backend (terminal A) | ||
|
|
||
| From repository root: | ||
|
|
||
| ```bash | ||
| cd backend && \ | ||
| PATH="/usr/sbin:$PATH" \ | ||
| LANGUAGES_CONFIG_PATH=./config/languages.json \ | ||
| AGENT_BINARY_PATH=./cloude-agentd \ | ||
| INIT_SCRIPT_PATH=./init.sh \ | ||
| VM_KERNEL_PATH=./vmlinux \ | ||
| VM_INITRAMFS_DIR=./tmp \ | ||
| VM_LOG_GUEST_CONSOLE=false \ | ||
| ../target/debug/backend | ||
| cargo xtask run-backend | ||
| ``` | ||
|
Comment on lines
+28
to
34
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Make backend start location explicit and consistent. There’s a mismatch between “From repository root” earlier and “from its project folder” on Line 30 while Line 33 uses Suggested doc patch-## 2) Start backend (terminal A)
-
-Run backend as root from its project folder:
+## 2) Start backend (terminal A)
+
+From repository root, run:
```bash
cargo xtask run-backendNote: The backend requires root privileges (sudo) to setup network interfaces and run VMs. The Verify each finding against the current code and only fix it if needed. In |
||
|
|
||
| Tip: set `VM_LOG_GUEST_CONSOLE=true` only when debugging VM boot/agent startup. | ||
| **Note:** The backend requires root privileges (sudo) to setup network interfaces and run VMs. The `run-backend` command will automatically use `sudo` to start the backend. | ||
|
|
||
| Tip: Set `VM_LOG_GUEST_CONSOLE=true` in `backend/.env` only when debugging VM boot/agent startup. | ||
|
|
||
| ## 3) Run code with CLI (terminal B) | ||
|
|
||
| Submit a job: | ||
| ### Run sample Python code | ||
|
|
||
| Submit a Python job using the example file: | ||
|
|
||
| ```bash | ||
| cargo run -p cli -- go --language python --file agent/examples/hello.py | ||
| cargo xtask run-cli -- go --language python --file agent/examples/hello.py | ||
| ``` | ||
|
|
||
| Then check status/result: | ||
| This will: | ||
| 1. Upload the Python code to the backend | ||
| 2. Backend creates a VM with Python runtime | ||
| 3. Agent executes the code inside the VM | ||
| 4. Returns the job ID (e.g., `job-abc123`) | ||
|
|
||
| ### Check job status | ||
|
|
||
| Use the job ID from the previous command: | ||
|
|
||
| ```bash | ||
| cargo run -p cli -- status <JOB_ID> | ||
| cargo xtask run-cli -- status job-abc123 | ||
| ``` | ||
|
|
||
| ## 4) Optional reset if initramfs cache is stale | ||
| ### Create and run your own Python script | ||
|
|
||
| 1. Create a Python file (e.g., `my_script.py`): | ||
|
|
||
| ```bash | ||
| cat > my_script.py << 'EOF' | ||
| # Simple Python script | ||
| import random | ||
|
|
||
| print("Generating 5 random numbers between 1 and 100:") | ||
| for _ in range(5): | ||
| print(random.randint(1, 100)) | ||
| EOF | ||
| ``` | ||
|
|
||
| 2. Submit the job: | ||
|
|
||
| ```bash | ||
| cargo xtask run-cli -- go --language python --file my_script.py | ||
| ``` | ||
|
|
||
| ### Run other languages | ||
|
|
||
| The system supports multiple languages. Example with Node.js: | ||
|
|
||
| ```bash | ||
| cargo xtask run-cli -- go --language node --file agent/examples/hello.js | ||
| ``` | ||
|
|
||
| Check the `backend/config/languages.json` file for available languages and versions. | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| ### Reset stale initramfs cache | ||
|
|
||
| If agent changes are not reflected inside VM: | ||
|
|
||
| ```bash | ||
| rm -f backend/tmp/*.cpio.gz | ||
| cargo xtask reset-initramfs | ||
| ``` | ||
|
|
||
| Then restart backend (step 2). | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -38,10 +38,7 @@ sudo apt install -y nftables | |
| From repository root: | ||
|
|
||
| ```bash | ||
| rustup target add x86_64-unknown-linux-musl | ||
| cargo build -p backend -p agent --target x86_64-unknown-linux-musl | ||
| cp ./target/x86_64-unknown-linux-musl/debug/agent ./backend/cloude-agentd | ||
| chmod +x ./backend/cloude-agentd | ||
| cargo xtask build | ||
| ``` | ||
|
|
||
| Why musl: runtime initramfs images are Alpine-based, so a glibc-linked agent | ||
|
|
@@ -53,18 +50,11 @@ The musl build is static and runs correctly in Alpine initramfs. | |
| From repository root: | ||
|
|
||
| ```bash | ||
| cd backend | ||
| sudo env \ | ||
| PATH="/usr/sbin:$PATH" \ | ||
| LANGUAGES_CONFIG_PATH=./config/languages.json \ | ||
| AGENT_BINARY_PATH=./cloude-agentd \ | ||
| INIT_SCRIPT_PATH=./init.sh \ | ||
| VM_KERNEL_PATH=./vmlinux \ | ||
| VM_INITRAMFS_DIR=./tmp \ | ||
| VM_LOG_GUEST_CONSOLE=false \ | ||
| ../target/debug/backend | ||
| cargo xtask run-backend | ||
| ``` | ||
|
|
||
| To enable verbose VM guest console logging, add `VM_LOG_GUEST_CONSOLE=true` to the `backend/.env` file. | ||
|
|
||
| Expected log: | ||
|
|
||
| ```text | ||
|
|
@@ -73,6 +63,8 @@ INFO backend: Starting Backend server on 127.0.0.1:8080 | |
|
|
||
| ## Environment variables | ||
|
|
||
| Configuration is read from `backend/.env` (create from `.env.exemple` if needed). | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Maybe move that line above |
||
|
|
||
| ### Required in practice | ||
|
|
||
| - `VM_KERNEL_PATH` (default `./vmlinux`): Linux kernel used to boot each VM. | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| [package] | ||
| name = "xtask" | ||
| version = "0.1.0" | ||
| edition = "2021" | ||
|
|
||
| [dependencies] | ||
| clap = { version = "4", features = ["derive"] } | ||
| dotenvy = "0.15" |
Uh oh!
There was an error while loading. Please reload this page.