feat(simulator): 3D robotics simulator playground for R1-R10 by SeventhOdyssey71 · Pull Request #163 · MystenLabs/sui-move-bootcamp

SeventhOdyssey71 · 2026-03-17T09:06:22Z

Summary

Adds a hardware-free 3D robot simulator so learners can complete the entire R1-R10 robotics track without a physical Petoi Bittle X robot. Every robot action is a real Sui testnet transaction signed by the user's wallet.

Simulator core

Next.js playground at simulator/ — Three.js virtual Unitree GO1 with animated poses and walk cycles
TCP serial bridge on port 8375 — R modules connect via a zero-dependency VirtualSerialPort adapter
Simulation entry points — src/sim.ts files and pnpm sim scripts added to R1, R4, R5, and R7
Module browser — renders each module's README with architecture diagrams and simulator instructions
Sui.io design system — dark navbar, dotted grid background, TWK Everett font, clean flat cards

On-chain actions (wallet-gated)

The robot requires a connected Sui wallet. Without one, the playground shows a connect prompt. With a wallet connected, every action follows this flow:

User clicks a button or presses a keyboard shortcut
A PTB calling robot_queue::add_action is built and sent to the wallet for signing
The wallet prompts the user to approve the transaction
Once the transaction confirms on Sui testnet, the robot animates
A success toast appears with the action name

If the transaction fails or is rejected, the robot does not move and an error toast is shown.

Deployed contract (testnet)

Uses the R2 action_queue::robot_queue contract:

	Address
Package	`0x27a3292a055a7904753a8c741579d9cdebc17010c8b65d3d1f00da43047962b7`
ActionQueue	`0x83ba18609f73b99518b7aaa13ce4a17293c4d18c4e2bab38ce59c7dc0fef355c`

Wallet integration

Custom connect wallet modal (full-view centered dialog via React portal, z-index 99999)
Wallet list with icons, connection states, error handling, Escape to close
Connected state shows address + disconnect button in navbar
On-chain status cards: chain, queue length, total on-chain actions, pending txs
Terminal shows tx signing status and digests
Success/error toasts on transaction confirmation

What the simulator provides

Interface	Address	Purpose
Web UI	`http://localhost:3000`	3D viewport, command buttons, module browser
TCP Serial	`localhost:8375`	Drop-in serial port replacement for R modules
WebSocket	`ws://localhost:3000/ws`	Real-time robot state to browser
REST API	`/api/state`, `/api/command`, `/api/modules`	Scripting and testing

How modules connect

# Terminal 1: start the simulator
cd simulator && npm install && npm run dev

# Terminal 2: run any hardware module
cd R1/hello-bittle && pnpm sim       # serial demo
cd R4/processor   && pnpm sim       # blockchain processor
cd R5/server      && pnpm sim       # WebSocket control
cd R7/part-a-offchain && pnpm sim   # Ed25519 auth

Files added/changed

simulator/ — Next.js app, Three.js robot, custom server, Tailwind CSS
simulator/src/hooks/useOnChainAction.ts — sign-then-animate hook with toast notifications
simulator/src/hooks/useOnChainQueue.ts — polls ActionQueue shared object every 5s
simulator/src/components/NavbarWalletControls.tsx — custom connect modal via React portal
simulator/src/components/Toast.tsx — success/error toast system
simulator/src/lib/sui-dapp-kit.ts — dapp-kit config + deployed contract addresses
simulator/.env.example — testnet contract addresses
simulator/lib/virtual-serial.ts — shared TCP adapter (64 lines, zero deps)
R1, R4, R5, R7 — added src/sim.ts entry points and pnpm sim scripts
No changes to existing module source code, Move contracts, or solution branches

Test plan

Add a hardware-free simulator so learners can complete the entire R1-R10 robotics track without a physical Petoi Bittle X robot. Simulator (simulator/): - Next.js 14 app with Three.js 3D robot viewport - Virtual Bittle state machine (17 commands + 30 aliases) - TCP serial bridge on port 8375 for R module compatibility - WebSocket server for real-time UI state broadcasting - Module browser with architecture diagrams and rendered lesson content - Tailwind CSS dark-themed dashboard Simulation adapters (R1, R4, R5, R7): - VirtualSerialPort TCP adapter (zero external deps, uses net.Socket) - src/sim.ts entry points for each hardware-dependent module - pnpm sim scripts added to package.json files No changes to existing module source code or Move contracts.

Wire every robot action button to submit real Sui testnet transactions via a dual-path pattern: WebSocket fires immediately for instant animation, while a PTB is signed and submitted on-chain in the background. Graceful fallback when wallet is disconnected or contract env vars are not set. - Add useOnChainAction hook (dual-path: WebSocket + on-chain PTB) - Add useOnChainQueue hook (polls ActionQueue shared object every 5s) - Add SUI_CONTRACT config with env var support (NEXT_PUBLIC_SUI_*) - Replace dapp-kit ConnectButton web component with custom full-view connect modal (centered dialog, wallet list with icons, error states) - Add disconnect button and connected address display in navbar - Add on-chain status cards (chain, queue, total, pending txs) - Remove SuiDropletIcon, "Powered by Sui" text, fake branding - Add ChainIcon for on-chain status section - Make on-chain hooks SSR-safe (check DAppKitContext before calling)

Use createPortal to render the connect modal directly into document.body so it renders above all page content, wallet extension overlays, and stacking contexts. Adds backdrop-blur-md with 50% opacity, body scroll lock, and Escape key to dismiss.

Deploy the robot_queue contract and create a shared ActionQueue object on Sui testnet. Hardcode deployed addresses as defaults so on-chain mode works out of the box when a wallet is connected. Contract: action_queue::robot_queue Package ID: 0x27a3292a055a7904753a8c741579d9cdebc17010c8b65d3d1f00da43047962b7 Queue ID: 0x83ba18609f73b99518b7aaa13ce4a17293c4d18c4e2bab38ce59c7dc0fef355c Network: testnet Every button click and keyboard shortcut now triggers a real on-chain add_action transaction when a Sui wallet is connected. The wallet prompts the user to sign each transaction.

Add on-chain actions section to README covering the dual-path pattern, deployed testnet contract addresses, wallet integration flow, and configuration. Update tech stack to include Sui dependencies.

Flip the action flow so the wallet signs and the transaction confirms on-chain before the robot moves. If no wallet is connected, the robot animates immediately as before (local-only fallback). - useOnChainAction: sign → confirm → sendAction (was: sendAction → sign) - Add Toast component (portal-rendered, auto-dismiss after 4s) - Show success toast with action name on tx confirmation - Show error toast on tx failure or user rejection - Fix hydration mismatch in SuiWalletProvider (useEffect for client init) - Remove stale dapp-kit web component CSS variables

Apply an immediate y-position correction whenever any foot drops below y=0 instead of gradually lerping toward the corrected position. The robot now stays firmly on the surface during all animations.

…ghts Restyle the entire simulator UI to match Sui.io's design language: - Dark navbar (#1a1a1a) with white text, blue connect wallet button - White backgrounds with subtle dotted grid pattern - Clean flat cards with gray-200 borders instead of rounded blue panels - Blue accent squares on eyebrow labels (Sui.io pattern) - Two-column feature grid, dark stats bar, full-width footer - TWK Everett as primary font (Inter fallback) - All font weights reduced to font-medium (no bold/semibold anywhere) - Lesson grid changed from 5-col to 3-col for better readability - Full-width landing page layout (no max-width constraints) - Remove water droplet icon from navbar and footer - Consistent gray-50/100/200/400/500 color scale across all components

The playground now requires a connected Sui wallet before showing the robot viewport, control panel, and terminal. Without a wallet, users see a centered prompt directing them to connect. This ensures every robot interaction is a real on-chain transaction.

Add professional robotics simulator features inspired by Webots: - DOF indicators at each joint: colored torus rings showing rotation plane, axis arrows, and dynamic arcs showing current angle - Joint colors: orange for hips, cyan for knees, red for body pitch, blue for body roll - World coordinate frame: RGB XYZ arrows with letter labels at origin - Enhanced grid: colored axis lines (red=X, blue=Z) on the ground plane - Real-time joint angles panel: overlay showing all 8 joint angles plus body pitch/roll, updated at 10Hz - DOF toggle button in toolbar with active state highlighting - All DOF geometry uses pre-allocated buffers updated per frame (no allocations in render loop, ~31 extra draw calls)

Add unitree-go1.ts configuration module extracted from the official unitree_ros SDK (github.com/unitreerobotics/unitree_ros): - All 12 DOF joint definitions (hip/thigh/calf × 4 legs) with URDF names (FR/FL/RR/RL_hip_joint, _thigh_joint, _calf_joint) - Joint limits from const.xacro: hip ±49.4°, thigh -39.3° to 257.8°, calf -161.5° to -50.9° - Link dimensions: thigh/calf 0.213m, trunk 0.376m × 0.094m × 0.114m - Hip origins, PID gains, actuator effort/velocity limits - Camera and ultrasound sensor positions - Simulator-to-URDF joint name mapping Update DOF panel to show: - URDF joint names (FL_thigh, FR_calf, etc.) - Per-joint limit progress bars - Kinematics section (thigh/calf length, mass, DOF count) - Source attribution: unitree_ros / go1_description

Implement playground capabilities matching every module in the R1-R10 robotics track: R2/R4 — Command history with tx digests and Suiscan links - Action event bus (action-events.ts) connects on-chain hook to UI - useCommandHistory hook tracks last 20 txns with status - Recent txns panel in ControlPanel with explorer links R5 — WebSocket latency measurement - Server ping/pong handler for RTT measurement - useSimulator exposes latencyMs (3s interval) - Viewport header shows colored latency indicator (green/yellow/red) R9 — Multiplayer indicator - Server broadcasts client count on connect/disconnect - useSimulator exposes connectedClients count - ControlPanel shows Multiplayer section with client count - Viewport header shows connected client count R10 — Simulated rental session tracker - useSessionTracker hook simulates 10-minute rental window - Tracks sequence number, commands used, estimated cost - SessionPanel shows countdown, rate, session ID - Starts on first command after wallet connect Infrastructure: - action-events.ts typed pub/sub for tx lifecycle events - useOnChainAction emits confirmed/failed events with digests - All new sections use collapsible Section component pattern

Add a lesson tab bar to the playground so users can select which R1-R10 module they're working through while using the simulator: - Top tab bar with Free Play + R1-R10 module buttons - Context bar shows active module phase, title, concepts, and link to the full lesson page - Lesson description card with summary, focus areas, time estimate, artifact, and sim run command - Terminal and viewport are always visible when wallet connected - Free Play mode hides the lesson context (just the robot) - Connect prompt shows module preview badges

Deploy a real COOKIE token contract to Sui testnet with: - 1,000,000,000 (1B) max supply, 0 decimals - Public faucet: anyone can mint, 100 per request, 1000 per day - Rate-limited via per-user daily tracking (Table<address, MintRecord>) - Burn function for token destruction - Events: FaucetCreated, TokensMinted Contract: cookie_token::cookie Package: 0x1a0761c44b99b65d9d4220d7be34c9042954126699c4f4ce7339d9ac90a11821 Faucet: 0xcdefdd53f71d25b76020aa5420dfc4950228f3c8f87ecc2f38eff19444d405f0 Playground integration: - useCookieToken hook: reads balance (listCoins), mints via wallet signing - COOKIE section in ControlPanel: shows balance, supply, mint button - Balance polls every 10 seconds when wallet connected - Mint 100 COOKIE button with loading state and success/error toasts

Add 5 foundational robotics simulation modules: 1. Rapier3D physics engine (WASM) - rapier-world.ts: async init, gravity world, ground plane, debug render - addEnvironmentMesh: load GLB trimesh as static colliders - next.config.mjs: enable asyncWebAssembly for Rapier WASM 2. Forward kinematics - legFK: compute foot position from 3 joint angles per leg - allFeetFK: batch compute all 4 feet from 12-DOF angles - Uses real GO1 dimensions (thigh/calf 0.213m) 3. Analytical inverse kinematics - legIK: solve 3-DOF leg IK (hip roll + thigh + calf) - Same algorithm as unitree_legged_sdk footPositionInHipFrame2JointAngle - Closed-form solution using law of cosines - Joint limit clamping from URDF 4. Unitree SDK controller interface (TypeScript) - LowCmd/LowState types matching unitree_legged_sdk/comm.h - JointIndex enum (FR_0..RL_2, 12 joints) - MotorCmd: position mode with PD gains - UnitreeController class with software PD simulation - IMU state, foot force, motor temperature 5. Robot camera simulation - 5 PerspectiveCamera objects at URDF positions (face, chin, L, R, rear) - WebGLRenderTarget for off-screen rendering (160x120) - Round-robin rendering (1 camera/frame = 12fps each) - copyToCanvas for PiP display with vertical flip

…olbar Wire the 5 core simulation modules into the viewport UI: Robot camera simulation (5 POV cameras): - PiPCameraPanel: 5 live camera feeds (face, chin, left, right, rear) rendered at 160x120, round-robin 1 per frame (~12fps each) - Cam toggle button in toolbar with active state highlighting - Cameras parented to BodyBone, positioned from URDF data - Uses WebGLRenderTarget for off-screen rendering Environment loading: - EnvironmentLoader: drag-and-drop or file picker for GLB/GLTF files - Loaded meshes added to scene with shadow casting/receiving - Clear button to remove loaded environment - Positioned bottom-left of viewport Toolbar expansion: - DOF button: toggle joint visualization + angles panel - Cam button: toggle 5 robot POV camera feeds - Reset camera button - Grid toggle button

raphaelchiaml · 2026-04-15T11:16:53Z

+            </div>
+
+            {module.readme ? (
+              <div className="prose-light" dangerouslySetInnerHTML={{ __html: simpleMarkdown(module.readme) }} />


Can you help me verify and list down the risk of using simplemarkdown function inside dangerouslysetInnerHtml?

Where are the possible data sources?
What are the possible attack angles?
Does the escaping inside the simpleMarkdown function escape html/javascripts?
Is there a better library to purify/hadnle xss vulnerabilities?

raphaelchiaml · 2026-04-15T14:10:16Z

+  // Start listening
+  // -------------------------------------------------------------------------
+
+  server.listen(HTTP_PORT, () => {


Can we bind the custom servers explicitly to 127.0.0.1 instead of relying on default host behavior?

Why this matters:

server.listen(port) may bind to all interfaces (0.0.0.0 / ::) depending on runtime/system, which can expose the dev server to the local network.

127.0.0.1 guarantees loopback-only access (same machine), matching our “local-only simulator” threat model.

SeventhOdyssey71 added 16 commits March 17, 2026 10:03

docs(simulator): document on-chain robot actions and wallet integration

b2d18e8

Add on-chain actions section to README covering the dual-path pattern, deployed testnet contract addresses, wallet integration flow, and configuration. Update tech stack to include Sui dependencies.

fix(simulator): prevent robot legs from clipping below the ground plane

5f44d87

Apply an immediate y-position correction whenever any foot drops below y=0 instead of gradually lerping toward the corrected position. The robot now stays firmly on the surface during all animations.

raphaelchiaml self-assigned this Mar 27, 2026

raphaelchiaml reviewed Apr 23, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(simulator): 3D robotics simulator playground for R1-R10#163

feat(simulator): 3D robotics simulator playground for R1-R10#163
SeventhOdyssey71 wants to merge 16 commits into
MystenLabs:mainfrom
SeventhOdyssey71:feature/robotics-simulator-playground

SeventhOdyssey71 commented Mar 17, 2026 •

edited

Loading

Uh oh!

raphaelchiaml Apr 15, 2026

Uh oh!

raphaelchiaml Apr 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

SeventhOdyssey71 commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Simulator core

On-chain actions (wallet-gated)

Deployed contract (testnet)

Wallet integration

What the simulator provides

How modules connect

Files added/changed

Test plan

Uh oh!

raphaelchiaml Apr 15, 2026

Choose a reason for hiding this comment

Uh oh!

raphaelchiaml Apr 15, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

SeventhOdyssey71 commented Mar 17, 2026 •

edited

Loading