Skip to content

Commit f84b7de

Browse files
committed
refactor: gate the AI assistant behind Experimental settings, not a feature flag
1 parent f96249c commit f84b7de

10 files changed

Lines changed: 109 additions & 7 deletions

File tree

ONBOARDING.md

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
# Creator Hub — Onboarding
2+
3+
Creator Hub is Decentraland's desktop app (Electron) for building SDK7 scenes. It's an npm-workspaces monorepo:
4+
5+
- **`@dcl/asset-packs`** — curated 3D assets + Smart Items.
6+
- **`@dcl/inspector`** — the web-based 3D scene editor (Babylon.js + Redux/Redux-Saga), embedded as an iframe.
7+
- **`creator-hub`** — the Electron app (main / preload / renderer) that wraps the inspector.
8+
9+
Deps flow `asset-packs → inspector → creator-hub` (each linked via `file:`).
10+
11+
## Get set up
12+
13+
```bash
14+
make init # clean, install, protoc, build all
15+
make build # build all packages (asset-packs → inspector → creator-hub)
16+
cd packages/creator-hub && npm run start # run the app in watch mode
17+
make test # unit tests (vitest) · make typecheck · make lint
18+
```
19+
20+
Node 22+. Read **`CLAUDE.md`** (architecture + hard-won gotchas), **`docs/coding-standards.md`**, and **`docs/testing-standards.md`** before touching code — they override defaults.
21+
22+
---
23+
24+
## The AI Scene Agent
25+
26+
An in-editor AI chat panel that drives the user's **own installed `claude` / `codex` CLI** to build scenes — read the scene, write SDK7 code, mutate the scene graph, and run + drive the preview. It bills against the user's subscription (API keys are stripped by default), so there are no metered keys to manage.
27+
28+
**Turn it on** (it's behind a feature flag, off by default): in a dev build set
29+
`localStorage['creator-hub:feature-flags'] = '{"creatorhub-ai-chat":true}'`. The 🤖 "Toggle AI assistant" button then appears in the editor header.
30+
31+
### Mental model
32+
33+
Four moving parts, one per process boundary:
34+
35+
1. **Chat panel** (renderer) — `renderer/src/components/AiChatPanel`, a flex sibling of the inspector iframe in `EditorPage`. Redux slice `renderer/src/modules/store/ai`. It only sends prompts and renders streamed events.
36+
2. **Agent runner** (main) — `main/src/modules/ai.ts` raw-spawns the user's CLI, one child per turn, and parses its NDJSON stdout into events streamed back over the `ai.*` IPC channels. System prompt in `ai-prompt.ts`.
37+
3. **Scene MCP server** (main) — `main/src/modules/scene-mcp.ts`. A localhost, token-gated, **stateful** MCP server (streamable HTTP, one transport per session) handed to the CLI as its tool source. Read tools read the scene's `main.composite` straight from disk; **mutations, metrics, and selection go main → renderer → inspector-iframe over the SceneRpc bridge**, where the inspector runs its real operations on the live engine — so the viewport updates and undo + autosave come for free.
38+
4. **Explorer gateway** (main) — `main/src/modules/explorer-gateway.ts`. On `launch_preview` it starts the Decentraland preview with its `unity-explorer-mcp` server on, connects to it as an MCP *client*, and republishes the Explorer's runtime tools as dynamic `explorer_*` tools (screenshot, walk, click, logs, perf). The stateful server is what lets it push `tools/list_changed` so those tools appear/disappear live.
39+
40+
### The tools the agent gets
41+
42+
Read: `get_project_info`, `scene_state`, `entity_detail`, `get_selection`, `get_scene_metrics`, `editor_screenshot`.
43+
Mutate (all live + undoable): `create_entity`, `remove_entity`, `set_parent`, `set_component`, `remove_component`, `search_catalog`, `place_smart_item`, `attach_script`.
44+
Preview: `launch_preview`, `preview_status`, `stop_preview`, `explorer_call` + the dynamic `explorer_*` set.
45+
46+
Each turn's mutations are counted, so a one-click **"Undo AI changes"** reverts a whole turn. Conversations persist per project (transcript in the renderer, `--resume` ids in main) and survive a restart.
47+
48+
### Where the code lives
49+
50+
| Layer | Files |
51+
|---|---|
52+
| main | `modules/ai.ts` (spawn/stream/env), `ai-prompt.ts`, `scene-mcp.ts` (server + tools + bridge), `explorer-gateway.ts`, `skills.ts` (sdk-skills) |
53+
| preload | `modules/ai.ts` |
54+
| renderer | `components/AiChatPanel`, `components/EditorPage` (the `SCENE_OP_HANDLERS` bridge + effects), `modules/store/ai` (slice, `persistence.ts`, `labels.ts`) |
55+
| inspector | `src/lib/rpc/scene/server.ts` — the `SceneServer`: every read/mutation handler runs here on the live engine |
56+
| shared | `types/ai.ts`, `types/ipc.ts` |
57+
58+
### How to test it
59+
60+
- **Unit:** `npm run test:unit` (main/preload/renderer/shared) and, in `packages/inspector`, `npm run test`. The load-bearing suites are `main/tests/{ai,scene-mcp,explorer-gateway}.test.ts` and `inspector` `scene.spec.ts`.
61+
- **Live (needs a real GPU window):** the viewport, mutations, metrics, and the gateway only work with real WebGL, which Playwright's `_electron.launch` does **not** give you. Launch a raw `electron .` with `--remote-debugging-port` and attach via `chromium.connectOverCDP`. Enable the flag (temp-patch `overrides.ts` or DEV localStorage), and open a scene that **already has `node_modules`** — the unpackaged `electron .` can't npm-install on open.
62+
63+
### Gotchas
64+
65+
- **The engine owns the scene graph.** Never write `main.composite`, `scene.json`, or `main.crdt` on disk — the inspector's autosave clobbers external edits within ~100 ms. Change the scene only through the tools.
66+
- **Stateful MCP server on purpose.** One transport per `Mcp-Session-Id`. That's the correct fix for the original "Server already initialized" bug (which came from sharing one transport) and it's what enables `tools/list_changed`. Don't revert to a shared/stateless transport.
67+
- **Redux freeze.** Deep-clone (`structuredClone`) any Redux-sourced payload before passing it to an in-place mutating helper, or it throws.
68+
- **Providers.** Claude and Codex both get the full tool suite (Codex via `-c mcp_servers…` overrides). Gemini isn't wired yet.
69+
70+
Deeper design notes and the full test/harness recipes live in `CLAUDE.md` and `docs/`.

packages/creator-hub/renderer/src/components/EditorPage/component.tsx

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,6 @@ import EditorPng from '/assets/images/editor.png';
3434
import { ai } from '#preload';
3535
import { useDispatch, useSelector } from '#store';
3636
import { useFeatureFlags } from '/@/hooks/useFeatureFlags';
37-
import { FeatureFlag } from '/@/modules/store/featureFlags';
3837
import { actions as snackbarActions } from '/@/modules/store/snackbar';
3938
import { actions as editorActions } from '/@/modules/store/editor';
4039
import { createGenericNotification } from '/@/modules/store/snackbar/utils';
@@ -131,8 +130,10 @@ export function EditorPage() {
131130
} = useEditor();
132131
const { settings, updateAppSettings } = useSettings();
133132
const { updatePackages } = useWorkspace();
134-
const { flags: featureFlags, isEnabled } = useFeatureFlags();
135-
const aiChatEnabled = isEnabled(FeatureFlag.AI_CHAT);
133+
const { flags: featureFlags } = useFeatureFlags();
134+
// The AI assistant is an experimental opt-in (Settings → Experimental), like the Bevy
135+
// renderer — not a remote feature flag.
136+
const aiChatEnabled = settings.aiAssistant;
136137
const { executeDeployment, getDeployment } = useDeploy();
137138
const deployment = project ? getDeployment(project.path) : undefined;
138139

packages/creator-hub/renderer/src/components/Modals/AppSettings/Tabs/EditorTab/component.tsx

Lines changed: 21 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -74,17 +74,25 @@ const EditorTab: React.FC<EditorTabProps> = ({
7474

7575
const handleExperimentalChange = useCallback(
7676
(checked: boolean) => {
77-
// Turning experimental off returns to the stable default renderer so an
78-
// experimental renderer can't stay active while the picker is hidden.
77+
// Turning experimental off returns to the stable defaults so no experimental feature
78+
// stays active while its controls are hidden (renderer back to Babylon, AI off).
7979
updateSettings({
8080
...settings,
8181
experimental: checked,
8282
renderer: checked ? settings.renderer : RENDERER.BABYLON,
83+
aiAssistant: checked ? settings.aiAssistant : false,
8384
});
8485
},
8586
[settings, updateSettings],
8687
);
8788

89+
const handleAiAssistantChange = useCallback(
90+
(checked: boolean) => {
91+
updateSettings({ ...settings, aiAssistant: checked });
92+
},
93+
[settings, updateSettings],
94+
);
95+
8896
// MCP server connection details, fetched only while the toggle is on. Not persisted:
8997
// the URL/token are runtime values (a fresh port + token per app launch), so they're
9098
// read on demand rather than stored in settings.
@@ -258,6 +266,17 @@ const EditorTab: React.FC<EditorTabProps> = ({
258266
{t('modal.app_settings.fields.renderer.bevy')}
259267
</MenuItem>
260268
</Select>
269+
<FormControlLabel
270+
control={
271+
<Checkbox
272+
checked={!!settings.aiAssistant}
273+
onChange={(event: React.ChangeEvent<HTMLInputElement>) =>
274+
handleAiAssistantChange(event.target.checked)
275+
}
276+
/>
277+
}
278+
label={t('modal.app_settings.fields.ai_assistant.label')}
279+
/>
261280
</Box>
262281
)}
263282
</FormGroup>

packages/creator-hub/renderer/src/modules/store/featureFlags/types.ts

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,4 @@
66
export enum FeatureFlag {
77
/** The Analytics section. Off until the analytics API is deployed. */
88
ANALYTICS = 'creatorhub-analytics',
9-
/** The in-editor AI scene assistant chat panel. Dark until turned on. */
10-
AI_CHAT = 'creatorhub-ai-chat',
119
}

packages/creator-hub/renderer/src/modules/store/translation/locales/en.json

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -417,6 +417,9 @@
417417
"experimental": {
418418
"label": "Experimental features"
419419
},
420+
"ai_assistant": {
421+
"label": "AI scene assistant"
422+
},
420423
"mcp_server": {
421424
"label": "Expose AI assistant MCP server",
422425
"help": "Let an AI agent running outside Creator Hub control the open scene. Paste this into the agent's MCP configuration.",

packages/creator-hub/renderer/src/modules/store/translation/locales/es.json

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -417,6 +417,9 @@
417417
"experimental": {
418418
"label": "Funciones experimentales"
419419
},
420+
"ai_assistant": {
421+
"label": "Asistente de escena con IA"
422+
},
420423
"mcp_server": {
421424
"label": "Exponer el servidor MCP del asistente de IA",
422425
"help": "Permite que un agente de IA externo a Creator Hub controle la escena abierta. Pega esto en la configuración MCP del agente.",

packages/creator-hub/renderer/src/modules/store/translation/locales/zh.json

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -417,6 +417,9 @@
417417
"experimental": {
418418
"label": "实验性功能"
419419
},
420+
"ai_assistant": {
421+
"label": "AI 场景助手"
422+
},
420423
"mcp_server": {
421424
"label": "公开 AI 助手 MCP 服务器",
422425
"help": "让 Creator Hub 之外运行的 AI 代理控制打开的场景。将此内容粘贴到该代理的 MCP 配置中。",

packages/creator-hub/renderer/src/modules/store/workspace/slice.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@ const initialState: Async<Workspace> = {
3636
optimizedAssetsByPath: {},
3737
experimental: false,
3838
renderer: DEFAULT_RENDERER,
39+
aiAssistant: false,
3940
exposeMcpServer: false,
4041
useApiKeyFromEnv: false,
4142
},

packages/creator-hub/shared/types/config.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,7 @@ export const DEFAULT_CONFIG: Config = {
4949
optimizedAssetsByPath: {},
5050
experimental: false,
5151
renderer: DEFAULT_RENDERER,
52+
aiAssistant: false,
5253
exposeMcpServer: false,
5354
useApiKeyFromEnv: false,
5455
},

packages/creator-hub/shared/types/settings.ts

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,9 @@ export type AppSettings = {
5252
experimental: boolean;
5353
// Which engine the inspector uses to render the scene in the editor viewport.
5454
renderer: RENDERER;
55+
// Experimental: the in-editor AI scene assistant chat panel. Opt-in under Experimental
56+
// features (like the renderer picker); off by default and reset when experimental is off.
57+
aiAssistant: boolean;
5558
// Opt-in: expose the Creator Hub MCP server's URL + token in settings so an AI agent
5659
// running OUTSIDE the app (e.g. a studio agent) can register it and drive the open scene.
5760
// The in-app assistant uses that server regardless; this only reveals its connection

0 commit comments

Comments
 (0)