Skip to content

feat(app-core): apps open in dedicated native windows (Ghost-style)#26

Closed
Dexploarer wants to merge 3407 commits intodevelopfrom
feat/apps-native-windows
Closed

feat(app-core): apps open in dedicated native windows (Ghost-style)#26
Dexploarer wants to merge 3407 commits intodevelopfrom
feat/apps-native-windows

Conversation

@Dexploarer
Copy link
Copy Markdown
Owner

Summary

  • Every app — internal tool, overlay, registry/catalog — launches in its own Electrobun BrowserWindow with slug-based dedupe and persisted bounds.
  • New AppWindowRenderer mounts only the app's content (no surrounding shell chrome) at /apps/<slug> when appWindow=1.
  • Application menu and tray now expose every known app via a single Apps menu (replaces per-surface menus).
  • <scheme>://apps/<slug> deep links open or focus the matching app window.

Architecture

Renderer

  • AppWindowRenderer (new): three-branch full-bleed component that resolves the slug to one of (a) lazy-loaded internal-tool tab component, (b) overlay app's Component, (c) catalog app's viewer iframe with postMessage auth handshake.
  • AppsView.handleLaunch in Electrobun mode always routes through openAppRouteWindow (no fallback to in-shell iframe attach).
  • internal-tool-apps declares an explicit windowPath per tool; new getInternalToolAppWindowPath / getInternalToolAppDescriptors exposed.
  • Foundation files added (not yet wired): per-app-config.ts, launch-history.ts for the upcoming AppDetailsView.

Bun

  • desktopOpenAppWindow accepts slug; threaded through DesktopManagerSurfaceWindowManagerManagedWindowRecord.
  • Slug-based dedupe in createManagedWindow focuses an existing window instead of spawning a duplicate.
  • Per-slug bounds persistence via injected BoundsStore (concrete fs implementation lives in milady root). Saved frame restored on slug match; debounced 500ms saves on resize/move.
  • Per-surface submenus replaced with a unified Apps menu; bun-side APP_MENU_ENTRIES mirror keeps the renderer module out of the bun bundle.
  • Tray gains one entry per app (tray-app-<slug>), routed through handleApplicationMenuAction.
  • setupDeepLinks recognizes <scheme>://apps/<slug> and opens the matching window via findAppMenuEntryBySlug + getDesktopManager().openAppWindow.

Test plan

  • Click any app in AppsView → opens in its own native window with no Milady shell chrome
  • Re-clicking same app focuses existing window (no duplicate)
  • Resize/move an app window, close it, relaunch → restores last bounds
  • Apps menu shows every app; clicking opens its window
  • Tray menu shows every app; clicking opens its window
  • open elizaos://apps/plugins from terminal opens Plugin Viewer
  • Web fallback (open in browser) still uses iframe + sidebar tab path
  • Catalog/registry app windows show launch viewer iframe with auth handshake
  • Overlay (companion) opens in its own window with exitToApps closing the window

renovate Bot and others added 30 commits April 23, 2026 23:07
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
When the Claude Code CLI is installed and authenticated on the user's
machine, getSubscriptionStatus() detected its OAuth blob and reported
the Anthropic subscription panel as "Connected" — but the in-app
Disconnect button only clears app-owned credentials (~/.eliza/auth),
never the CLI's (~/.claude). Result: a phantom "Connected" state the
user could never dismiss.

Surface the credential source on GET /api/subscription/status
(source: "app" | "claude-code-cli" | "setup-token" | "codex-cli"),
and teach the UI to:
  - only show Disconnect when source === "app"
  - render a distinct "Claude Code CLI detected" notice + Connect
    button when the CLI is the only source
  - keep the in-app OAuth path working (app-owned creds take priority
    over CLI creds when both exist)

DELETE /api/subscription/{provider} is unchanged — it should never
try to clear CLI credentials on disk or in the keychain; those
belong to the Claude Code CLI, not to this app.
…asks

Two stacked issues caused the agent to dump its internal task table into
chat whenever the planner picked MANAGE_TASKS with operation=list. A
"doomscroll the web for me" prompt, for example, produced a Discord
reply listing 100+ EMBEDDING_DRAIN ticks alongside PROACTIVE_AGENT,
LIFEOPS_SCHEDULER, and heartbeat entries.

(1) toWorkbenchTask filter was permissive

workbench-helpers.ts:109 excluded only trigger configs and todo tasks.
The runtime's batch-queue util (utils/batch-queue) registers system
ticks like EMBEDDING_DRAIN as elizaOS Tasks in the same table; those
have no trigger config and are not todos, so they sailed through and
appeared in the user-facing workbench list.

The CREATE side of MANAGE_TASKS already tags user-created tasks with
WORKBENCH_TASK_TAG. Make the filter inclusive and require that tag,
matching the convention.

(2) MANAGE_TASKS list operation fired without explicit list intent

manage-tasks.ts validate admits the action when looksLikeTaskIntent
matches either the current message or any of the last 4 room messages.
That lookback handles "yes, list them" follow-ups, but combined with
TEXT_SMALL extraction occasionally classifying vague prompts as
operation=list, the action could fire on a message that did not ask
for a task list at all.

Add a defensive guard at the top of the LIST branch using a tighter
LIST_INTENT_TERMS subset ("list tasks", "show tasks", "my tasks",
"what are my tasks", "task list"). If the current message does not
match list intent, the action returns success=false with an error and
nothing is sent to chat.
setSwarmCompleteCallback was wired to a deliberate no-op, with a
comment claiming the streamer handles synthesis instead. In practice
handleSwarmSynthesis is never called by anyone, so when a task agent
completes the swarm_complete event fires but no user-facing message
is emitted. The agent silently drops the final result (PR URL, build
outcome, etc.) on the floor.

routeSynthesisToConnector additionally gates on coordinator.sourceRoomId,
but that field is declared on the coordinator interface and never
assigned anywhere in the orchestrator, so even when something did
reach this path it would early-return on the null check.

Changes:

1. Wire the callback to invoke handleSwarmSynthesis(st, payload).

2. Thread a fallbackRoomId through routeSynthesisToConnector. Derive
   it from the most recently terminal task in the payload: that is
   the task whose completion fired this swarm_complete, and whose
   room the user is waiting in. Naively taking "first task with a
   roomId" leaked results into stale rooms when the coordinator
   carried tasks across rooms.

3. Read the agent's actual final assistant message (from the
   claude-code session jsonl at ~/.claude/projects/<sanitized-
   workdir>/*.jsonl) as the primary synthesis content, preferring
   it over the coordinator's completionSummary. The completion
   summary is a meta-judgment about whether the task finished,
   not the content the agent produced. The sanitizer replaces both
   "/" and "." with "-" so hidden paths like /home/u/.milady/... map
   correctly.

Verified live: Discord-triggered coding-delegation flows now post the
real deliverable (news brief, code summary, PR url, etc.) into the
originating room, matching the per-task roomId carried through the
payload.
…he turn

stripReplyWhenActionOwnsTurn previously only suppressed REPLY when
another action with suppressPostActionContinuation = true (e.g.
SPAWN_AGENT) was selected for the same turn. The planner commonly
also picks MANAGE_TASKS (operation=create) alongside SPAWN_AGENT for
delegation prompts ("build me a tip calculator", "send a PR to
eliza"). MANAGE_TASKS is not suppressed, so the user gets a noisy
"Created task <name>." confirmation followed by the real delegated
result moments later.

Generalize the dedup: define PASSIVE_TURN_ACTIONS = {REPLY,
MANAGE_TASKS} and drop all of them when a turn-owning action is also
picked. Keeps the same fall-back to ["REPLY"] when everything was
passive. No-op for turns that don't include a passive action, so no
behavior change for the common cases.
…ure replies

buildStructuredFailureReply previously instructed the recovery model to
"Explain what failed and why using only the diagnostics below" and fed
it raw structured-output diagnostics plus action-result summaries. The
resulting replies leaked terminology users have no business seeing.
Observed in discord:

  planner blew up again. the action_planner call to sonnet-4-6 failed
  all 4 retries with a generic anthropic "unexpected error", so no xml
  plan came back and i couldn't commit to a real action this turn.

Rewrite the prompt so the recovery model stays in character, avoids
internal mechanism words (planner, action_planner, XML, TOON, JSON,
schema, model names, etc.), and just acknowledges a hiccup + suggests a
retry. The hard-fallback string used when every model retry also fails
now defaults to a voice-neutral "Something went wrong on my end. Please
try again." and is overridable per character via
character.templates.transientFailureReply.

Removed the now-unused `failure` and `actionResults` locals and the
dependency on summarizeStructuredOutputFailure / summarizeActionResultsForUser
inside this function (still used elsewhere).
The planner LLM occasionally emits an unclosed <action> wrapper around
a well-formed <name>X</name>, e.g.:

  <actions><action><name>REPLY</name></actions>

(closing </action> missing). The strict <action>...</action> regex used
to fall through to "return trimmed" and the entire XML chunk became the
action identifier. That never matched a registered action and the bot
logged "Dropping unknown planner action" then stayed silent.

Symptom in prod logs:

  Dropping unknown planner action (actionName=<action><name>REPLY</name>)

Fix: when the input begins with <action> and contains <name>X</name>,
prefer the name even if the wrapper is unclosed or has trailing content.
This adds one fall-through branch in unwrapPlannerIdentifier; the strict
patterns still take precedence.

Tests cover canonical, flat, missing-close, trailing-garbage, empty,
comma-separated, and multi-action shapes.
lalalune and others added 9 commits April 26, 2026 02:49
Apps that declare configurable runtime / settings / widgets now show a
Details page (config + diagnostics + widget toggles + Launch button)
when clicked, instead of launching directly. Zero-config apps keep the
existing direct-launch behavior.

Per-app declaration
- internal-tool-apps: new optional `hasDetailsPage?: boolean` field on
  InternalToolAppDefinition. Marked true for LifeOps, Fine Tuning,
  Steward, ElizaMaker — the four internal tools with real configuration
  surface area. The other 8 viewers/inspectors stay zero-config and
  open straight in their dedicated window.
- New helper `getInternalToolAppHasDetailsPage(name)` and exposed
  `InternalToolAppDescriptor.hasDetailsPage` for cross-layer mirroring.

AppDetailsView
- New page at packages/app-core/src/components/pages/AppDetailsView.tsx
  (~510 lines). Resolves slug → app via three sources:
    1. internal-tool descriptors (windowPath match)
    2. overlay registry (getAppSlug match, via overlayAppToRegistryInfo)
    3. catalog (client.listApps + client.listCatalogApps, findAppBySlug)
  Sections: Header (hero/name/source/running count), About (description
  + capabilities chips), Recent Runs, Launch Diagnostics (last 5 from
  launch-history.ts), Widgets (own plugin's widgets with enable toggle
  via existing chat-sidebar visibility store + collapsible live
  preview), Config (launch mode + always-on-top), Launch button.
- Launch button reads loaded per-app config (per-app-config.ts):
    * launchMode "window" → invokeDesktopBridgeRequest("desktopOpenAppWindow")
    * launchMode "inline" + internal-tool → setTab(targetTab)
    * launchMode "inline" + overlay → setState("activeOverlayApp", name)
    * launchMode "inline" + other → falls back to window mode
  Inline mode is disabled in the radio for apps that don't support it.
- Helper `appNeedsDetailsPage(name)` exported for AppsView routing.
  Defaults: catalog apps → true, overlays → false, internal tools →
  declared value.

AppsView routing
- New `parseAppsRoute(path)` helper — recognizes /apps/<slug>/details
  and returns {slug, action: "details" | null}.
- `appsDetailsSlug` state with hashchange + popstate listeners so
  back/forward navigation unmounts AppDetailsView.
- pushAppsUrl(slug, subPath?) extended to accept a "details" sub-path.
- handleLaunch checks `appNeedsDetailsPage` first: if true, navigates
  to /apps/<slug>/details and returns (no bridge call). Falls through
  to the existing window/iframe flow for direct-launch apps.
- Auto-launch-from-URL effect skips when action === "details" so
  refreshing the details URL doesn't cause a redirect loop.
- Render mounts <AppDetailsView slug={appsDetailsSlug}> in the apps
  content area when the slug is set, otherwise the catalog grid.

Bun-side menu / tray dispatch
- AppMenuEntry gains `hasDetailsPage: boolean`. Mirrored on each entry
  in APP_MENU_ENTRIES.
- handleApplicationMenuAction for `apps:<slug>` and `tray-app-<slug>`:
    * if entry.hasDetailsPage → restoreWindow() + sendToActiveRenderer
      ("desktopAppDetailsRequested", {slug})
    * else → existing direct getDesktopManager().openAppWindow()
- AppsView subscribes to "desktopAppDetailsRequested" and navigates the
  apps URL to /apps/<slug>/details. Switches to the apps tab + browse
  sub-tab so the user lands on the details page reliably.
…s-native-windows

# Conflicts:
#	packages/app-core/src/components/pages/AppsView.tsx
Comment on lines +19 to +22
export const rs2004scapePlugin: Plugin = gatePluginSessionForHostedApp(
rawRs2004scapePlugin,
"@elizaos/app-2004scape",
);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Potential Security Issue:
The gatePluginSessionForHostedApp function wraps the plugin for session gating. If this function does not enforce strict session validation or access control, it could allow unauthorized access to the plugin's services and actions. Ensure that gatePluginSessionForHostedApp implements robust session validation and restricts access appropriately.

Recommended Solution:
Review the implementation of gatePluginSessionForHostedApp to confirm it enforces proper authentication and authorization checks for hosted app sessions.

export { BotActions } from "./sdk/actions.js";
export { startGateway } from "./gateway/index.js";
export type { GatewayHandle, GatewayOptions } from "./gateway/index.js";
export type * from "./sdk/types.js";
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Type Re-export Risk:
The statement export type * from "./sdk/types.js"; re-exports all types from the SDK. This can unintentionally expose internal or sensitive types, or cause naming conflicts if types are not carefully managed.

Recommended Solution:
Consider explicitly exporting only the necessary types to avoid accidental exposure or conflicts:

export type { SpecificType1, SpecificType2 } from "./sdk/types.js";

app,
}: AppDetailExtensionProps) {
return (
<TwoThousandFourScapeOperatorSurface appName={app.name} variant="detail" />
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no validation for the presence of app or app.name. If app is undefined or does not have a name property, this will result in a runtime error. Consider adding a conditional check or fallback value to ensure robustness:

<TwoThousandFourScapeOperatorSurface appName={app?.name ?? ""} variant="detail" />

Comment on lines +417 to +441
async (content: string) => {
if (!run || content.length === 0 || sending) return false;

setSending(true);
setStatusMessage(null);
try {
if (run.runId) {
const response = await client.sendAppRunMessage(run.runId, content);
setStatusMessage(response.message ?? "Operator message sent.");
return response.success;
}
setStatusMessage("Waiting for the 2004scape command bridge.");
return false;
} catch (error) {
setStatusMessage(
error instanceof Error
? error.message
: "Failed to send the 2004scape operator message.",
);
return false;
} finally {
setSending(false);
}
},
[run, sending],
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Potential race condition in sendOperatorMessage:

The sending state is checked before the async operation, but since React state updates are asynchronous, there is a risk that multiple calls to sendOperatorMessage could bypass the check and execute concurrently. This could result in multiple operator messages being sent unintentionally.

Recommended solution:
Consider using a ref to track the sending state, or disabling the UI trigger (e.g., the Send button) immediately upon invocation to ensure only one message is sent at a time.

const sendingRef = useRef(false);
const sendOperatorMessage = useCallback(async (content: string) => {
  if (!run || content.length === 0 || sendingRef.current) return false;
  sendingRef.current = true;
  ...
  finally {
    sendingRef.current = false;
  }
}, [run]);

Comment on lines +460 to +481
const handleControl = useCallback(
async (action: "pause" | "resume") => {
if (!run) return;
setStatusMessage(null);
try {
const response = await client.controlAppRun(run.runId, action);
setStatusMessage(
response.message ??
(action === "pause"
? "2004scape session paused."
: "2004scape session resumed."),
);
} catch (error) {
setStatusMessage(
error instanceof Error
? error.message
: `Failed to ${action} the 2004scape session.`,
);
}
},
[run],
);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing check for run.runId in handleControl:

In the handleControl function, there is no check to ensure that run.runId is defined before calling client.controlAppRun. If run.runId is undefined, this could result in an error or unintended behavior.

Recommended solution:
Add a check for run.runId before invoking the client method:

if (!run || !run.runId) return;

Comment on lines +889 to +897
const body = await ctx.readJsonBody();
const content =
isRecord(body) && typeof body.content === "string"
? body.content.trim()
: "";
if (!content) {
ctx.error(ctx.res, "Command content is required.", 400);
return true;
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Insufficient input validation for command content

In the POST /message handler, the code checks if body.content is a non-empty string, but does not perform further validation or sanitization. This could allow malicious or malformed input to be processed downstream, potentially leading to security issues or unexpected behavior.

Recommendation:

  • Implement stricter validation and sanitization of user input before processing or forwarding commands. Consider using a schema validation library (e.g., zod, joi) to enforce expected structure and content.

Comment on lines +4 to +6
export function ClawvilleDetailExtension({ app }: AppDetailExtensionProps) {
return <ClawvilleOperatorSurface appName={app.name} variant="detail" />;
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The component assumes that the app prop and its name property are always defined. If app is undefined or does not contain a name, this will result in a runtime error. Consider adding a guard clause or fallback value to handle cases where app or app.name may be missing:

if (!app || !app.name) return null; // or display an error message

This will improve the component's robustness and prevent potential runtime exceptions.

Comment on lines +166 to +229
async (content: string, clearDraftOnSuccess = false) => {
const trimmed = content.trim();
if (!run?.runId || !trimmed || sendingCommand) return;

setSendingCommand(trimmed);
setLocalEvents((current) => [
...current,
{
id: localEventId("clawville-user"),
label: "You",
message: trimmed,
tone: "user",
timestamp: Date.now(),
},
]);

try {
const response = await client.sendAppRunMessage(run.runId, trimmed);
const persistedSession =
response.run?.session ?? response.session ?? null;
if (response.run) {
setState("appRuns", replaceRun(appRuns, response.run));
}
if (clearDraftOnSuccess) {
setDraft((current) => (current.trim() === trimmed ? "" : current));
}
if (persistedSession) {
setLocalEvents([]);
return;
}
setLocalEvents((current) => [
...current,
{
id: localEventId("clawville-game"),
label: response.disposition === "queued" ? "Queued" : "ClawVille",
message: response.message ?? "Command accepted.",
tone:
response.disposition === "accepted"
? "success"
: response.disposition === "queued"
? "info"
: "error",
timestamp: Date.now(),
},
]);
} catch (error) {
setLocalEvents((current) => [
...current,
{
id: localEventId("clawville-error"),
label: "Error",
message:
error instanceof Error
? error.message
: "ClawVille command failed.",
tone: "error",
timestamp: Date.now(),
},
]);
} finally {
setSendingCommand(null);
}
},
[appRuns, run?.runId, sendingCommand, setState],
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Potential race condition in sendCommand

The check for sendingCommand before sending a command is not atomic with the state update. If a user triggers multiple sends in rapid succession (e.g., by double-clicking the send button), it is possible for multiple commands to be sent before sendingCommand is set, leading to duplicate or out-of-order commands. To mitigate this, consider using a ref to track the sending state synchronously, or disable the send button immediately upon initiation of the send operation.

Recommended solution:

const sendingRef = useRef(false);

const sendCommand = useCallback(async (content: string, clearDraftOnSuccess = false) => {
  if (sendingRef.current) return;
  sendingRef.current = true;
  // ...
  try {
    // ...
  } finally {
    sendingRef.current = false;
  }
}, [/* dependencies */]);

This ensures that the sending state is checked and set synchronously, preventing race conditions.

Comment on lines +69 to +74
function replaceRun(appRuns: AppRunSummary[], nextRun: AppRunSummary) {
return [
...appRuns.filter((candidate) => candidate.runId !== nextRun.runId),
nextRun,
].sort((left, right) => right.updatedAt.localeCompare(left.updatedAt));
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assumption about updatedAt in replaceRun

The replaceRun function sorts runs by updatedAt using localeCompare, assuming that updatedAt is always a valid string. If updatedAt is missing or not a valid ISO string, this could result in incorrect sorting and potentially display the wrong run as the most recent.

Recommended solution:
Add a fallback or validation for updatedAt:

.sort((left, right) => {
  const leftDate = left.updatedAt || '';
  const rightDate = right.updatedAt || '';
  return rightDate.localeCompare(leftDate);
});

Or, if possible, ensure that updatedAt is always present and valid at the data source.

Comment on lines +8 to +9
registerOperatorSurface("@clawville/app-clawville", ClawvilleOperatorSurface);
registerDetailExtension("clawville-control", ClawvilleDetailExtension);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lack of error handling for registration calls

The calls to registerOperatorSurface and registerDetailExtension are made without any error handling. If these functions throw exceptions or fail, the module could fail to load, potentially impacting application stability.

Recommended solution:
Wrap the registration calls in a try-catch block to handle potential errors gracefully:

try {
  registerOperatorSurface("@clawville/app-clawville", ClawvilleOperatorSurface);
  registerDetailExtension("clawville-control", ClawvilleDetailExtension);
} catch (error) {
  // Handle or log the error appropriately
}

github-actions Bot and others added 18 commits April 26, 2026 11:08
- Deep link `<scheme>://apps/<slug>` now respects `hasDetailsPage`,
  routing to the details page instead of opening the window directly,
  matching the menu/tray click flow. Also strips extra path segments.
- AppsView's `desktopAppDetailsRequested` listener calls
  `setAppsDetailsSlug` directly so the details view mounts even when
  routing uses `history.replaceState` (which fires no event).
- Bounds store `isFrame` adds a `y > -16000` lower-bound guard,
  mirroring the existing x-guard, to reject saved frames from windows
  dragged off the top of a multi-monitor setup.
- Extract shared `useRegistryCatalog` hook with module-level inflight
  promise so `AppDetailsView` and `RegistryAppWindowView` share one
  catalog fetch when both mount for the same slug.
@Dexploarer
Copy link
Copy Markdown
Owner Author

Superseded by elizaOS#7115 (org fork → upstream).

@Dexploarer Dexploarer closed this Apr 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants