release: v0.3.0 — input-landing feedback, off-screen detection, window geometry tools

Fixes a set of agent papercuts found while driving a live desktop:

- AT-SPI connect no longer fails when an AppArmor-confined snap app (e.g.
  Slack) sits on the a11y bus: the atspi crate's default "p2p" feature
  enumerates every peer during connect and one denied GetInterfaces call
  aborted the whole AccessibilityConnection. Depend on atspi-connection
  directly with p2p off (we never use P2P routing; see #31).
- Targeted press_key/type_text results now append focused-element feedback
  (role, name, editable) read back from AT-SPI after the input, and warn
  when no editable element holds focus — previously keystrokes could land
  nowhere while the result reported verified window focus.
- Screenshot, click, and targeted input results warn when the target window
  or coordinate is partially or fully off-screen. Monitor layout comes from
  the GNOME Shell extension (logical space, used for window bounds) or the
  captured frame size (physical space, used for click coordinates) — the
  two spaces are never mixed, so fractional scaling cannot false-positive.
- New move_window / resize_window tools (GNOME Shell extension v2, which
  also gains GetMonitorLayout) to recover off-screen windows. Old installed
  extensions degrade with an explicit "rerun setup_window_targeting and
  re-login" hint. resize_window unmaximizes first, handling both the
  pre-49 (get_maximized + MaximizeFlags) and 49+ (is_maximized, flagless
  unmaximize) mutter APIs.
- get_app_state now returns a compact readiness block by default and the
  full diagnostics report only with verbose=true; failures point to the
  flag instead of dumping portal/process diagnostics every call.
- scroll gains click-parity window targeting and window-relative
  coordinates; press_key documents its full key grammar in the tool
  description.
- Accessibility errors surface their full anyhow chain ({error:#}) instead
  of the outermost context line.
- cargo update for RUSTSEC-2026-0190 (anyhow); justified cargo-audit
  ignores for the build-time-only quick-xml advisories pulled in via
  wayland-scanner (RUSTSEC-2026-0194/0195).

Author: avifenesh

.cargo/audit.toml

@@ -0,0 +1,16 @@
+# cargo-audit configuration.
+#
+# Ignored advisories must carry a justification and an unignore condition.
+[advisories]
+ignore = [
+    # RUSTSEC-2026-0194 / RUSTSEC-2026-0195: quick-xml 0.39.x DoS issues
+    # (quadratic dup-attribute check, unbounded namespace allocation).
+    # quick-xml is reachable only through wayland-scanner, a proc-macro that
+    # parses the *vendored* Wayland protocol XML at build time; it never
+    # touches attacker-controlled input and is absent from the shipped binary.
+    # Unignore when wayland-scanner releases with quick-xml >= 0.41
+    # (https://github.com/Smithay/wayland-rs — check `cargo update -p
+    # wayland-scanner && cargo tree -i quick-xml`).
+    "RUSTSEC-2026-0194",
+    "RUSTSEC-2026-0195",
+]

Cargo.lock

@@ -1,6 +1,6 @@
 [package]
 name = "computer-use-linux"
-version = "0.2.10"
+version = "0.3.0"
 edition = "2021"
 authors = ["Avi Fenesh <avifenesh@users.noreply.github.com>"]
 license = "MIT"
@@ -23,7 +23,14 @@ path = "src/bin/computer-use-linux-cosmic.rs"
 [dependencies]
 mimalloc = { version = "0.1", default-features = false }
 anyhow = "1.0.102"
-atspi = { version = "0.29.0", features = ["tokio"] }
+# atspi without its "connection" feature: that feature pulls atspi-connection
+# with default features, which include "p2p" — and p2p peer enumeration during
+# connect() calls GetInterfaces on every a11y-bus peer, so a single
+# AppArmor-confined peer (e.g. snap-packaged Slack) aborts the whole
+# AccessibilityConnection. We deliberately never use P2P routing (issue #31),
+# so depend on atspi-connection directly with p2p off.
+atspi = { version = "0.29.0", default-features = false, features = ["proxies", "wrappers", "tokio"] }
+atspi-connection = { version = "0.13.0", default-features = false, features = ["wrappers"] }
 base64 = "0.22.1"
 evdev = "0.13.2"
 cosmic-protocols = { version = "0.2.0", default-features = false, features = ["client"] }

Cargo.toml

@@ -56,12 +56,15 @@ Screenshot payloads are size-bounded by default before they are returned to the
 - `press_key` — keys / chords; can focus a window or terminal first
 - `type_text` — literal text input, optionally targeted at a window or terminal
 
+Targeted `press_key`/`type_text` results append focused-element feedback from AT-SPI (role, name, editable) and warn when no editable element holds focus. Click/screenshot/input results warn when the target window or coordinate is partially or fully off-screen. `get_app_state` returns a compact readiness block by default; pass `verbose: true` for the full diagnostics report.
+
 **Semantic actions**
 - `perform_action` — invoke any AT-SPI action exposed by an element (`Press`, `Activate`, `Toggle`, …); defaults to the primary action
 - `set_value` — write to a settable accessibility element (text fields, sliders, spinners)
 
 **Navigation**
 - `activate_window` — focus a window by `window_id`, `pid`, `app_id`, `wm_class`, `title`, or terminal selectors
+- `move_window` / `resize_window` — reposition or resize a window in desktop coordinates (GNOME Shell extension backend); useful to recover windows that are partially off-screen
 
 ### MCP safety contract
 
@@ -71,7 +74,7 @@ Screenshot payloads are size-bounded by default before they are returned to the
 | --- | --- | --- |
 | Read-only observation | `doctor`, `list_apps`, `list_windows`, `focused_window`, `get_app_state` | `readOnlyHint=true`; may reveal app, window, accessibility, and screenshot contents. `get_app_state` may trigger the desktop screenshot portal prompt. |
 | Local setup mutators | `setup_accessibility`, `setup_window_targeting` | `readOnlyHint=false`, `destructiveHint=false`, `idempotentHint=true`; modifies user desktop configuration by enabling accessibility or installing/enabling the GNOME window-targeting extension. |
-| UI state mutators | `activate_window`, `scroll`, `screenshot` | `readOnlyHint=false`, `destructiveHint=false`; changes focus or scroll position in the live desktop, or raises a window to capture it. |
+| UI state mutators | `activate_window`, `move_window`, `resize_window`, `scroll`, `screenshot` | `readOnlyHint=false`, `destructiveHint=false`; changes focus, geometry, or scroll position in the live desktop, or raises a window to capture it. |
 | Desktop action mutators | `click`, `drag`, `press_key`, `type_text`, `perform_action`, `set_value` | `readOnlyHint=false`, `destructiveHint=true`, `openWorldHint=true`; can trigger arbitrary actions in whatever local application is targeted. |
 
 Annotations are safety hints, not an authorization system. MCP hosts should still ask the user before calls that could submit, delete, send, purchase, overwrite, or otherwise commit state.

README.md

@@ -0,0 +1,27 @@
+// Scratch probe: surface the real AT-SPI connection error chain.
+use atspi_connection::AccessibilityConnection;
+
+#[tokio::main]
+async fn main() {
+    println!(
+        "AT_SPI_BUS_ADDRESS={:?}",
+        std::env::var("AT_SPI_BUS_ADDRESS")
+    );
+    println!(
+        "DBUS_SESSION_BUS_ADDRESS={:?}",
+        std::env::var("DBUS_SESSION_BUS_ADDRESS")
+    );
+    match AccessibilityConnection::new().await {
+        Ok(conn) => {
+            println!("connected OK");
+            match conn.root_accessible_on_registry().await {
+                Ok(root) => match root.get_children().await {
+                    Ok(children) => println!("registry children: {}", children.len()),
+                    Err(e) => println!("get_children error: {e:#?}"),
+                },
+                Err(e) => println!("registry root error: {e:#?}"),
+            }
+        }
+        Err(e) => println!("connect error: {e:#?}"),
+    }
+}

examples/atspi_probe.rs

@@ -22,6 +22,23 @@ const WINDOW_CONTROL_XML = `
       <arg name="ok" type="b" direction="out"/>
       <arg name="message" type="s" direction="out"/>
     </method>
+    <method name="MoveWindow">
+      <arg name="window_id" type="t" direction="in"/>
+      <arg name="x" type="i" direction="in"/>
+      <arg name="y" type="i" direction="in"/>
+      <arg name="ok" type="b" direction="out"/>
+      <arg name="message" type="s" direction="out"/>
+    </method>
+    <method name="ResizeWindow">
+      <arg name="window_id" type="t" direction="in"/>
+      <arg name="width" type="i" direction="in"/>
+      <arg name="height" type="i" direction="in"/>
+      <arg name="ok" type="b" direction="out"/>
+      <arg name="message" type="s" direction="out"/>
+    </method>
+    <method name="GetMonitorLayout">
+      <arg name="json" type="s" direction="out"/>
+    </method>
   </interface>
 </node>
 `;
@@ -89,6 +106,71 @@ class WindowControlDBus extends GObject.Object {
         }
     }
 
+    MoveWindowAsync([windowId, x, y], invocation) {
+        this._withWindow(windowId, invocation, window => {
+            // move_frame positions the frame rect (what list_windows reports).
+            window.move_frame(true, x, y);
+            return `Moved window_id ${Number(windowId)} to ${x},${y}`;
+        });
+    }
+
+    ResizeWindowAsync([windowId, width, height], invocation) {
+        this._withWindow(windowId, invocation, window => {
+            if (width <= 0 || height <= 0)
+                throw new Error(`invalid size ${width}x${height}`);
+            // GNOME 49 removed Meta.Window.get_maximized() (use is_maximized())
+            // and dropped the flags argument from unmaximize(). Support both
+            // API generations: shell 45-48 (get_maximized + flags) and 49+.
+            if (window.is_maximized?.())
+                window.unmaximize();
+            else if (window.get_maximized?.())
+                window.unmaximize(Meta.MaximizeFlags.BOTH);
+            const rect = window.get_frame_rect();
+            window.move_resize_frame(true, rect.x, rect.y, width, height);
+            return `Resized window_id ${Number(windowId)} to ${width}x${height}`;
+        });
+    }
+
+    GetMonitorLayoutAsync(_params, invocation) {
+        const monitors = Main.layoutManager.monitors.map(monitor => ({
+            index: monitor.index,
+            x: monitor.x,
+            y: monitor.y,
+            width: monitor.width,
+            height: monitor.height,
+            primary: monitor.index === Main.layoutManager.primaryIndex,
+            scale: monitor.geometry_scale ?? 1,
+        }));
+        this._returnJson(invocation, monitors);
+    }
+
+    _withWindow(windowId, invocation, action) {
+        const requestedId = Number(windowId);
+        const window = this._listMetaWindows().find(
+            candidate => Number(candidate.get_id()) === requestedId);
+
+        if (!window) {
+            invocation.return_value(new GLib.Variant('(bs)', [
+                false,
+                `No window matched window_id ${requestedId}`,
+            ]));
+            return;
+        }
+
+        try {
+            const message = action(window);
+            invocation.return_value(new GLib.Variant('(bs)', [
+                true,
+                message,
+            ]));
+        } catch (error) {
+            invocation.return_value(new GLib.Variant('(bs)', [
+                false,
+                `Window operation failed: ${error.message}`,
+            ]));
+        }
+    }
+
     _returnJson(invocation, value) {
         invocation.return_value(new GLib.Variant('(s)', [
             JSON.stringify(value),

gnome-shell-extension/computer-use-linux@avifenesh.dev/extension.js

@@ -10,5 +10,5 @@
     "49",
     "50"
   ],
-  "version": 1
-}
+  "version": 2
+}

gnome-shell-extension/computer-use-linux@avifenesh.dev/metadata.json

@@ -1,6 +1,6 @@
 {
   "name": "@agent-sh/computer-use-linux",
-  "version": "0.2.10",
+  "version": "0.3.0",
   "description": "Linux desktop-control MCP server: AT-SPI accessibility trees, Wayland/X11 input, screenshots, and compositor window targeting.",
   "license": "MIT",
   "type": "commonjs",

package.json

@@ -23,6 +23,8 @@
     "focused_window",
     "screenshot",
     "activate_window",
+    "move_window",
+    "resize_window",
     "click",
     "drag",
     "scroll",
@@ -103,6 +105,8 @@
     "setup_accessibility",
     "setup_window_targeting",
     "activate_window",
+    "move_window",
+    "resize_window",
 }
 
 OPEN_WORLD_TOOLS = EXPECTED_TOOLS - {

scripts/mcp_safety_check.py

@@ -5,8 +5,11 @@ use atspi::{
         accessible::{AccessibleProxy, ObjectRefExt},
         proxy_ext::ProxyExt,
     },
-    AccessibilityConnection, CoordType, ObjectRef, ObjectRefOwned, StateSet,
+    CoordType, ObjectRef, ObjectRefOwned, StateSet,
 };
+// Direct dependency (p2p feature off) — see Cargo.toml for why we bypass
+// atspi's "connection" re-export.
+use atspi_connection::AccessibilityConnection;
 use schemars::JsonSchema;
 use serde::Serialize;
 use std::collections::VecDeque;
@@ -157,6 +160,67 @@ pub async fn snapshot_tree(
     Ok(nodes)
 }
 
+/// Compact description of the AT-SPI element that currently holds keyboard
+/// focus, used as post-input feedback for type_text/press_key.
+#[derive(Debug, Clone, Serialize, JsonSchema)]
+pub struct FocusedElementSummary {
+    pub role: String,
+    pub name: Option<String>,
+    pub editable: bool,
+    pub states: Vec<String>,
+}
+
+const FOCUS_PROBE_MAX_NODES: usize = 400;
+const FOCUS_PROBE_MAX_DEPTH: u32 = 16;
+
+/// Find the element with the `focused` state inside the target app (by pid) or
+/// across all apps. Best-effort and bounded: returns Ok(None) when no focused
+/// element is reachable through AT-SPI (common for apps without accessibility
+/// support, e.g. Electron without --force-renderer-accessibility).
+pub async fn focused_element_summary(
+    target_pid: Option<u32>,
+) -> Result<Option<FocusedElementSummary>> {
+    let conn = connect().await?;
+    let roots = registry_children(&conn).await?;
+    let selected_roots = select_roots(&conn, roots, None, target_pid).await;
+    let mut visited = 0_usize;
+    let mut queue = VecDeque::new();
+
+    for object_ref in selected_roots {
+        queue.push_back((object_ref, 0_u32));
+    }
+
+    while let Some((object_ref, depth)) = queue.pop_front() {
+        if visited >= FOCUS_PROBE_MAX_NODES {
+            break;
+        }
+        visited += 1;
+
+        let Ok(proxy) = open_accessible(&conn, &object_ref).await else {
+            continue;
+        };
+        let Ok(state) = proxy.get_state().await else {
+            continue;
+        };
+        if state.contains(atspi::State::Focused) {
+            let proxies = proxy.proxies().await.ok();
+            return Ok(Some(FocusedElementSummary {
+                role: role_name(&proxy).await,
+                name: optional_string(proxy.name().await.ok()),
+                editable: supports_editable_text(proxies.as_ref()).await,
+                states: state_labels(state),
+            }));
+        }
+        if depth < FOCUS_PROBE_MAX_DEPTH {
+            for child in proxy.get_children().await.unwrap_or_default() {
+                queue.push_back((child, depth + 1));
+            }
+        }
+    }
+
+    Ok(None)
+}
+
 pub async fn perform_action(
     object_ref_id: &str,
     requested_action: Option<&str>,