fix(python): hand PyRuntime drop to shutdown_background inside a tokio context

Addresses review feedback on the ctx.fs PR after #2009 (deterministic
teardown) landed and changed PyRuntime::drop underneath it.

#2009's PyRuntime::drop does a blocking runtime join (join_without_gil)
while the interpreter is alive. The ctx.fs adapters now hold PyRuntime
clones, so when a Bash is dropped while `await execute()` is in flight,
the in-flight future drops the last Arc<Mutex<Bash>> — and thus the last
PyRuntime clone — on a pyo3-async-runtimes worker thread. A blocking
runtime drop inside a tokio context panics ("Cannot drop a runtime in a
context where blocking is not allowed"), surfacing as RustPanic to the
awaiting caller. Neither suite caught it because the merge stays green.

- PyRuntime::drop: also use shutdown_background() when
  Handle::try_current().is_ok(), keeping #2009's deterministic
  join_without_gil only on the normal Python-thread path.
- Regression test: drop a Bash while an async-callback execute() is in
  flight, then await it (test_teardown_determinism.py). Verified it
  panics without the fix and passes with it.
- Document and cover the third with_fs dispatch path: a Static handle
  used from a thread with no tokio context (async callbacks under
  execute_sync's private loop / await execute's caller loop) falls
  through to self.rt.block_on. Adds execute_sync + async-callback +
  ctx.fs tests (test_jupyter_compat.py).
- Docs: note that a stashed ctx.fs handle outlives the Bash and keeps
  the VFS + runtime alive (specs + .pyi), and the in-tokio-context drop
  handoff in the teardown-determinism section.

Author: dedeswim

crates/bashkit-python/bashkit/_bashkit.pyi

@@ -95,7 +95,11 @@ class BuiltinContext:
             files created by earlier commands and writes are visible to later
             ones. Same API as ``Bash.fs()`` — e.g. ``ctx.fs.read_file(path)``
             returns ``bytes``. Operates directly on the interpreter's VFS, so
-            it is safe to use from inside the callback.
+            it is safe to use from inside the callback. Ops are synchronous and,
+            while a runtime is active, may run off-thread (relatively expensive
+            per call), so batch filesystem work rather than looping over many
+            tiny ops. May be retained past the callback: a stashed handle keeps
+            the VFS (and its runtime) alive even after the ``Bash`` is dropped.
     """
 
     name: str

crates/bashkit-python/src/lib.rs

@@ -238,7 +238,12 @@ fn join_without_gil<F: FnOnce() + Send>(f: F) {
 /// hang in `test_async_callback_execute_sync_honors_timeout`). Once the
 /// interpreter is exiting, joining is unsafe (threads attaching during
 /// finalization abort the process on CPython < 3.13), so the drop falls
-/// back to `shutdown_background()` and the OS reclaims resources.
+/// back to `shutdown_background()` and the OS reclaims resources. The same
+/// hands-off path is taken when the last clone drops *inside* a tokio context
+/// (e.g. a `Bash` dropped mid-`await execute()` finishes on a runtime worker
+/// thread, dropping its custom-builtin adapters' `PyRuntime` clones there): a
+/// blocking runtime drop in that context panics, so `shutdown_background()`
+/// is used instead.
 struct PyRuntime(Option<Arc<Runtime>>);
 
 impl Clone for PyRuntime {
@@ -259,7 +264,21 @@ impl Drop for PyRuntime {
         let Some(rt) = self.0.take().and_then(Arc::into_inner) else {
             return;
         };
-        if interpreter_at_exit() {
+        // Two cases must avoid the blocking join:
+        //   - `interpreter_at_exit()`: joining threads that may re-attach is
+        //     unsafe during finalization (see the type doc).
+        //   - `Handle::try_current().is_ok()`: the last clone is dropping
+        //     *inside* a tokio context. This happens when a `Bash` is dropped
+        //     while `await execute()` is still in flight: the future holds the
+        //     last `Arc<Mutex<Bash>>` and, on completion, drops it on a
+        //     pyo3-async-runtimes worker thread — cascading into the
+        //     custom-builtin adapters that hold `PyRuntime` clones. Dropping a
+        //     runtime (a blocking join) from within another runtime panics
+        //     with "Cannot drop a runtime in a context where blocking is not
+        //     allowed", so hand off to `shutdown_background()` instead.
+        // The normal path — last clone dropping on a Python thread with no
+        // tokio context — keeps #2009's deterministic `join_without_gil`.
+        if interpreter_at_exit() || Handle::try_current().is_ok() {
             rt.shutdown_background();
         } else {
             join_without_gil(move || drop(rt));
@@ -1634,6 +1653,18 @@ impl PyFileSystem {
         // reused across ops via a channel (note: `block_in_place` is not an
         // option while `make_runtime` builds a current-thread runtime).
         //
+        // A `Static` handle used from a thread with *no* tokio context falls
+        // through to `self.rt.block_on` below. This is the async-callback case:
+        // the callback body runs on an asyncio loop thread — the caller's loop
+        // under `await execute()`, the private loop under `execute_sync()` — not
+        // a tokio worker, so `Handle::try_current()` is `Err`. Under
+        // `execute_sync()` that `block_on` runs on a *second* thread while the
+        // main thread holds the current-thread scheduler core; it completes only
+        // because tokio's parker fallback polls the future without owning the
+        // core. That is fine for VFS ops (they never need the runtime's
+        // timer/IO drivers) — but a timer- or IO-dependent fs future added here
+        // would stall this path, so keep `inner.resolve()` + fs ops driver-free.
+        //
         // `Live` handles keep the original fast path: they lock the interpreter.
         // Re-entrant use of a `Live` handle from inside the shared runtime is
         // unsupported — it re-enters `self.rt.block_on` and panics with the same

crates/bashkit-python/tests/test_jupyter_compat.py

@@ -282,3 +282,50 @@ async def rw(ctx: BuiltinContext) -> str:
     assert result.exit_code == 0
     assert result.stdout == "await\n"
     assert captured == [caller_loop]
+
+
+# ---------------------------------------------------------------------------
+# ctx.fs from an *async* callback driven by execute_sync — the third dispatch
+# path. The async callback body runs on an asyncio loop thread (the private
+# loop, or the background daemon loop under a live loop), which has no tokio
+# context, so ctx.fs ops fall through to `self.rt.block_on` rather than the
+# worker-thread branch the sync-callback case takes. Sync callbacks +
+# execute_sync (worker-thread branch) and async callbacks + await execute
+# (caller loop) are covered above; this fills the remaining combination.
+# ---------------------------------------------------------------------------
+
+
+@pytest.mark.parametrize("factory", [Bash, BashTool], ids=["bash", "bash_tool"])
+def test_execute_sync_async_ctx_fs_builtin(factory):
+    """An async ctx.fs builtin driven by execute_sync() (no running loop) runs on
+    the private loop thread; ctx.fs resolves via the block_on fallback there."""
+
+    async def rw(ctx: BuiltinContext) -> str:
+        await asyncio.sleep(0)  # force a real await so the body runs on the loop
+        ctx.fs.write_file("/async-sync.txt", b"async-sync\n")
+        return ctx.fs.read_file("/async-sync.txt").decode()
+
+    shell = factory(custom_builtins={"rw": rw})
+    result = shell.execute_sync("rw")
+
+    assert result.exit_code == 0
+    assert result.stdout == "async-sync\n"
+    assert shell.fs().read_file("/async-sync.txt").decode() == "async-sync\n"
+
+
+def test_execute_sync_async_ctx_fs_inside_asyncio_run():
+    """Same third path under a live outer loop: execute_sync() inside
+    asyncio.run() drives the async callback on a background daemon loop, and
+    ctx.fs works there too."""
+
+    async def rw(ctx: BuiltinContext) -> str:
+        await asyncio.sleep(0)
+        return ctx.fs.read_file("/seed.txt").decode()
+
+    async def jupyter_cell():
+        bash = Bash(custom_builtins={"rw": rw}, files={"/seed.txt": "seeded\n"})
+        return bash.execute_sync("rw")
+
+    result = asyncio.run(jupyter_cell())
+    assert result.exit_code == 0
+    assert result.stdout == "seeded\n"

crates/bashkit-python/tests/test_teardown_determinism.py

@@ -18,7 +18,7 @@
 
 import pytest
 
-from bashkit import ScriptedTool
+from bashkit import Bash, ScriptedTool
 
 PROC_AVAILABLE = os.path.exists("/proc/self/task")
 
@@ -78,6 +78,34 @@ def test_fds_stable_across_tool_churn():
         assert _fd_count() <= baseline
 
 
+@pytest.mark.asyncio
+async def test_drop_bash_while_async_execute_in_flight_does_not_panic():
+    """A `Bash` dropped while `await execute()` is in flight must not panic.
+
+    The in-flight future holds the last `Arc<Mutex<Bash>>`; when it completes on
+    a pyo3-async-runtimes worker thread, the `Bash` — and its custom-builtin
+    adapters, which each hold a `PyRuntime` clone — drop *there*, so the last
+    `PyRuntime` clone drops inside a tokio context. `PyRuntime::drop` must hand
+    off to `shutdown_background()` rather than a blocking runtime join, which in
+    that context panics ("Cannot drop a runtime in a context where blocking is
+    not allowed") and surfaces as `pyo3_async_runtimes.RustPanic` to the
+    awaiting caller. Regression for the #2009/#2010 interaction.
+    """
+
+    async def slow(ctx):
+        await asyncio.sleep(0.3)
+        return "done\n"
+
+    b = Bash(custom_builtins={"slow": slow})
+    fut = asyncio.ensure_future(b.execute("slow"))
+    await asyncio.sleep(0.05)  # let execution start; callback is in flight
+    del b
+    gc.collect()
+    result = await fut  # RustPanic here before the drop-path fix
+    assert result.exit_code == 0
+    assert result.stdout == "done\n"
+
+
 def test_dropped_tool_cancels_abandoned_callback():
     """Teardown is bounded by cancellation, not callback duration."""
     started = threading.Event()

specs/python-package.md

@@ -309,6 +309,11 @@ calling back into the owning instance's `Bash.fs()` / `Bash.read_file()`, which
 is unsupported re-entrancy: it re-enters the interpreter's runtime and panics
 with a nested-runtime error (not a deadlock, and not caught by the
 `external_handler` reentry guard, which does not fire for custom builtins).
+A callback may retain `ctx.fs` beyond the invocation: the handle stays valid
+even after the `Bash` instance is dropped, and keeps the underlying VFS and
+its tokio runtime alive until the handle itself is released — so stashing it
+extends resource lifetime past `del bash` (see the teardown-determinism
+notes below).
 
 **Sync callbacks** are called directly under the session's captured `contextvars`
 snapshot and may return either a stdout string or a `BuiltinResult` with
@@ -341,7 +346,11 @@ Callbacks that block without awaiting (e.g. `time.sleep` inside `async def`)
 cannot be cancelled mid-section; teardown then waits for the current section
 to reach an await point or return. At interpreter exit (boundary: an `atexit`
 handler registered at module import), teardown goes hands-off — native threads
-must not touch a finalizing CPython — and the OS reclaims resources.
+must not touch a finalizing CPython — and the OS reclaims resources. The same
+hands-off path applies when the last runtime handle is dropped *inside* a tokio
+context (a `Bash` dropped while `await execute()` is still in flight finishes on
+a runtime worker thread): a blocking runtime join there would panic, so the drop
+falls back to `shutdown_background()` instead of the deterministic join.
 
 **ContextVar propagation**: ContextVars set before `execute()` or
 `execute_sync()` are captured at call time and replayed inside each callback