Skip to content

feat(core): stateful builtin infrastructure#1151

Open
lu-zero wants to merge 3 commits into
reubeno:mainfrom
lu-zero:stateful-builtins
Open

feat(core): stateful builtin infrastructure#1151
lu-zero wants to merge 3 commits into
reubeno:mainfrom
lu-zero:stateful-builtins

Conversation

@lu-zero

@lu-zero lu-zero commented May 9, 2026

Copy link
Copy Markdown
Contributor

Summary

Closes #1149

Adds a per-builtin state mechanism so builtins can carry typed Rust state that persists across invocations within a shell instance, without resorting to hidden shell variables or global mutable state.

Infrastructure (bb64b510)

  • AnyState trait with blanket impl for Clone + Send + Sync + 'static types — any suitable struct can be stored as builtin state with zero boilerplate
  • Command::State associated type on the existing Command trait — defaults to () for all 53 existing builtins (no behavior change)
  • Command::state() / state_mut() ergonomic helpers — builtins call self.state(&context)? with no turbofish
  • state_init fn pointer on Registration — seeded automatically when builtins are registered via Shell::register_builtin or Shell::new
  • builtin_states: HashMap<String, Box<dyn AnyState>> on Shell with typed accessors (builtin_state_of::<B>, builtin_state_mut::<B>, builtin_state::<T>, builtin_state_mut::<T>)
  • ExecutionContext helperscontext.builtin_state::<Self>() / context.builtin_state_mut::<Self>()
  • BuiltinStateNotRegistered error variant

Note: Box<dyn AnyState> itself satisfies the blanket impl bounds, so all accessors use explicit deref (**state) to dispatch through the vtable. The Clone impl for Box<dyn AnyState> uses the same pattern to avoid infinite recursion. This is documented on the trait.

Smoke test: getopts conversion (1e2ed4a3)

Converts getopts from hidden shell variables (__GETOPTS_NEXT_CHAR, __GETOPTS_LAST_OPTIND) to a typed GetOptsState struct:

#[derive(Clone, Debug)]
pub(crate) struct GetOptsState {
    next_char_index: usize,
    last_optind: Option<i32>,
}

Usage pattern — borrow state, copy out, release, do work, borrow mutably, write back:

let mut next_char_index = {
    let state = self.state(&context)?;
    if state.last_optind == Some(next_index_signed) {
        state.next_char_index
    } else {
        DEFAULT_NEXT_CHAR_INDEX
    }
};
// ... parsing work ...
let state = self.state_mut(&mut context)?;
state.next_char_index = next_char_index;
state.last_optind = Some(result.optind as i32);

Net result: 48 fewer lines, no hidden variables leaking into the shell environment, type-safe state access.

Testing

  • All 2434 integration tests pass (including 41 getopts compat tests)
  • 5 new AnyState unit tests (clone roundtrip, mut roundtrip, wrong type, downcast, complex type)
  • cargo clippy, cargo fmt --check, schema check — all clean

@github-actions

github-actions Bot commented May 9, 2026

Copy link
Copy Markdown

Public API changes for crate: brush-builtins

Removed items

-pub trait brush_builtins::ShellBuilderExt
-pub fn brush_builtins::ShellBuilderExt::default_builtins(self, brush_builtins::BuiltinSet) -> Self
-impl<SE: brush_core::extensions::ShellExtensions, S: brush_core::shell::builder::shell_builder::State> brush_builtins::ShellBuilderExt for brush_core::shell::builder::ShellBuilder<SE, S>
-pub fn brush_core::shell::builder::ShellBuilder<SE, S>::default_builtins(self, brush_builtins::BuiltinSet) -> Self
-pub fn brush_builtins::default_builtins<SE: brush_core::extensions::ShellExtensions>(brush_builtins::BuiltinSet) -> std::collections::hash::map::HashMap<alloc::string::String, brush_core::builtins::Registration<SE>>

Added items

+pub trait brush_builtins::ShellExt
+pub fn brush_builtins::ShellExt::register_default_builtins(&mut self, brush_builtins::BuiltinSet)
+impl<SE: brush_core::extensions::ShellExtensions> brush_builtins::ShellExt for brush_core::shell::Shell<SE>
+pub fn brush_core::shell::Shell<SE>::register_default_builtins(&mut self, brush_builtins::BuiltinSet)
+pub fn brush_builtins::register_default_builtins<SE: brush_core::extensions::ShellExtensions>(&mut brush_core::shell::Shell<SE>, brush_builtins::BuiltinSet)

Public API changes for crate: brush-core

Removed items

-impl<SE: brush_core::extensions::ShellExtensions> brush_core::builtins::Registration<SE>
-pub const fn brush_core::builtins::Registration<SE>::special(self) -> Self
-pub fn brush_core::Shell<SE>::register_builtin<S: core::convert::Into<alloc::string::String>>(&mut self, S, brush_core::builtins::Registration<SE>)
-pub fn brush_core::Shell<SE>::register_builtin_if_unset<S: core::convert::Into<alloc::string::String>>(&mut self, S, brush_core::builtins::Registration<SE>)

Added items

+pub struct brush_core::builtins::HasLocalState
+pub struct brush_core::builtins::NeedsLocalState<St>(_)
+pub brush_core::builtins::Registration::_local: core::marker::PhantomData<L>
+pub brush_core::builtins::Registration::_shared: core::marker::PhantomData<S>
+pub brush_core::builtins::Registration::local_override: core::option::Option<alloc::boxed::Box<dyn brush_core::builtins::AnyState>>
+pub brush_core::builtins::Registration::state_init: fn() -> alloc::boxed::Box<dyn brush_core::builtins::AnyState>
+impl<SE: brush_core::extensions::ShellExtensions, S, L> brush_core::builtins::Registration<SE, S, L>
+pub fn brush_core::builtins::Registration<SE, S, L>::special(self) -> Self
+impl<SE: brush_core::extensions::ShellExtensions, S, St: core::clone::Clone + core::marker::Send + core::marker::Sync + 'static> brush_core::builtins::Registration<SE, S, brush_core::builtins::NeedsLocalState<St>>
+pub fn brush_core::builtins::Registration<SE, S, brush_core::builtins::NeedsLocalState<St>>::with_state(self, St) -> brush_core::builtins::Registration<SE, S, brush_core::builtins::HasLocalState>
+impl<SE: brush_core::extensions::ShellExtensions, S, L> core::clone::Clone for brush_core::builtins::Registration<SE, S, L>
+pub fn brush_core::builtins::Registration<SE, S, L>::clone(&self) -> Self
+pub struct brush_core::builtins::SharedBuilder<T, SE: brush_core::extensions::ShellExtensions>
+impl<T: core::clone::Clone + core::marker::Send + core::marker::Sync + 'static, SE: brush_core::extensions::ShellExtensions> brush_core::builtins::SharedBuilder<T, SE>
+pub fn brush_core::builtins::SharedBuilder<T, SE>::builtin(self, impl core::convert::Into<alloc::string::String>, brush_core::builtins::Registration<SE, T>) -> Self
+pub const fn brush_core::builtins::SharedBuilder<T, SE>::new(T) -> Self
+pub struct brush_core::builtins::SharedHandle<'a, T, SE: brush_core::extensions::ShellExtensions>
+impl<T: core::clone::Clone + core::marker::Send + core::marker::Sync + 'static, SE: brush_core::extensions::ShellExtensions> brush_core::builtins::SharedHandle<'_, T, SE>
+pub fn brush_core::builtins::SharedHandle<'_, T, SE>::builtin(&mut self, impl core::convert::Into<alloc::string::String>, brush_core::builtins::Registration<SE, T>)
+pub trait brush_core::builtins::AnyState: core::marker::Send + core::marker::Sync + 'static
+pub fn brush_core::builtins::AnyState::as_any(&self) -> &dyn core::any::Any
+pub fn brush_core::builtins::AnyState::as_any_mut(&mut self) -> &mut dyn core::any::Any
+pub fn brush_core::builtins::AnyState::clone_box(&self) -> alloc::boxed::Box<dyn brush_core::builtins::AnyState>
+impl<T: core::clone::Clone + core::marker::Send + core::marker::Sync + 'static> brush_core::builtins::AnyState for T
+pub fn T::as_any(&self) -> &(dyn core::any::Any + 'static)
+pub fn T::as_any_mut(&mut self) -> &mut (dyn core::any::Any + 'static)
+pub fn T::clone_box(&self) -> alloc::boxed::Box<dyn brush_core::builtins::AnyState>
+pub type brush_core::builtins::Command::SharedState: core::clone::Clone + core::marker::Send + core::marker::Sync + 'static
+pub type brush_core::builtins::Command::State: core::clone::Clone + core::default::Default + core::marker::Send + core::marker::Sync + 'static
+pub fn brush_core::builtins::Command::shared<'a, SE: brush_core::extensions::ShellExtensions>(&self, &'a brush_core::commands::ExecutionContext<'_, SE>) -> core::result::Result<&'a Self::SharedState, brush_core::error::Error>
+pub fn brush_core::builtins::Command::state<'a, SE: brush_core::extensions::ShellExtensions>(&self, &'a brush_core::commands::ExecutionContext<'_, SE>) -> core::result::Result<&'a Self::State, brush_core::error::Error>
+pub fn brush_core::builtins::Command::state_mut<'a, SE: brush_core::extensions::ShellExtensions>(&self, &'a mut brush_core::commands::ExecutionContext<'_, SE>) -> core::result::Result<&'a mut Self::State, brush_core::error::Error>
+pub fn brush_core::commands::ExecutionContext<'_, SE>::builtin_state<B: brush_core::builtins::Command>(&self) -> core::result::Result<&<B as brush_core::builtins::Command>::State, brush_core::error::Error>
+pub fn brush_core::commands::ExecutionContext<'_, SE>::builtin_state<B: brush_core::builtins::Command>(&self) -> core::result::Result<&<B as brush_core::builtins::Command>::State, brush_core::error::Error>
+pub fn brush_core::commands::ExecutionContext<'_, SE>::builtin_state_mut<B: brush_core::builtins::Command>(&mut self) -> core::result::Result<&mut <B as brush_core::builtins::Command>::State, brush_core::error::Error>
+pub fn brush_core::commands::ExecutionContext<'_, SE>::builtin_state_mut<B: brush_core::builtins::Command>(&mut self) -> core::result::Result<&mut <B as brush_core::builtins::Command>::State, brush_core::error::Error>
+pub fn brush_core::commands::ExecutionContext<'_, SE>::shared<T: core::clone::Clone + core::marker::Send + core::marker::Sync + 'static>(&self) -> core::result::Result<&T, brush_core::error::Error>
+pub fn brush_core::commands::ExecutionContext<'_, SE>::shared<T: core::clone::Clone + core::marker::Send + core::marker::Sync + 'static>(&self) -> core::result::Result<&T, brush_core::error::Error>
+pub brush_core::error::ErrorKind::BuiltinStateNotRegistered(alloc::string::String)
+pub brush_core::error::ErrorKind::SharedStateNotRegistered(alloc::string::String)
+pub brush_core::ErrorKind::BuiltinStateNotRegistered(alloc::string::String)
+pub brush_core::ErrorKind::SharedStateNotRegistered(alloc::string::String)
+pub fn brush_core::Shell<SE>::builtin_state<T: 'static>(&self, &str) -> core::option::Option<&T>
+pub fn brush_core::Shell<SE>::builtin_state_mut<T: 'static>(&mut self, &str) -> core::option::Option<&mut T>
+pub fn brush_core::Shell<SE>::builtin_state_mut_of<B: brush_core::builtins::Command>(&mut self, &str) -> core::option::Option<&mut <B as brush_core::builtins::Command>::State>
+pub fn brush_core::Shell<SE>::builtin_state_of<B: brush_core::builtins::Command>(&self, &str) -> core::option::Option<&<B as brush_core::builtins::Command>::State>
+pub fn brush_core::Shell<SE>::get_shared<T: core::clone::Clone + core::marker::Send + core::marker::Sync + 'static>(&self) -> core::result::Result<&T, brush_core::error::Error>
+pub fn brush_core::Shell<SE>::register_builtin<L>(&mut self, impl core::convert::Into<alloc::string::String>, brush_core::builtins::Registration<SE, (), L>)
+pub fn brush_core::Shell<SE>::register_builtin_if_unset<L>(&mut self, impl core::convert::Into<alloc::string::String>, brush_core::builtins::Registration<SE, (), L>)
+pub fn brush_core::Shell<SE>::register_shared<T>(&mut self, brush_core::builtins::SharedBuilder<T, SE>) where T: core::clone::Clone + core::marker::Send + core::marker::Sync + 'static
+pub fn brush_core::Shell<SE>::set_shared<T: core::clone::Clone + core::marker::Send + core::marker::Sync + 'static>(&mut self, T)
+pub fn brush_core::Shell<SE>::shared<T: 'static>(&self) -> core::option::Option<&T>
+pub fn brush_core::Shell<SE>::shared_handle<T>(&mut self) -> brush_core::builtins::SharedHandle<'_, T, SE> where T: core::clone::Clone + core::marker::Send + core::marker::Sync + 'static

Changed items

-pub struct brush_core::builtins::Registration<SE: brush_core::extensions::ShellExtensions>
+pub struct brush_core::builtins::Registration<SE: brush_core::extensions::ShellExtensions, S, L>
-pub fn brush_core::builtins::builtin<B: brush_core::builtins::Command + core::marker::Send + core::marker::Sync, SE: brush_core::extensions::ShellExtensions>() -> brush_core::builtins::Registration<SE>
+pub fn brush_core::builtins::builtin<B: brush_core::builtins::Command + core::marker::Send + core::marker::Sync, SE: brush_core::extensions::ShellExtensions>() -> brush_core::builtins::Registration<SE, <B as brush_core::builtins::Command>::SharedState, brush_core::builtins::NeedsLocalState<<B as brush_core::builtins::Command>::State>>
-pub fn brush_core::builtins::decl_builtin<B: brush_core::builtins::DeclarationCommand + core::marker::Send + core::marker::Sync, SE: brush_core::extensions::ShellExtensions>() -> brush_core::builtins::Registration<SE>
+pub fn brush_core::builtins::decl_builtin<B: brush_core::builtins::DeclarationCommand + core::marker::Send + core::marker::Sync, SE: brush_core::extensions::ShellExtensions>() -> brush_core::builtins::Registration<SE, <B as brush_core::builtins::Command>::SharedState, brush_core::builtins::NeedsLocalState<<B as brush_core::builtins::Command>::State>>
-pub fn brush_core::builtins::raw_arg_builtin<B: brush_core::builtins::DeclarationCommand + core::default::Default + core::marker::Send + core::marker::Sync, SE: brush_core::extensions::ShellExtensions>() -> brush_core::builtins::Registration<SE>
+pub fn brush_core::builtins::raw_arg_builtin<B: brush_core::builtins::DeclarationCommand + core::default::Default + core::marker::Send + core::marker::Sync, SE: brush_core::extensions::ShellExtensions>() -> brush_core::builtins::Registration<SE, <B as brush_core::builtins::Command>::SharedState, brush_core::builtins::NeedsLocalState<<B as brush_core::builtins::Command>::State>>
-pub fn brush_core::ShellBuilder<SE, S>::builtin(self, impl core::convert::Into<alloc::string::String>, brush_core::builtins::Registration<SE>) -> Self
+pub fn brush_core::ShellBuilder<SE, S>::builtin<L>(self, impl core::convert::Into<alloc::string::String>, brush_core::builtins::Registration<SE, (), L>) -> Self

Public API changes for crate: brush-experimental-builtins

Removed items

-pub trait brush_experimental_builtins::ShellBuilderExt
-pub fn brush_experimental_builtins::ShellBuilderExt::experimental_builtins(self) -> Self
-impl<SE: brush_core::extensions::ShellExtensions, S: brush_core::shell::builder::shell_builder::State> brush_experimental_builtins::ShellBuilderExt for brush_core::shell::builder::ShellBuilder<SE, S>
-pub fn brush_core::shell::builder::ShellBuilder<SE, S>::experimental_builtins(self) -> Self
-pub fn brush_experimental_builtins::experimental_builtins<SE: brush_core::extensions::ShellExtensions>() -> std::collections::hash::map::HashMap<alloc::string::String, brush_core::builtins::Registration<SE>>

Added items

+pub fn brush_experimental_builtins::register_experimental_builtins<SE: brush_core::extensions::ShellExtensions>(&mut brush_core::shell::Shell<SE>)

Performance Benchmark Report

Benchmark name Baseline (μs) Test/PR (μs) Delta (μs) Delta %
clone_shell_object 17.50 μs 21.90 μs 4.40 μs 🟠 +25.17%
eval_arithmetic 0.18 μs 0.16 μs -0.02 μs 🟢 -11.48%
expand_one_string 1.65 μs 1.68 μs 0.03 μs ⚪ Unchanged
for_loop 33.88 μs 32.00 μs -1.88 μs 🟢 -5.55%
full_peg_complex 56.30 μs 57.02 μs 0.72 μs ⚪ Unchanged
full_peg_for_loop 6.22 μs 6.27 μs 0.05 μs 🟠 +0.76%
full_peg_nested_expansions 16.69 μs 16.53 μs -0.17 μs ⚪ Unchanged
full_peg_pipeline 4.25 μs 4.28 μs 0.03 μs 🟠 +0.80%
full_peg_simple 1.77 μs 1.79 μs 0.03 μs ⚪ Unchanged
function_call 3.45 μs 3.57 μs 0.12 μs ⚪ Unchanged
instantiate_shell 55.77 μs 57.16 μs 1.38 μs 🟠 +2.48%
instantiate_shell_with_init_scripts 29191.36 μs 39813.74 μs 10622.38 μs 🟠 +36.39%
parse_peg_bash_completion 2086.96 μs 2110.93 μs 23.97 μs ⚪ Unchanged
parse_peg_complex 19.10 μs 19.19 μs 0.09 μs ⚪ Unchanged
parse_peg_for_loop 1.83 μs 1.87 μs 0.03 μs 🟠 +1.69%
parse_peg_pipeline 1.93 μs 1.93 μs -0.01 μs ⚪ Unchanged
parse_peg_simple 1.02 μs 1.03 μs 0.01 μs ⚪ Unchanged
run_echo_builtin_command 15.52 μs 15.65 μs 0.13 μs ⚪ Unchanged
tokenize_sample_script 3.52 μs 3.58 μs 0.07 μs 🟠 +1.93%

Code Coverage Report: Only Changed Files listed

Package Base Coverage New Coverage Difference
brush-builtins/src/getopts.rs 🟢 93.28% 🟢 93.53% 🟢 0.25%
brush-core/src/builtins.rs 🟠 68.25% 🟠 67.35% 🔴 -0.9%
brush-core/src/commands.rs 🟢 91.59% 🟢 90.85% 🔴 -0.74%
brush-core/src/results.rs 🟢 83.33% 🟢 80.16% 🔴 -3.17%
brush-core/src/shell.rs 🟢 93.99% 🟢 90.65% 🔴 -3.34%
brush-core/src/shell/builder.rs 🟢 79.8% 🟠 66.67% 🔴 -13.13%
brush-core/src/shell/builtin_registry.rs 🔴 30% 🔴 30.61% 🟢 0.61%
brush-core/src/variables.rs 🟢 91.55% 🟢 90.86% 🔴 -0.69%
brush-shell/src/brushctl.rs 🔴 6.9% 🔴 8.99% 🟢 2.09%
brush-shell/src/bundled.rs 🔴 18.75% 🔴 18.95% 🟢 0.2%
brush-shell/src/entry.rs 🟢 90.31% 🟢 90.34% 🟢 0.03%
Overall Coverage 🟢 75.3% 🟢 74.94% 🔴 -0.36%

Minimum allowed coverage is 70%, this run produced 74.94%
Maximum allowed coverage difference is -5%, this run produced -0.36%

Test Summary: bash-completion test suite

Outcome Count Percentage
✅ Pass 1581 74.96
❗️ Error 17 0.81
❌ Fail 157 7.44
⏩ Skip 339 16.07
❎ Expected Fail 13 0.62
✔️ Unexpected Pass 2 0.09
📊 Total 2109 100.00

@lu-zero

lu-zero commented May 9, 2026

Copy link
Copy Markdown
Contributor Author

This is a breaking change, if we can take more of the pending PRs (including the async one) we might have a quick 0.5 while with the larger PRs like the winnow parsers wait for more time for review.

@reubeno

reubeno commented May 11, 2026

Copy link
Copy Markdown
Owner

This is a breaking change, if we can take more of the pending PRs (including the async one) we might have a quick 0.5 while with the larger PRs like the winnow parsers wait for more time for review.

Agreed that it would be better to get in the breaks sooner than later.

I really like trying to type the state.

Do you think we might have need for multiple builtins to share the same state? I.e., same shared state but different commands that operate on it. Or would you expect that's the responsibility of the builtins to make happen?

@lu-zero
lu-zero force-pushed the stateful-builtins branch 3 times, most recently from 367941f to 60c177e Compare May 11, 2026 21:28
@lu-zero
lu-zero force-pushed the stateful-builtins branch from 60c177e to 79c9c2f Compare May 25, 2026 04:08

/// Registers experimental built-in commands on the given shell.
pub fn register_experimental_builtins<SE: brush_core::extensions::ShellExtensions>(
shell: &mut brush_core::Shell<SE>,

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems a false positive...

@lu-zero

lu-zero commented May 25, 2026

Copy link
Copy Markdown
Contributor Author

This branch now support both per-builtin and cross-builtin shared state, if you like its shape would be nice to land it so it is one less thing from my long pending set... ^^

lu-zero added 3 commits June 14, 2026 17:24
Add a per-builtin state mechanism so builtins can carry typed Rust state
that persists across invocations within a shell instance.

- Add AnyState trait with blanket impl for Clone+Send+Sync+'static types
- Add Command::State associated type (defaults to () via explicit opt-in)
- Add Command::state() / state_mut() ergonomic helpers (no turbofish)
- Add state_init fn pointer to Registration, seeded at registration time
- Add builtin_states HashMap to Shell with typed accessors
- Seed builtin_states in Shell::new from builder-provided registrations
- Add ExecutionContext::builtin_state[_mut] helpers
- Add BuiltinStateNotRegistered error variant
- Update all 53 existing Command impls with type State = ()

Note: Box<dyn AnyState> satisfies the blanket impl bounds, so all
accessors use explicit deref (**state) to dispatch through the vtable
rather than the blanket impl on Box itself. The Clone impl for
Box<dyn AnyState> uses the same pattern to avoid infinite recursion.

Ref: reubeno#1149
Replace the hidden shell variables (`__GETOPTS_NEXT_CHAR` and
`__GETOPTS_LAST_OPTIND`) with a typed `GetOptsState` struct stored
via the new stateful builtin infrastructure.

The state holds:
- `next_char_index`: position within a combined-flag argument
- `last_optind`: the raw OPTIND value we last set (as i32), to detect
  external changes including OPTIND=0

Uses the ergonomic `self.state(&context)` / `self.state_mut(&mut context)`
helpers from the Command trait, requiring no turbofish.

Ref: reubeno#1149
Add type-safe shared state to brush-core's builtin system, allowing
multiple builtins to share state (e.g. eclass AST caches) across a
shell and its clones via SharedBuilder/SharedHandle.

- Registration<SE, S, L> with phantom types for shared-state routing
  and local-state typestate (NeedsLocalState<St> / HasLocalState)
- with_state(state: St) for custom per-builtin initial state
- SharedBuilder<T> consuming builder for bulk shared registration
- SharedHandle<T> for runtime addition against existing shared state
- Shell::shared_states HashMap<TypeId, Box<dyn AnyState>>
- ExecutionContext::shared::<T>() for immutable shared state access
- SharedStateNotRegistered error variant
- Removed register_builtin_with_state (superseded by with_state())
@lu-zero
lu-zero force-pushed the stateful-builtins branch from 79c9c2f to 8572790 Compare June 14, 2026 16:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Expand the builtin capabilities

3 participants