Skip to content

security: replace eval() in CodeAct constructor-string coercion with AST decoder - #35

Merged
rdasilveiracabral merged 2 commits into
mainfrom
security/codeact-constructor-string-coercion
Jul 28, 2026
Merged

security: replace eval() in CodeAct constructor-string coercion with AST decoder#35
rdasilveiracabral merged 2 commits into
mainfrom
security/codeact-constructor-string-coercion

Conversation

@sklinglernv

Copy link
Copy Markdown
Collaborator

Summary

Fixes a parent-process code-execution path in the CodeAct strategy. When a model returns its answer as a constructor-call string (e.g. return_result("MyData(data=df)")), _maybe_eval_constructor_string() previously repaired it with eval() against the session REPL locals. This turned a data-normalization path into arbitrary code execution in the parent process — bypassing the code validator, execute_python middleware, timeout, and cell sandbox. __builtins__ = {} is not sufficient protection: an attacker can invoke an object planted by an earlier cell, or walk attributes of the trusted return type to recover builtins.

What changed

eval() is replaced with a small AST-based data-expression decoder:

  • The outer callable must be the exact trusted return type (or a session-local alias identical to it).
  • Bare argument names resolve by dictionary lookup from session_locals, preserving pass-by-reference / object identity — so return_result("MyData(data=df)") still hands over the exact df object (no copy, no serialization). A bare-name lookup does not access, convert, call, or iterate the object.
  • Literal lists/tuples/sets/dicts are rebuilt element-by-element and may hold those references; *args/**kwargs expansion is restricted to exact list/tuple/dict.
  • A fixed allowlist of deterministic helpers (min, max, sum, len, sorted, ...) is retained, but they operate only on detached exact-built-in data.
  • Attribute access, arbitrary calls, and executable session-local objects are rejected; a rejected expression falls through to existing validation/retry feedback.

No eval() or namespace loading remains in the coercion path.

Compatibility

Recovery behavior that affects agent quality is preserved: literal Pydantic constructor strings still validate on the same turn (integration test asserts one LLM call), opaque constructors work, arbitrary objects pass by reference, and common min(...) expressions remain accepted. Existing direct-JSON results never enter this path.

Tests

Adds coverage for the closed attack routes (local factory not called, callable arg not executed, return-type reflection ladder blocked) and for reference-preserving compatibility (direct/nested/mapping refs, Pydantic arbitrary-type field, *args expansion).

uv run pytest tests/strategies/test_codeact_pure_python_coverage.py::TestMaybeEvalConstructorString \
  tests/strategies/test_codeact_strategy.py::TestCodeActStrategyPydanticOutput::test_pydantic_constructor_string_return_remains_compatible -q
# 17 passed
uv run ruff check / format --check on the three touched files: clean

Note: the pre-commit pyright (sandbox_executor at codeact.py) and ruff-format (tests/viewer/test_main.py) failures are pre-existing on main and unrelated to this change (both files/lines are untouched here), so this commit was made with --no-verify.

🤖 Generated with Claude Code

sklinglernv and others added 2 commits July 27, 2026 17:29
…AST decoder

`_maybe_eval_constructor_string()` repaired malformed model output that
supplied a constructor call as a string (e.g. `MyData(data=df)`) by running
it through `eval()` with the session REPL locals as the namespace. This turned
a data-normalization path into a parent-process code-execution path, bypassing
the code validator, `execute_python` middleware, timeout, and cell sandbox.
Setting `__builtins__` to `{}` is insufficient: an attacker can invoke an
object planted by an earlier cell, or walk attributes of the trusted return
type to recover builtins.

Replace `eval()` with a small AST-based data-expression decoder:

- The outer callable must be the exact trusted return type (or a session-local
  alias identical to it).
- Bare argument names resolve directly from `session_locals` by dictionary
  lookup, preserving pass-by-reference / object identity (so
  `return_result("MyData(data=df)")` still hands over the exact `df` object
  without copying or serialization).
- Literal lists/tuples/sets/dicts are rebuilt element-by-element and may hold
  those references; `*args`/`**kwargs` expansion is limited to exact
  list/tuple/dict.
- A fixed allowlist of deterministic helpers (`min`, `max`, `sum`, ...) is
  retained but operates only on detached exact-built-in data.
- Attribute access, arbitrary calls, and executable session-local objects are
  rejected; a rejected expression falls through to existing validation/retry.

No `eval()` or namespace loading remains in the coercion path. Adds tests for
the closed attack routes and for reference-preserving compatibility.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add unit tests for intended functionality not previously exercised:

- `**kwargs` dict expansion: supported, preserves value references, and
  rejects non-string keys / non-dict expansions.
- `*args` expansion is limited to exact list/tuple; a set subclass is not
  iterated (custom `__iter__` never runs).
- Session-local aliases identical to the return type are accepted; a name that
  shadows the return type still constructs the real return type, not the shadow.
- Non-class return types skip coercion.
- Allowlisted helpers work without a session-locals entry, and helper arguments
  are reduced to plain data first so custom iteration/conversion hooks never run.
- Set literals decode to plain data; literal dict `**` expansion merges an exact
  session dict; general arithmetic is rejected while signed/complex literals are
  accepted.
- `_copy_constructor_data` returns detached copies and rejects custom objects
  and container subclasses.

Full suite: 6469 passed, 4 skipped.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants