Commit 80c424a
fix(test): rewrite memory test for Monty, skip unreachable recursion test
The two failing E2Es from 3f508f0 were both written against a Monty
sandbox that doesn't expose the primitives I assumed:
* ``test_memory_limit_enforced`` used ``bytearray(N)`` — Monty raises
``LookupError: Unable to find 'bytearray' in external functions
dict`` because ``bytearray`` isn't an injected helper, so the test
hit ``SANDBOX_RUNTIME_ERROR`` instead of the intended
``SANDBOX_LIMIT_EXCEEDED``. Replaced with a string-multiplication
allocation (``'x' * (12 * 1024 * 1024)``) which uses only the
string-multiply operator — no builtin lookup — and produces a 12 MB
allocation that exceeds the 10 MB ``CODE_MODE_MAX_MEMORY`` default.
* ``test_recursion_limit_enforced`` used an assigned recursive lambda
(``f = lambda n: 1 if n <= 0 else 1 + f(n - 1); f(500)``). Monty
doesn't resolve the lambda's binding name from inside its own body
(``LookupError: Unable to find 'f' in external functions dict``) so
the recursion never actually starts. The test was structurally
incapable of triggering the limit. Renamed to
``test_recursion_limit_unreachable_from_user_code`` and replaced
with a documenting ``pytest.skip`` that explains why no E2E variant
is possible — Monty doesn't allow ``def`` and assigned lambdas
can't recurse — and points at the unit-level coverage that locks
in the classifier mapping.
Also added ``tests/src/unit/test_classify_sandbox_error.py`` (10
tests) covering ``_classify_sandbox_error`` directly: each of the
three buckets (LIMIT_EXCEEDED, SYNTAX_UNSUPPORTED, RUNTIME_ERROR)
gets its trigger exception types exercised, including the
``RecursionError`` path the E2E can't reach and the
``LookupError: Unable to find 'X' in external functions dict``
that Monty produces for missing builtins (it falls into the default
``SANDBOX_RUNTIME_ERROR`` bucket — the right call because the
"use the injected helpers" advice is already in the default
suggestions).
The other 5 new E2E tests from 3f508f0 (invocation cap, traversal,
proxy laundering blocked × 4) all passed in CI; only the two above
were broken.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>1 parent b3f2089 commit 80c424a
2 files changed
Lines changed: 143 additions & 25 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1575 | 1575 | | |
1576 | 1576 | | |
1577 | 1577 | | |
| 1578 | + | |
| 1579 | + | |
| 1580 | + | |
| 1581 | + | |
| 1582 | + | |
| 1583 | + | |
1578 | 1584 | | |
1579 | 1585 | | |
1580 | 1586 | | |
1581 | 1587 | | |
1582 | | - | |
| 1588 | + | |
1583 | 1589 | | |
1584 | 1590 | | |
1585 | 1591 | | |
1586 | 1592 | | |
1587 | | - | |
| 1593 | + | |
1588 | 1594 | | |
1589 | 1595 | | |
1590 | 1596 | | |
| |||
1596 | 1602 | | |
1597 | 1603 | | |
1598 | 1604 | | |
1599 | | - | |
1600 | | - | |
1601 | | - | |
| 1605 | + | |
| 1606 | + | |
| 1607 | + | |
| 1608 | + | |
| 1609 | + | |
| 1610 | + | |
| 1611 | + | |
| 1612 | + | |
| 1613 | + | |
| 1614 | + | |
| 1615 | + | |
| 1616 | + | |
| 1617 | + | |
| 1618 | + | |
| 1619 | + | |
| 1620 | + | |
| 1621 | + | |
| 1622 | + | |
| 1623 | + | |
| 1624 | + | |
| 1625 | + | |
| 1626 | + | |
| 1627 | + | |
| 1628 | + | |
1602 | 1629 | | |
1603 | | - | |
1604 | | - | |
1605 | | - | |
1606 | | - | |
1607 | | - | |
1608 | | - | |
1609 | | - | |
1610 | | - | |
1611 | | - | |
1612 | | - | |
1613 | | - | |
1614 | | - | |
1615 | | - | |
1616 | | - | |
1617 | | - | |
1618 | | - | |
1619 | | - | |
1620 | | - | |
1621 | | - | |
| 1630 | + | |
| 1631 | + | |
| 1632 | + | |
| 1633 | + | |
| 1634 | + | |
1622 | 1635 | | |
1623 | | - | |
1624 | 1636 | | |
1625 | 1637 | | |
1626 | 1638 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
0 commit comments