Skip to content

Commit bd51d16

Browse files
docs: sync tests/README
Closes #48
2 parents bea4614 + 3983cfc commit bd51d16

1 file changed

Lines changed: 64 additions & 30 deletions

File tree

tests/README.md

Lines changed: 64 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -3,55 +3,88 @@
33
## Structure
44

55
### Root-level tests
6-
Broad framework testsmetaclass behavior, event system, sandbox, error formatting, actor, decorator/strategy wiring, method definition, and module imports.
6+
Broad framework tests: metaclass behavior, event system, sandbox, error formatting, actor, decorator/strategy wiring, method definition, media handling, storage compatibility, skill registry, visibility, and module imports.
77

8-
### `runtime/` — Runtime internals
9-
Context building, event manager, code execution, hooks, pure Python executor/REPL, structured output executor, async deadlock prevention, span relationships, and truncation behavior.
8+
### `agentdoc/` - Agent documentation and visibility
9+
`doc(self)` rendering, visibility rules, referenced types, annotated parameters and fields, truncating `pformat`/`pprint`, metadata, protocols, and fixture modules under `agentdoc/fixtures/`.
1010

11-
### `strategies/` — Generation strategies
12-
`CodeActStrategy`, `PurePythonStrategy`, `ReflexionStrategy`, `TemplateStrategy`, argument validation, return type validation, helper method manager, and `RuntimeServices`.
11+
### `agents/` - Agent-level behavior
12+
Agent imports, render configuration, method summarization, and summarization agents.
1313

14-
### `core_runtime/` — Task lifecycle
15-
Task queuing and serialization, code caching (ONCE vs AGENT lifetime), execution, LLM client reuse, and implemented plan behavior.
14+
### `atif/` - Agent Trace Interchange Format
15+
ATIF schema validation, exporter state machines, event typing, nested traces, standalone entrypoints, and end-to-end CodeAct traces.
16+
17+
### `capability/` - Capability routing
18+
Router repeated runs, class method replacement, scorer behavior, structured output scoring, and support agents/data under `capability/agents/` and `capability/data/`.
19+
20+
### `config/` - Configuration resolution
21+
Execution, strategy, summarizer, blocked, and resolved configuration behavior.
22+
23+
### `context_blocks/` - Context block rendering
24+
Context block models, renderers, scoped blocks, cached rendering, event formatting, stats, and truncation.
1625

17-
### `coordinator/` Code validation
26+
### `coordinator/` - Code validation
1827
AST validation, forbidden features, and retry logic.
1928

20-
### `external/` — Public API surface
29+
### `core_runtime/` - Task lifecycle
30+
Task queuing and serialization, code caching (ONCE vs AGENT lifetime), execution, LLM client reuse, and implemented plan behavior.
31+
32+
### `edge_cases/` - Boundary conditions
33+
Generation lock contention, nested generation, child agent edge cases, sandbox edge cases, signal edge cases, missing await detection, builtin shadowing, task wrappers, and agent initialization.
34+
35+
### `external/` - Public API surface
2136
Decorator semantics, stub layer, agent method requirements, provider configuration, and end-to-end notebook scenarios (gold standard for user-facing behavior).
2237

23-
### `integration/` — Cross-cutting tests
24-
Nested generation, concurrent traces, hook failure traces, nested agent history, and CodeAct structured output.
38+
### `helpers/` - Shared support modules
39+
Reusable helper modules for cross-module inheritance, OpenTelemetry assertions, and signature utilities. These are support files, not standalone test suites.
2540

26-
### `edge_cases/` — Boundary conditions
27-
Generation lock contention, nested generation, child agent edge cases, sandbox edge cases, signal edge cases, missing await detection, builtin shadowing, and agent initialization.
41+
### `integration/` - Cross-cutting tests
42+
Nested generation, concurrent traces, hook failure traces, nested agent history, CodeAct structured output, archival behavior, journal fanout, import round-trips, nested bug fixtures under `integration/nested_bug/`, and fixture data under `integration/fixtures/`.
2843

29-
### `agents/` — Agent-level tests
30-
Agent imports and summarization agent behavior.
44+
### `onboarding/` - Model onboarding
45+
Tests for evaluating model performance on framework capabilities (code generation, REPL behavior, validation retry, working context), with sample inputs under `onboarding/test_data/`. Used for model onboarding optimization. See `tests/onboarding/README.md`.
46+
47+
### `performance/` - Performance benchmarks
48+
Client creation overhead.
3149

32-
### `capability/` — Capability routing
33-
Router repeated runs and class method replacement.
50+
### `provider_compat/` - Provider compatibility
51+
Provider compatibility checks across model backends and API shapes.
3452

35-
### `evaluation/` — Evaluation backend
36-
Unified backend with LLM tests.
53+
### `runtime/` - Runtime internals
54+
Context building, event manager, code execution, hooks, pure Python executor/REPL, structured output executor, async deadlock prevention, span relationships, truncation behavior, and runtime evaluation.
3755

38-
### `onboarding/` — Model onboarding
39-
Tests for evaluating model performance on framework capabilities (code generation, REPL behavior, validation retry, working context). Used for model onboarding optimization. See `tests/onboarding/README.md`.
56+
### `runtime/sandbox/` - Sandbox executor
57+
Sandbox executor behavior, broker deadlines, CodeAct sandbox integration, guard enforcement, sandbox config, nested async proxies, read-only module state, teardown behavior, and implicit return handling.
4058

41-
### `tools/` — Built-in tools
59+
### `storage/` - Storage backends
60+
In-memory storage, snapshot markers, serialization, snapshot variables, and snapshot persistence.
61+
62+
### `strategies/` - Generation strategies
63+
`CodeActStrategy`, `PurePythonStrategy`, `ReflexionStrategy`, `TemplateStrategy`, argument validation, return type validation, helper method manager, and `RuntimeServices`.
64+
65+
### `test_mcp/` - MCP client and tool integration
66+
MCP client, tool calls, OAuth discovery, browser detection, exception descriptions, and timeout refresh behavior.
67+
68+
### `tools/` - Built-in tools
4269
Bash tool and file tool tests.
4370

44-
### `utils/` — Utility modules
45-
`doc`, `logger`, `message`, and `task` utility tests.
71+
### `trace_explorer/` - Trace explorer storage and queries
72+
Trace explorer cache, client, database queries, filtered loading, OTLP span conversion, fast path loading, and OpenInference backcompat.
4673

47-
### `performance/` — Performance benchmarks
48-
Client creation overhead.
74+
### `tracing/` - Tracing pipeline
75+
Span context isolation, exporters, journal invariants, metadata, OpenInference conformance, OTLP probing, secret scrubbing, subprocess hooks, viewer attributes, and wire-format stripping.
76+
77+
### `unifiedllm/` - Unified LLM client
78+
Provider detection, cache control, context windows, retries, HTTP config/logging, JSON parsing, schema sanitization, Responses client behavior, strict schema fallback, and tool schema cleanup.
4979

50-
### `unit/` — Isolated unit tests
51-
Mechanical checks schema.
80+
### `unit/` - Isolated unit tests
81+
Focused coverage for context vars, actor behavior, prompts, pragma replacements, skill loading, strategy behavior, utilities, and remaining coverage gaps.
82+
83+
### `utils/` - Utility modules
84+
`doc`, `logger`, `message`, and `task` utility tests.
5285

53-
### `fixtures/` and `helpers/`
54-
Shared test fixtures and helper utilities.
86+
### `viewer/` - Trace viewer API and stores
87+
Viewer main routes, journal and OTLP stores, OTLP ingest, stress coverage, image resolution, message resolution, and block export.
5588

5689
---
5790

@@ -60,6 +93,7 @@ Shared test fixtures and helper utilities.
6093
```bash
6194
uv run pytest # all tests
6295
uv run pytest tests/runtime/ -v # single directory
96+
uv run pytest tests/runtime/sandbox/ # nested runtime area
6397
uv run pytest tests/test_metaclass.py # single file
6498
uv run pytest -k "test_codeact" -v # by name pattern
6599
```

0 commit comments

Comments
 (0)