Skip to content

Commit cfeb2a9

Browse files
ishymkoishymko
authored
test: extend install smoke test with runtime checks (#1025)
This is a follow-up for #1019 to prevent similar regressions. Existing install smoke tests are updated to actually run some code with the dependencies listed in the profile. This is different from the unit tests which are executed with dev deps and all extras. In addition this PR restructures this test harness and moves it under `tests/` folder. Although those files do not use pytest and are not executed by pytest their current location under `scripts/` makes less sense as other files in this folder are helpers for local runs. A proper readme is added to clarify how it should be used.
1 parent b387cdd commit cfeb2a9

8 files changed

Lines changed: 286 additions & 161 deletions

File tree

‎.github/workflows/install-smoke.yml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ on:
88
- 'src/**'
99
- 'pyproject.toml'
1010
- 'uv.lock'
11-
- 'scripts/test_install_smoke.py'
11+
- 'tests/install_smoke/**'
1212
- 'scripts/test_install_smoke.sh'
1313
# Self-callout: re-run when this workflow changes so YAML edits are validated in PRs.
1414
- '.github/workflows/install-smoke.yml'
@@ -58,5 +58,5 @@ jobs:
5858
- name: List installed packages
5959
run: VIRTUAL_ENV=.venv-smoke uv pip list
6060

61-
- name: Run import smoke test
62-
run: .venv-smoke/bin/python scripts/test_install_smoke.py ${{ matrix.profile.name }}
61+
- name: Run smoke test (imports + runtime checks)
62+
run: .venv-smoke/bin/python -m tests.install_smoke ${{ matrix.profile.name }}

‎scripts/test_install_smoke.py‎

Lines changed: 0 additions & 152 deletions
This file was deleted.

‎scripts/test_install_smoke.sh‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -2,11 +2,11 @@
22
# Local equivalent of .github/workflows/install-smoke.yml.
33
#
44
# For each install profile, builds the wheel and installs it into a
5-
# clean venv (no dev deps), then runs the import smoke test for that
6-
# profile. By default runs every known profile; pass a profile name
7-
# to run just one.
5+
# clean venv (no dev deps), then runs the smoke test for that profile
6+
# (imports + any per-profile runtime checks). By default runs every
7+
# known profile; pass a profile name to run just one.
88
#
9-
# Available profiles (must match those in scripts/test_install_smoke.py):
9+
# Available profiles (must match those in tests/install_smoke/__main__.py):
1010
# base -- `pip install a2a-sdk`
1111
# http-server -- `pip install a2a-sdk[http-server]`
1212
# grpc -- `pip install a2a-sdk[grpc]`
@@ -87,8 +87,8 @@ for profile in "${PROFILES[@]}"; do
8787
echo "--- Installed packages ---"
8888
VIRTUAL_ENV="$venv_dir" uv pip list
8989

90-
echo "--- Running import smoke test ---"
91-
if ! "$venv_dir/bin/python" scripts/test_install_smoke.py "$profile"; then
90+
echo "--- Running smoke test (imports + runtime checks) ---"
91+
if ! "$venv_dir/bin/python" -m tests.install_smoke "$profile"; then
9292
FAILED_PROFILES+=("$profile")
9393
fi
9494
done

‎tests/install_smoke/README.md‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# Install-smoke harness
2+
3+
This package is **not** a pytest test suite. It is invoked as a
4+
standalone module from a freshly-installed, dev-deps-free venv:
5+
6+
```bash
7+
python -m tests.install_smoke <profile>
8+
```
9+
10+
The smoke venv is created by either
11+
[`scripts/test_install_smoke.sh`](../../scripts/test_install_smoke.sh)
12+
(local) or
13+
[`.github/workflows/install-smoke.yml`](../../.github/workflows/install-smoke.yml)
14+
(CI). The harness has no pytest dependency and uses only the Python
15+
standard library plus the freshly-installed `a2a-sdk` wheel for the
16+
profile under test.
17+
18+
For a given install profile (`base`, `http-server`, `grpc`,
19+
`telemetry`, `sql`) it runs two phases:
20+
21+
1. **Imports**: every module listed for the profile in `__main__.py`
22+
must import cleanly. Catches missing deps and accidental top-level
23+
imports of optional extras.
24+
2. **Runtime checks**: small public-API exercises that
25+
actually call into the SDK. These catch regressions where imports
26+
succeed but a real call fails.
27+
28+
#### Adding a new runtime check
29+
30+
1. Drop a module under `tests/install_smoke/runtime/` exposing two
31+
names:
32+
- `NAME: str` — short human-readable label.
33+
- `check() -> None` — callable that raises on failure.
34+
2. Register it in `RUNTIME_CHECKS` in
35+
[`__main__.py`](./__main__.py) under each profile whose extras it
36+
needs.
37+
38+
Use only the dependencies guaranteed by the target profile. Do not import
39+
`pytest` or any dev-deps.

‎tests/install_smoke/__init__.py‎

Whitespace-only changes.

‎tests/install_smoke/__main__.py‎

Lines changed: 142 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,142 @@
1+
"""Entry point for the install-smoke harness. See README.md."""
2+
3+
from __future__ import annotations
4+
5+
import importlib
6+
import sys
7+
8+
9+
# Modules under each list MUST be importable with only that profile's
10+
# extras installed -- no leakage from other extras (grpc, http-server,
11+
# sql, signing, telemetry, vertex, etc.). Modules that require
12+
# optional extras must use try/except ImportError guards internally.
13+
CORE_MODULES = [
14+
'a2a',
15+
'a2a.client',
16+
'a2a.client.auth',
17+
'a2a.client.base_client',
18+
'a2a.client.card_resolver',
19+
'a2a.client.client',
20+
'a2a.client.client_factory',
21+
'a2a.client.errors',
22+
'a2a.client.interceptors',
23+
'a2a.client.optionals',
24+
'a2a.client.transports',
25+
'a2a.server',
26+
'a2a.server.agent_execution',
27+
'a2a.server.context',
28+
'a2a.server.events',
29+
'a2a.server.request_handlers',
30+
'a2a.server.tasks',
31+
'a2a.types',
32+
'a2a.utils',
33+
'a2a.utils.constants',
34+
'a2a.utils.error_handlers',
35+
'a2a.utils.version_validator',
36+
'a2a.utils.proto_utils',
37+
'a2a.utils.task',
38+
'a2a.helpers.agent_card',
39+
'a2a.helpers.proto_helpers',
40+
]
41+
42+
HTTP_SERVER_MODULES = [
43+
'a2a.server.routes',
44+
'a2a.server.routes.agent_card_routes',
45+
'a2a.server.routes.common',
46+
'a2a.server.routes.jsonrpc_dispatcher',
47+
'a2a.server.routes.jsonrpc_routes',
48+
'a2a.server.routes.rest_dispatcher',
49+
'a2a.server.routes.rest_routes',
50+
]
51+
52+
GRPC_MODULES = [
53+
'a2a.server.request_handlers.grpc_handler',
54+
'a2a.client.transports.grpc',
55+
'a2a.compat.v0_3.grpc_handler',
56+
'a2a.compat.v0_3.grpc_transport',
57+
]
58+
59+
TELEMETRY_MODULES = [
60+
'a2a.utils.telemetry',
61+
]
62+
63+
SQL_MODULES = [
64+
'a2a.server.models',
65+
'a2a.server.tasks.database_task_store',
66+
'a2a.server.tasks.database_push_notification_config_store',
67+
]
68+
69+
70+
PROFILES: dict[str, list[str]] = {
71+
'base': CORE_MODULES,
72+
'http-server': CORE_MODULES + HTTP_SERVER_MODULES,
73+
'grpc': CORE_MODULES + GRPC_MODULES,
74+
'telemetry': CORE_MODULES + TELEMETRY_MODULES,
75+
'sql': CORE_MODULES + SQL_MODULES,
76+
}
77+
78+
79+
# Imported lazily in `main()` so a check that needs one profile's
80+
# extras can't break the harness when running a different profile.
81+
RUNTIME_CHECKS: dict[str, list[str]] = {
82+
'base': ['tests.install_smoke.runtime.base_send_message'],
83+
}
84+
85+
86+
def main(argv: list[str]) -> int:
87+
profile = argv[1] if len(argv) > 1 else 'base'
88+
if profile not in PROFILES:
89+
print(f'Unknown profile {profile!r}. Available: {sorted(PROFILES)}')
90+
return 1
91+
92+
modules = PROFILES[profile]
93+
import_failures: list[str] = []
94+
for module_name in modules:
95+
try:
96+
importlib.import_module(module_name)
97+
except Exception as e: # noqa: BLE001, PERF203
98+
import_failures.append(f'{module_name}: {e}')
99+
100+
print(f'Profile: {profile}')
101+
print(f'Tested {len(modules)} modules')
102+
print(f' Passed: {len(modules) - len(import_failures)}')
103+
print(f' Failed: {len(import_failures)}')
104+
105+
if import_failures:
106+
print('\nFAILED imports:')
107+
for failure in import_failures:
108+
print(f' - {failure}')
109+
return 1
110+
111+
print('\nAll modules imported successfully.')
112+
113+
runtime_checks = RUNTIME_CHECKS.get(profile, [])
114+
if not runtime_checks:
115+
return 0
116+
117+
print(f'\nRunning {len(runtime_checks)} runtime check(s):')
118+
runtime_failures: list[str] = []
119+
for module_path in runtime_checks:
120+
label = module_path
121+
try:
122+
module = importlib.import_module(module_path)
123+
label = module.NAME
124+
module.check()
125+
except Exception as e: # noqa: BLE001, PERF203
126+
runtime_failures.append(f'{label}: {type(e).__name__}: {e}')
127+
print(f' - FAIL: {label}')
128+
else:
129+
print(f' - OK: {label}')
130+
131+
if runtime_failures:
132+
print('\nFAILED runtime checks:')
133+
for failure in runtime_failures:
134+
print(f' - {failure}')
135+
return 1
136+
137+
print('\nAll runtime checks passed.')
138+
return 0
139+
140+
141+
if __name__ == '__main__':
142+
sys.exit(main(sys.argv))

‎tests/install_smoke/runtime/__init__.py‎

Whitespace-only changes.

0 commit comments

Comments
 (0)