feat: add client restart retries and orphan container cleanup options (#18)

* feat: add client restart retries and orphan container cleanup options

* fix: preserve 120s execution client stop timeout in refactored cleanup

The cleanup refactor moved the execution client teardown into
_teardown_container with stop_timeout=60/5, which would revert the
120s flush window (PGO profiles, RocksDB flush, dotTrace snapshots)
introduced on main. Use 120s unconditionally to keep that behavior.

* fix: validate client_restart_retries and correct run_k6 return type

Address PR review: reject negative client_restart_retries with a clear
ValueError (and min=0 at the CLI), and annotate run_k6 as returning
bytes since detach=False returns the container logs, not a Container.

* ci: add ruff and ty checks and fix existing lint/type issues

Add ruff and ty as dev dependencies and run them as dedicated CI jobs
alongside pytest. Fix the issues they surface:
- drop unused pydantic.NewPath import in scenarios.py
- annotate get_execution_client_volumes as list[dict[str, Any]]
- guard container.attrs before subscripting in the compressor

Author: cbermudez97

.github/workflows/tests.yml

@@ -22,3 +22,37 @@ jobs:
 
       - name: Run tests
         run: uv run pytest -v
+
+  ruff:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v6
+
+      - name: Set up Python
+        run: uv python install
+
+      - name: Install dependencies
+        run: uv sync --dev
+
+      - name: Run ruff
+        run: uv run ruff check .
+
+  ty:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v6
+
+      - name: Set up Python
+        run: uv python install
+
+      - name: Install dependencies
+        run: uv sync --dev
+
+      - name: Run ty
+        run: uv run ty check

pyproject.toml

@@ -39,6 +39,8 @@ packages = ["src/expb"]
 dev = [
     "hatchling>=1.29.0",
     "pytest>=8.0.0",
+    "ruff>=0.10.0",
+    "ty>=0.0.1a1",
 ]
 
 [tool.pytest.ini_options]

src/expb/configs/scenarios.py

@@ -4,7 +4,6 @@
     BaseModel,
     Field,
     FilePath,
-    NewPath,
     field_serializer,
     field_validator,
     model_validator,

src/expb/execute_scenario.py

@@ -83,6 +83,20 @@ def execute_scenario(
             help="Enable JetBrains dotTrace profiling. Auto-installs if needed. Snapshot saved to outputs directory.",
         ),
     ] = False,
+    client_restart_retries: Annotated[
+        int,
+        typer.Option(
+            min=0,
+            help="Auto-restart the execution client on failure up to N times (0 = never restart). Infrastructure containers (K6, Alloy, payload server) never restart.",
+        ),
+    ] = 0,
+    reap_orphans: Annotated[
+        bool,
+        typer.Option(
+            "--reap-orphans/--no-reap-orphans",
+            help="Before running, force-remove leftover containers/networks from prior runs of this same scenario (e.g. orphaned by a hard-killed run). Scoped to the scenario, so it won't disturb other runs on the machine. Off by default.",
+        ),
+    ] = False,
     use_lock: Annotated[
         bool,
         typer.Option(
@@ -150,6 +164,8 @@ def execute_scenario(
                         client_metrics=client_metrics,
                         stable_cpu=stable_cpu,
                         dottrace=dottrace,
+                        client_restart_retries=client_restart_retries,
+                        reap_orphans=reap_orphans,
                     ),
                 )
     except ExecutionLockError as e:

src/expb/execute_scenarios.py

@@ -89,6 +89,20 @@ def execute_scenarios(
             help="Enable JetBrains dotTrace profiling. Auto-installs if needed. Snapshot saved to outputs directory.",
         ),
     ] = False,
+    client_restart_retries: Annotated[
+        int,
+        typer.Option(
+            min=0,
+            help="Auto-restart the execution client on failure up to N times (0 = never restart). Infrastructure containers (K6, Alloy, payload server) never restart.",
+        ),
+    ] = 0,
+    reap_orphans: Annotated[
+        bool,
+        typer.Option(
+            "--reap-orphans/--no-reap-orphans",
+            help="Before running, force-remove leftover containers/networks from prior runs of this same scenario (e.g. orphaned by a hard-killed run). Scoped to the scenario, so it won't disturb other runs on the machine. Off by default.",
+        ),
+    ] = False,
     use_lock: Annotated[
         bool,
         typer.Option(
@@ -186,6 +200,8 @@ def execute_scenarios(
                                 client_metrics=client_metrics,
                                 stable_cpu=stable_cpu,
                                 dottrace=dottrace,
+                                client_restart_retries=client_restart_retries,
+                                reap_orphans=reap_orphans,
                             ),
                         )
                 if not loop:

src/expb/payloads/compressor/compressor.py

@@ -236,6 +236,8 @@ def start_nethermind(
         )
 
         container.reload()
+        if container.attrs is None:
+            raise ValueError("Container attributes are not available")
         container_ip = container.attrs["NetworkSettings"]["Networks"][
             container_network.name
         ]["IPAddress"]