Skip to content

Commit ee7c1f4

Browse files
huyouareclaude
andauthored
Mirror trial artifacts to sauron with run-meta.json manifest (#39)
* Add sauron S3 mirror — upload trial artifacts to sauron's AWS bucket When ODDISH_SAURON_S3_BUCKET is set, oddish workers also mirror trial results to sauron's `abundant-github-workflows-bucket` in sauron's expected directory layout. This allows sauron's frontend to render oddish-originated experiments natively using its existing components, renderers, and trajectory viewer. Files added: - oddish/integrations/sauron/__init__.py - oddish/integrations/sauron/s3_uploader.py: SauronS3Uploader class - Constructs sauron paths: {org}/{repo}/pr-{n}/run-{exp_id}/agent-{name}:{model}/{task}/attempt_{n}/ - Unwraps Harbor's task-{hash}/ subdirectory (sauron expects flat layout) - Uploads result.json, agent/trajectory.json, verifier/* at attempt root - Best-effort upload of task/instruction.md, task.toml, solution/, tests/ - Generates experiment-manifest.yaml (sauron uses for pre-rendering) Files modified: - oddish/config.py: 6 new sauron_s3_* settings - oddish/workers/queue/trial_handler.py: - Expanded PreparedTrialRun with task_name, experiment_id, experiment_name, attempt_number, task_tags - Added sauron upload call after existing oddish S3 upload - Reordered cleanup to happen AFTER both uploads - backend/.env.example: documented new env vars Behavior: - Disabled when ODDISH_SAURON_S3_BUCKET is empty (existing deployments unaffected). - PR-triggered experiments use github_meta from task.tags for org/repo/pr. - CLI-triggered experiments use ODDISH_SAURON_S3_ORG/REPO fallback with pr-0. - Sauron upload is wrapped in try/except — never blocks trial completion. To enable in production, add to oddish-prod Modal secret: ODDISH_SAURON_S3_BUCKET=abundant-github-workflows-bucket ODDISH_SAURON_S3_REGION=us-west-2 ODDISH_SAURON_S3_ACCESS_KEY=<aws-credentials access key> ODDISH_SAURON_S3_SECRET_KEY=<aws-credentials secret key> ODDISH_SAURON_S3_ORG=oddish ODDISH_SAURON_S3_REPO=cli-runs https://claude.ai/code/session_01A4jdAghus82QSiFwj51du5 * Fix: call ensure_experiment_manifest from trial handler The manifest upload method existed on SauronS3Uploader but was never called. Without the manifest, sauron can't pre-render pending cells or discover the expected experiment shape. Now called before each trial upload (idempotent — skips if already uploaded). The manifest initially contains just the current trial's task/agent; sauron's streaming logic handles partial manifests. https://claude.ai/code/session_01A4jdAghus82QSiFwj51du5 * Fix: use trial index (not retry count) for attempt number Critical bug: trial.attempts is the retry counter, not the trial's position within its task. Three trials of the same task (trial-0, trial-1, trial-2) all had attempts=1 on first run, writing to the same attempt_1/ folder and overwriting each other. Fix: extract the 0-based index from trial_id ("{task_id}-{index}") and use index+1 as the sauron attempt number. trial-0 → attempt_1, trial-1 → attempt_2, trial-2 → attempt_3. Also: fall back to AWS_ACCESS_KEY_ID/SECRET_ACCESS_KEY env vars when ODDISH_SAURON_S3_ACCESS_KEY is not set. This eliminates 2 env vars if the aws-credentials Modal secret is already attached. Minimum env vars needed: ODDISH_SAURON_S3_BUCKET=abundant-github-workflows-bucket ODDISH_SAURON_S3_ORG=oddish (optional, defaults to "oddish") ODDISH_SAURON_S3_REPO=cli-runs (optional, defaults to "cli-runs") Plus AWS credentials from existing aws-credentials secret. https://claude.ai/code/session_01A4jdAghus82QSiFwj51du5 * Simplify: remove redundant sauron S3 credential settings The uploader now uses AWS_ACCESS_KEY_ID/SECRET_ACCESS_KEY directly from the environment (provided by aws-credentials Modal secret). No need for separate ODDISH_SAURON_S3_ACCESS_KEY/SECRET_KEY/REGION. Config reduced from 6 settings to 3: ODDISH_SAURON_S3_BUCKET (required — the only one that enables the mirror) ODDISH_SAURON_S3_ORG (optional, defaults to "oddish") ODDISH_SAURON_S3_REPO (optional, defaults to "cli-runs") https://claude.ai/code/session_01A4jdAghus82QSiFwj51du5 * Simplify sauron S3 uploader: 401 → 163 lines Removed: - SauronUploadContext dataclass (pass args directly) - ensure_experiment_manifest (incomplete, sauron falls back to S3 listing) - _upload_task_files (redundant — trial subdir already has these files) - _object_exists, _put_object (only used by manifest) - _yaml_quote, _build_manifest_yaml (manifest removed) - Unused imports (io, tarfile, re) - experiment_name from PreparedTrialRun (no longer needed) - ExperimentModel DB query in _prepare_trial_run What remains: one clean upload_trial() method that finds the trial subdirectory, unwraps it, and uploads to sauron's S3 path. https://claude.ai/code/session_01A4jdAghus82QSiFwj51du5 * Manifest-first sauron mirror; honest path for CLI runs - Drop the synthetic pr-0 layer for CLI runs. Path is now {org_slug}/runs/run-{experiment_id}/ - matches oddish's actual org -> experiment shape (no fake repo/PR levels). - PR-triggered path unchanged: {owner}/{repo}/pr-{N}/run-{id}/ - Write run-meta.json at run root for both kinds. Carries kind, experiment id/name, github metadata, free-form tags. Sauron reads this for run header rendering instead of parsing the path. - Remove sauron_s3_repo setting (no longer used). - Look up experiment name in trial handler and pass to mirror. * Pad CLI path to 4 segments to match sauron's existing route CLI path was {org}/runs/run-{id}/... (3 segments above run files), which would require sauron to refactor getRunStructure/streaming API/snapshot loader to support shorter prefixes (~300 LOC, wide blast radius). Pad to 4 segments using experiment_id as the grouping ("pr") segment: {org}/runs/{experiment_id}/run-{experiment_id}/ Sauron's existing [org]/[repo]/[pr]/[run] route renders this without modification. run-meta.json still carries the real metadata so sauron can show "Experiment {name}" instead of "PR #{number}" in a small follow-up. PR-triggered path unchanged. * Retrigger CI (Supabase preview migration failure was transient) --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 80d2108 commit ee7c1f4

5 files changed

Lines changed: 298 additions & 1 deletion

File tree

backend/.env.example

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,17 @@ ODDISH_S3_ACCESS_KEY=...
2727
ODDISH_S3_SECRET_KEY=...
2828
ODDISH_S3_ENDPOINT_URL=...
2929

30+
# =============================================================================
31+
# Sauron S3 mirror (optional)
32+
# =============================================================================
33+
# When configured, oddish workers also upload trial artifacts to sauron's
34+
# AWS S3 bucket in sauron's expected layout. Uses AWS_ACCESS_KEY_ID and
35+
# AWS_SECRET_ACCESS_KEY from environment for credentials (e.g. from the
36+
# aws-credentials Modal secret).
37+
# Disabled when ODDISH_SAURON_S3_BUCKET is empty.
38+
# ODDISH_SAURON_S3_BUCKET=abundant-github-workflows-bucket
39+
# ODDISH_SAURON_S3_ORG=oddish # top-level segment for CLI runs: {org}/runs/run-{id}/
40+
3041
# =============================================================================
3142
# LLM/Sandbox provider keys
3243
# =============================================================================

oddish/src/oddish/config.py

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -268,6 +268,16 @@ def asyncpg_server_settings(self) -> dict[str, str]:
268268
s3_bucket: str = "data"
269269
s3_region: str = "us-east-1"
270270

271+
# Sauron S3 mirror (optional, disabled when bucket is empty).
272+
# When configured, oddish workers also upload trial artifacts to sauron's
273+
# AWS S3 bucket in sauron's expected directory layout, allowing sauron's
274+
# frontend to render oddish-originated experiments natively.
275+
# Uses AWS_ACCESS_KEY_ID/SECRET_ACCESS_KEY from environment for credentials.
276+
sauron_s3_bucket: str = ""
277+
# Org slug used as the top-level path segment for non-PR (CLI-triggered)
278+
# experiments. PR-triggered runs derive owner/repo from task.tags.github_meta.
279+
sauron_s3_org: str = "oddish"
280+
271281
# Task upload limits (MB)
272282
max_task_upload_mb: int = 50
273283

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
"""Sauron S3 mirror — optional upload of trial results to sauron's AWS bucket."""
2+
3+
from oddish.integrations.sauron.s3_uploader import get_sauron_uploader
4+
5+
__all__ = ["get_sauron_uploader"]
Lines changed: 208 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,208 @@
1+
"""
2+
Mirror trial artifacts to sauron's AWS S3 bucket.
3+
4+
When ODDISH_SAURON_S3_BUCKET is set, trial results are uploaded to
5+
sauron's bucket so sauron can render oddish experiments using its
6+
existing UI components.
7+
8+
Layout:
9+
PR-triggered:
10+
{owner}/{repo}/pr-{n}/run-{experiment_id}/
11+
run-meta.json
12+
agent-{name}:{model}/{task_name}/attempt_{n}/...
13+
14+
CLI-triggered:
15+
{org_slug}/runs/{experiment_id}/run-{experiment_id}/
16+
run-meta.json
17+
agent-{name}:{model}/{task_name}/attempt_{n}/...
18+
19+
The CLI path uses the experiment_id literally as the grouping segment
20+
(not a synthetic pr-N) so sauron's existing 4-segment route renders it
21+
without modification. `run-meta.json` carries identity/metadata at the
22+
run root so sauron can render run headers without parsing the path or
23+
hitting GitHub.
24+
"""
25+
26+
from __future__ import annotations
27+
28+
import asyncio
29+
import json
30+
import logging
31+
import os
32+
from pathlib import Path
33+
from typing import Any
34+
35+
import aioboto3
36+
from botocore.config import Config
37+
38+
from oddish.config import settings
39+
from oddish.integrations.github.client import GitHubMeta
40+
41+
logger = logging.getLogger(__name__)
42+
43+
MANIFEST_SCHEMA_VERSION = 1
44+
45+
46+
class SauronS3Uploader:
47+
"""Best-effort mirror of trial artifacts to sauron's AWS S3 bucket."""
48+
49+
def __init__(self) -> None:
50+
self._client = None
51+
self._session = None
52+
53+
def is_enabled(self) -> bool:
54+
return bool(
55+
settings.sauron_s3_bucket
56+
and os.environ.get("AWS_ACCESS_KEY_ID")
57+
and os.environ.get("AWS_SECRET_ACCESS_KEY")
58+
)
59+
60+
async def _ensure_client(self) -> None:
61+
if self._client is not None:
62+
return
63+
self._session = aioboto3.Session()
64+
self._client = await self._session.client(
65+
"s3",
66+
aws_access_key_id=os.environ["AWS_ACCESS_KEY_ID"],
67+
aws_secret_access_key=os.environ["AWS_SECRET_ACCESS_KEY"],
68+
region_name=os.environ.get("AWS_REGION", "us-west-2"),
69+
config=Config(signature_version="s3v4"),
70+
).__aenter__()
71+
72+
async def upload_trial(
73+
self,
74+
*,
75+
harbor_job_dir: Path,
76+
task_name: str,
77+
agent: str,
78+
model: str | None,
79+
experiment_id: str,
80+
experiment_name: str | None,
81+
attempt_number: int,
82+
github_meta: GitHubMeta | None,
83+
task_tags: dict | None = None,
84+
) -> str | None:
85+
"""Upload trial artifacts. Returns the trial S3 prefix or None on failure."""
86+
if not self.is_enabled():
87+
return None
88+
89+
run_prefix = self._build_run_prefix(github_meta=github_meta, experiment_id=experiment_id)
90+
attempt_prefix = (
91+
f"{run_prefix}agent-{agent}:{(model or 'default').replace('/', '-')}/"
92+
f"{task_name}/attempt_{attempt_number}/"
93+
)
94+
95+
# Harbor's job_dir contains a task-{name}__{hash}/ subdirectory with
96+
# the actual trial output. Sauron expects these at the attempt root.
97+
source = self._find_trial_subdir(harbor_job_dir) or harbor_job_dir
98+
99+
try:
100+
await self._upload_directory(source, attempt_prefix)
101+
await self._write_manifest(
102+
run_prefix=run_prefix,
103+
experiment_id=experiment_id,
104+
experiment_name=experiment_name,
105+
github_meta=github_meta,
106+
task_tags=task_tags,
107+
)
108+
return attempt_prefix
109+
except Exception as e:
110+
logger.warning("Sauron mirror failed for %s: %s", attempt_prefix, e)
111+
return None
112+
113+
# -- Path construction ---------------------------------------------------
114+
115+
@staticmethod
116+
def _build_run_prefix(*, github_meta: GitHubMeta | None, experiment_id: str) -> str:
117+
if github_meta:
118+
return (
119+
f"{github_meta.owner}/{github_meta.repo}/"
120+
f"pr-{github_meta.pr_number}/run-{experiment_id}/"
121+
)
122+
org = settings.sauron_s3_org or "oddish"
123+
# 4-segment path so sauron's existing [org]/[repo]/[pr]/[run] route
124+
# renders without modification. The experiment_id appears twice:
125+
# once as the grouping ("pr") segment, once as the run identifier.
126+
return f"{org}/runs/{experiment_id}/run-{experiment_id}/"
127+
128+
# -- Manifest ------------------------------------------------------------
129+
130+
async def _write_manifest(
131+
self,
132+
*,
133+
run_prefix: str,
134+
experiment_id: str,
135+
experiment_name: str | None,
136+
github_meta: GitHubMeta | None,
137+
task_tags: dict | None,
138+
) -> None:
139+
"""Write run-meta.json at run root. Last-writer-wins for stable fields."""
140+
manifest: dict[str, Any] = {
141+
"schema_version": MANIFEST_SCHEMA_VERSION,
142+
"kind": "pr" if github_meta else "experiment",
143+
"experiment_id": experiment_id,
144+
"experiment_name": experiment_name,
145+
"github": (
146+
{
147+
"owner": github_meta.owner,
148+
"repo": github_meta.repo,
149+
"pr_number": github_meta.pr_number,
150+
}
151+
if github_meta
152+
else None
153+
),
154+
"tags": {k: v for k, v in (task_tags or {}).items() if k != "github_meta"},
155+
}
156+
body = json.dumps(manifest, indent=2, sort_keys=True).encode("utf-8")
157+
158+
await self._ensure_client()
159+
await self._client.put_object(
160+
Bucket=settings.sauron_s3_bucket,
161+
Key=f"{run_prefix}run-meta.json",
162+
Body=body,
163+
ContentType="application/json",
164+
)
165+
166+
# -- Harbor directory unwrapping -----------------------------------------
167+
168+
@staticmethod
169+
def _find_trial_subdir(harbor_job_dir: Path) -> Path | None:
170+
"""Find the trial subdirectory (task-name__hash/) inside job_dir."""
171+
if not harbor_job_dir.exists():
172+
return None
173+
subdirs = [d for d in harbor_job_dir.iterdir() if d.is_dir()]
174+
trial_dirs = [d for d in subdirs if "__" in d.name]
175+
if len(trial_dirs) == 1:
176+
return trial_dirs[0]
177+
if len(subdirs) == 1:
178+
return subdirs[0]
179+
return None
180+
181+
# -- S3 upload -----------------------------------------------------------
182+
183+
async def _upload_directory(self, local_dir: Path, s3_prefix: str) -> None:
184+
files = [p for p in local_dir.rglob("*") if p.is_file()]
185+
if not files:
186+
return
187+
188+
sem = asyncio.Semaphore(16)
189+
190+
async def upload_one(f: Path) -> None:
191+
key = f"{s3_prefix}{f.relative_to(local_dir).as_posix()}"
192+
async with sem:
193+
await self._ensure_client()
194+
await self._client.upload_file(str(f), settings.sauron_s3_bucket, key)
195+
196+
await asyncio.gather(*(upload_one(f) for f in files), return_exceptions=True)
197+
198+
199+
# -- Singleton ---------------------------------------------------------------
200+
201+
_uploader: SauronS3Uploader | None = None
202+
203+
204+
def get_sauron_uploader() -> SauronS3Uploader:
205+
global _uploader
206+
if _uploader is None:
207+
_uploader = SauronS3Uploader()
208+
return _uploader

oddish/src/oddish/workers/queue/trial_handler.py

Lines changed: 64 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@
1717
from oddish.config import settings
1818
from oddish.db import (
1919
AnalysisStatus,
20+
ExperimentModel,
2021
TaskModel,
2122
TaskStatus,
2223
TaskVersionModel,
@@ -32,6 +33,14 @@
3233
TRIAL_HEARTBEAT_INTERVAL_SECONDS = 30
3334

3435

36+
def _extract_trial_index(trial_id: str, task_id: str) -> int:
37+
"""Extract the 0-based trial index from a trial ID like '{task_id}-{index}'."""
38+
suffix = trial_id[len(task_id):] # e.g., "-0", "-1", "-2"
39+
if suffix.startswith("-") and suffix[1:].isdigit():
40+
return int(suffix[1:])
41+
return 0
42+
43+
3544
@dataclass(slots=True)
3645
class PreparedTrialRun:
3746
task_path: str | None
@@ -41,6 +50,12 @@ class PreparedTrialRun:
4150
trial_model: str
4251
trial_environment: str | None
4352
trial_harbor_config: dict | None
53+
# Fields for sauron S3 mirror
54+
task_name: str = ""
55+
experiment_id: str = ""
56+
experiment_name: str | None = None
57+
attempt_number: int = 1
58+
task_tags: dict | None = None
4459

4560

4661
@dataclass(slots=True)
@@ -336,6 +351,15 @@ async def _prepare_trial_run(
336351
task.started_at = utcnow()
337352

338353
task_id = task.id if task else trial.task_id
354+
task_name = task.name if task else trial.task_id
355+
task_tags = dict(task.tags) if task and task.tags else None
356+
357+
experiment_id = trial.experiment_id or ""
358+
experiment_name: str | None = None
359+
if experiment_id:
360+
experiment = await session.get(ExperimentModel, experiment_id)
361+
if experiment:
362+
experiment_name = experiment.name
339363

340364
# Prefer the version-specific path so the worker runs the exact
341365
# content the trial was created against.
@@ -375,6 +399,16 @@ async def _prepare_trial_run(
375399
trial_model=trial_model,
376400
trial_environment=trial_environment,
377401
trial_harbor_config=trial_harbor_config,
402+
task_name=task_name,
403+
experiment_id=experiment_id,
404+
experiment_name=experiment_name,
405+
# Extract trial index from trial_id ("{task_id}-{index}") for the
406+
# sauron attempt number. This is the trial's position within its
407+
# task (0, 1, 2...), NOT the retry count (trial.attempts).
408+
# Multiple trials of the same task must map to different attempt_N
409+
# folders to avoid overwriting each other.
410+
attempt_number=_extract_trial_index(trial_id, task_id) + 1, # 1-indexed
411+
task_tags=task_tags,
378412
)
379413

380414

@@ -784,6 +818,7 @@ async def run_trial_job(
784818

785819
# Upload trial results to S3.
786820
trial_s3_key = None
821+
oddish_uploaded = False
787822
if should_upload_to_s3 and execution.outcome and execution.outcome.job_dir:
788823
try:
789824
storage = get_storage_client()
@@ -793,12 +828,40 @@ async def run_trial_job(
793828
console.print(
794829
f"[dim]Uploaded trial results to S3: {trial_s3_key}[/dim]"
795830
)
796-
_cleanup_uploaded_job_dir(execution.outcome.job_dir, trial_id)
831+
oddish_uploaded = True
797832
except Exception as e:
798833
console.print(
799834
f"[yellow]Failed to upload trial results to S3: {e}[/yellow]"
800835
)
801836

837+
# Mirror to sauron's AWS S3 (best-effort).
838+
if execution.outcome and execution.outcome.job_dir:
839+
try:
840+
from oddish.integrations.sauron import get_sauron_uploader
841+
from oddish.integrations.github.client import GitHubMeta
842+
843+
sauron = get_sauron_uploader()
844+
if sauron.is_enabled():
845+
sauron_prefix = await sauron.upload_trial(
846+
harbor_job_dir=execution.outcome.job_dir,
847+
task_name=prepared_trial.task_name or prepared_trial.task_id,
848+
agent=prepared_trial.trial_agent,
849+
model=prepared_trial.trial_model,
850+
experiment_id=prepared_trial.experiment_id,
851+
experiment_name=prepared_trial.experiment_name,
852+
attempt_number=prepared_trial.attempt_number,
853+
github_meta=GitHubMeta.from_tags(prepared_trial.task_tags),
854+
task_tags=prepared_trial.task_tags,
855+
)
856+
if sauron_prefix:
857+
console.print(f"[dim]Mirrored to sauron S3: {sauron_prefix}[/dim]")
858+
except Exception as e:
859+
console.print(f"[yellow]Sauron mirror failed (non-fatal): {e}[/yellow]")
860+
861+
# Cleanup local Harbor artifacts AFTER both uploads complete.
862+
if oddish_uploaded and execution.outcome and execution.outcome.job_dir:
863+
_cleanup_uploaded_job_dir(execution.outcome.job_dir, trial_id)
864+
802865
await asyncio.shield(
803866
_store_trial_results(
804867
trial_id=trial_id,

0 commit comments

Comments
 (0)