Skip to content

Commit 3165f0c

Browse files
committed
merge: universal event architecture
Generalize mecha's event system from git-specific to provider-neutral, enabling any webhook source, active trigger, or scheduled event to flow through the same pipeline. Architecture: - Event struct: Actor, Subject, Attrs (replaces RepoOwner/Name/Ref/Number/Sender) - 5 provider interfaces: Source, Trigger, Hydrator, Verifier, Responder - Authenticated marker for webhook auth bypass control - Responder registry for provider-agnostic write-back - DB migration v3 (transactional table swap) - Crash recovery for stuck received events Quality: - 23 Codex audit findings fixed (1 critical, 8 high, 9 medium, 5 low) - 5 grill paranoid-mode agents, all findings addressed - 3 bugs found and fixed from cross-round analysis - 310 tests (was 166), serve coverage 77.2% (was 55%) - Disposable container lifecycle tested with real Docker - Documentation audit: all 4 dimensions at 100% 13 commits, 34 files changed
2 parents d664bef + c3ea281 commit 3165f0c

37 files changed

Lines changed: 2946 additions & 353 deletions

.claude/rules/domain-model.md

Lines changed: 26 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,12 +10,35 @@ Mecha has exactly four nouns: Event, Worker, Task, Policy.
1010
## Pipeline
1111

1212
```
13-
Event.arrive → Event.match → Task.create → Task.dispatch → Policy.filter → Task.complete
13+
Event.arrive → Event.match → Task.create → Task.dispatch → Policy.filter → Respond
1414
```
1515

16+
## Event Model (Universal)
17+
18+
Events are provider-neutral. All provider-specific data lives in `Attrs`.
19+
20+
| Field | Purpose | Example |
21+
|---|---|---|
22+
| `Source` | Provider name | `github`, `slack`, `telegram`, `cron` |
23+
| `Type` | Event type | `pull_request.opened`, `message`, `tick` |
24+
| `Actor` | Who triggered | username, phone, bot name |
25+
| `Subject` | What it's about | `owner/repo`, `#channel`, `schedule-daily` |
26+
| `Attrs` | Provider-specific fields | `repo_owner`, `number`, `diff`, `text` |
27+
| `DedupKey` | Semantic dedup | Content hash for polls/cron |
28+
29+
## Provider Interfaces
30+
31+
| Interface | Direction | Purpose |
32+
|---|---|---|
33+
| `Source` | Inbound (passive) | Parse webhooks into Events |
34+
| `Trigger` | Inbound (active) | Generate events (cron, polling) |
35+
| `Hydrator` | Enrichment | Fetch additional data via API |
36+
| `Verifier` | Handshake | Challenge-response verification |
37+
| `Responder` | Outbound | Write results back to platform |
38+
1639
## Nouns and Verbs
1740

18-
- **Event**: arrive, match
41+
- **Event**: arrive, match, hydrate, skip, fail
1942
- **Worker**: add, remove, start, stop, ls
2043
- **Task**: create, dispatch, complete, fail
2144
- **Policy**: filter
@@ -28,3 +51,4 @@ Event.arrive → Event.match → Task.create → Task.dispatch → Policy.filter
2851
- One noun, one lifecycle.
2952
- No hidden nouns. Unowned logic means a missing noun.
3053
- Verbs are idempotent where possible.
54+
- Responder is keyed by target, not source (ingress != egress).

.claude/rules/result-contract.md

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ globs: "**/*.go"
55

66
# Result Contract
77

8-
> Design spec. Implemented in TypeScript (`docker/runtime/types.ts`). No Go struct yet — Phase 3.
8+
> Implemented in Go (`internal/policy/result.go`) and TypeScript (`docker/runtime/types.ts`).
99
1010
Every worker returns the same structure. Every field optional.
1111

@@ -25,9 +25,19 @@ Every worker returns the same structure. Every field optional.
2525
}
2626
```
2727

28+
## Write-Back
29+
30+
Write-back is routed through the Responder registry, keyed by target platform:
31+
32+
- `source.Responder` interface: `Name() string`, `Respond(ctx, ev, result) error`
33+
- GitHub writeback.Client implements Responder (registered automatically)
34+
- Dispatch tries Responder registry first, falls back to legacy writeback
35+
- Responder is looked up by `ev.Source` (e.g., GitHub events use the GitHub responder)
36+
2837
## Rules
2938

3039
- Result and side effects are one thing. No separate "action" phase.
3140
- Policy filters the result before write-back. Denied fields are dropped.
3241
- A completed task means its result has already been written back.
3342
- The worker decides what to return. Policy decides what gets through.
43+
- Responder is looked up by `ev.Source`. Target override is planned but not yet implemented.

internal/cli/serve.go

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -78,10 +78,11 @@ func serveCmd() *cobra.Command {
7878
logger.Info("gitlab source registered")
7979
}
8080

81-
// Write-back client
81+
// Write-back client + responder registration
8282
var wb *writeback.Client
8383
if ghToken != "" {
8484
wb = writeback.NewClient(ghToken, logger)
85+
sources.RegisterResponder(wb)
8586
}
8687

8788
srv := serve.New(serve.Config{

internal/event/helpers.go

Lines changed: 10 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -13,11 +13,11 @@ type rowScanner interface{ Scan(dest ...any) error }
1313

1414
func scanEvent(s rowScanner) (*Event, error) {
1515
var ev Event
16-
var state, payloadStr, raw, wn, tid string
16+
var state, attrsStr, raw, wn, tid string
1717
var createdAt, updatedAt int64
18-
err := s.Scan(&ev.ID, &ev.DeliveryID, &ev.Source, &ev.Type,
19-
&ev.RepoOwner, &ev.RepoName, &ev.Ref, &ev.Number, &ev.Sender,
20-
&payloadStr, &raw, &state, &wn, &tid, &createdAt, &updatedAt)
18+
err := s.Scan(&ev.ID, &ev.DeliveryID, &ev.DedupKey,
19+
&ev.Source, &ev.Type, &ev.Actor, &ev.Subject,
20+
&attrsStr, &raw, &state, &wn, &tid, &createdAt, &updatedAt)
2121
if err != nil {
2222
if err == sql.ErrNoRows {
2323
return nil, fmt.Errorf("event not found")
@@ -30,11 +30,14 @@ func scanEvent(s rowScanner) (*Event, error) {
3030
ev.Raw = json.RawMessage(raw)
3131
ev.CreatedAt = time.Unix(createdAt, 0)
3232
ev.UpdatedAt = time.Unix(updatedAt, 0)
33-
if payloadStr != "" {
34-
if err := json.Unmarshal([]byte(payloadStr), &ev.Payload); err != nil {
35-
return nil, fmt.Errorf("unmarshal event payload: %w", err)
33+
if attrsStr != "" && attrsStr != "null" {
34+
if err := json.Unmarshal([]byte(attrsStr), &ev.Attrs); err != nil {
35+
return nil, fmt.Errorf("unmarshal event attrs: %w", err)
3636
}
3737
}
38+
if ev.Attrs == nil {
39+
ev.Attrs = make(Attrs)
40+
}
3841
return &ev, nil
3942
}
4043

internal/event/store.go

Lines changed: 49 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,9 @@ func NewStore(db *sql.DB) *Store {
1818
return &Store{db: db}
1919
}
2020

21+
const eventColumns = `id, delivery_id, dedup_key, source, type, actor, subject,
22+
attrs, raw, state, worker_name, task_id, created_at, updated_at`
23+
2124
// Create persists a new event in received state.
2225
func (s *Store) Create(ctx context.Context, ev *Event) error {
2326
if ev.ID == "" {
@@ -32,17 +35,17 @@ func (s *Store) Create(ctx context.Context, ev *Event) error {
3235
ev.CreatedAt = now
3336
ev.UpdatedAt = now
3437

35-
payload, err := json.Marshal(ev.Payload)
38+
attrs, err := json.Marshal(ev.Attrs)
3639
if err != nil {
37-
return fmt.Errorf("marshal event payload: %w", err)
40+
return fmt.Errorf("marshal event attrs: %w", err)
3841
}
3942

4043
_, err = s.db.ExecContext(ctx,
41-
`INSERT INTO events (id, delivery_id, source, type, repo_owner, repo_name,
42-
ref, number, sender, payload, raw, state, created_at, updated_at)
43-
VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?)`,
44-
ev.ID, ev.DeliveryID, ev.Source, ev.Type, ev.RepoOwner, ev.RepoName,
45-
ev.Ref, ev.Number, ev.Sender, string(payload), string(ev.Raw),
44+
`INSERT INTO events (id, delivery_id, dedup_key, source, type, actor, subject,
45+
attrs, raw, state, created_at, updated_at)
46+
VALUES (?,?,?,?,?,?,?,?,?,?,?,?)`,
47+
ev.ID, ev.DeliveryID, ev.DedupKey, ev.Source, ev.Type, ev.Actor, ev.Subject,
48+
string(attrs), string(ev.Raw),
4649
string(ev.State), now.Unix(), now.Unix(),
4750
)
4851
if err != nil {
@@ -69,17 +72,13 @@ func (s *Store) DeliveryExists(ctx context.Context, deliveryID string) (bool, er
6972
// Get retrieves an event by ID.
7073
func (s *Store) Get(ctx context.Context, id string) (*Event, error) {
7174
row := s.db.QueryRowContext(ctx,
72-
`SELECT id, delivery_id, source, type, repo_owner, repo_name,
73-
ref, number, sender, payload, raw, state, worker_name, task_id,
74-
created_at, updated_at FROM events WHERE id = ?`, id)
75+
`SELECT `+eventColumns+` FROM events WHERE id = ?`, id)
7576
return scanEvent(row)
7677
}
7778

7879
// List returns events, optionally filtered by state.
7980
func (s *Store) List(ctx context.Context, state string) ([]Event, error) {
80-
query := `SELECT id, delivery_id, source, type, repo_owner, repo_name,
81-
ref, number, sender, payload, raw, state, worker_name, task_id,
82-
created_at, updated_at FROM events`
81+
query := `SELECT ` + eventColumns + ` FROM events`
8382
var args []any
8483
if state != "" {
8584
query += " WHERE state = ?"
@@ -102,6 +101,22 @@ func (s *Store) List(ctx context.Context, state string) ([]Event, error) {
102101
return events, rows.Err()
103102
}
104103

104+
// UpdateAttrs persists hydrated attrs back to the database.
105+
func (s *Store) UpdateAttrs(ctx context.Context, id string, attrs Attrs) error {
106+
data, err := json.Marshal(attrs)
107+
if err != nil {
108+
return fmt.Errorf("marshal attrs: %w", err)
109+
}
110+
now := time.Now().Unix()
111+
res, err := s.db.ExecContext(ctx,
112+
"UPDATE events SET attrs = ?, updated_at = ? WHERE id = ?",
113+
string(data), now, id)
114+
if err != nil {
115+
return fmt.Errorf("update attrs: %w", err)
116+
}
117+
return checkRows(res, id, "update attrs")
118+
}
119+
105120
// SetMatched transitions received → matched with worker name.
106121
func (s *Store) SetMatched(ctx context.Context, id, workerName string) error {
107122
return s.transitionWithExtra(ctx, id, StateReceived, StateMatched, workerName,
@@ -138,6 +153,27 @@ func (s *Store) SetSkipped(ctx context.Context, id string) error {
138153
"UPDATE events SET state = ?, updated_at = ? WHERE id = ? AND state = ?")
139154
}
140155

156+
// Received returns IDs of events stuck in received state (for crash recovery).
157+
func (s *Store) Received(ctx context.Context) ([]string, error) {
158+
rows, err := s.db.QueryContext(ctx,
159+
`SELECT id FROM events WHERE state = ? ORDER BY created_at`,
160+
string(StateReceived),
161+
)
162+
if err != nil {
163+
return nil, fmt.Errorf("query received: %w", err)
164+
}
165+
defer rows.Close()
166+
var ids []string
167+
for rows.Next() {
168+
var id string
169+
if err := rows.Scan(&id); err != nil {
170+
return nil, err
171+
}
172+
ids = append(ids, id)
173+
}
174+
return ids, rows.Err()
175+
}
176+
141177
func (s *Store) transition(ctx context.Context, id string, from, to State, query string) error {
142178
now := time.Now().Unix()
143179
res, err := s.db.ExecContext(ctx, query, string(to), now, id, string(from))
@@ -155,4 +191,3 @@ func (s *Store) transitionWithExtra(ctx context.Context, id string, from, to Sta
155191
}
156192
return checkRows(res, id, string(to))
157193
}
158-

internal/event/store_test.go

Lines changed: 48 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -22,14 +22,12 @@ func testStore(t *testing.T) *Store {
2222
func TestCreateAndGet(t *testing.T) {
2323
s := testStore(t)
2424
ev := &Event{
25-
Source: "github",
26-
Type: "pull_request.opened",
27-
RepoOwner: "owner",
28-
RepoName: "repo",
29-
Number: 42,
30-
Sender: "user",
31-
Payload: Payload{"title": "Fix bug"},
32-
Raw: json.RawMessage(`{"action":"opened"}`),
25+
Source: "github",
26+
Type: "pull_request.opened",
27+
Actor: "user",
28+
Subject: "owner/repo",
29+
Attrs: Attrs{"title": "Fix bug", "number": 42, "repo_owner": "owner", "repo_name": "repo"},
30+
Raw: json.RawMessage(`{"action":"opened"}`),
3331
}
3432
if err := s.Create(context.Background(), ev); err != nil {
3533
t.Fatal(err)
@@ -48,8 +46,14 @@ func TestCreateAndGet(t *testing.T) {
4846
if got.State != StateReceived {
4947
t.Errorf("state = %q", got.State)
5048
}
51-
if got.Payload["title"] != "Fix bug" {
52-
t.Errorf("payload = %v", got.Payload)
49+
if got.Attrs["title"] != "Fix bug" {
50+
t.Errorf("attrs = %v", got.Attrs)
51+
}
52+
if got.Actor != "user" {
53+
t.Errorf("actor = %q", got.Actor)
54+
}
55+
if got.Subject != "owner/repo" {
56+
t.Errorf("subject = %q", got.Subject)
5357
}
5458
}
5559

@@ -137,3 +141,37 @@ func TestGetNotFound(t *testing.T) {
137141
t.Error("expected error")
138142
}
139143
}
144+
145+
func TestUpdateAttrs(t *testing.T) {
146+
s := testStore(t)
147+
ctx := context.Background()
148+
ev := &Event{Source: "github", Type: "push", Attrs: Attrs{"a": "1"}}
149+
s.Create(ctx, ev)
150+
151+
err := s.UpdateAttrs(ctx, ev.ID, Attrs{"a": "1", "diff": "some diff"})
152+
if err != nil {
153+
t.Fatal(err)
154+
}
155+
got, _ := s.Get(ctx, ev.ID)
156+
if got.Attrs["diff"] != "some diff" {
157+
t.Errorf("attrs = %v", got.Attrs)
158+
}
159+
}
160+
161+
func TestReceived(t *testing.T) {
162+
s := testStore(t)
163+
ctx := context.Background()
164+
ev1 := &Event{Source: "github", Type: "push"}
165+
ev2 := &Event{Source: "slack", Type: "message"}
166+
s.Create(ctx, ev1)
167+
s.Create(ctx, ev2)
168+
s.SetMatched(ctx, ev2.ID, "w")
169+
170+
ids, err := s.Received(ctx)
171+
if err != nil {
172+
t.Fatal(err)
173+
}
174+
if len(ids) != 1 || ids[0] != ev1.ID {
175+
t.Errorf("received = %v, want [%s]", ids, ev1.ID)
176+
}
177+
}

internal/event/types.go

Lines changed: 18 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -9,29 +9,35 @@ import (
99
type State string
1010

1111
const (
12-
StateReceived State = "received"
13-
StateMatched State = "matched"
12+
// StateReceived means the event arrived and was persisted, not yet matched.
13+
StateReceived State = "received"
14+
// StateMatched means the event matched a worker by event rules.
15+
StateMatched State = "matched"
16+
// StateDispatched means a task was created and sent to a worker.
1417
StateDispatched State = "dispatched"
18+
// StateCompleted means the worker returned a result and write-back succeeded.
1519
StateCompleted State = "completed"
16-
StateFailed State = "failed"
17-
StateSkipped State = "skipped"
20+
// StateFailed means processing errored or timed out.
21+
StateFailed State = "failed"
22+
// StateSkipped means no matching worker was found for this event.
23+
StateSkipped State = "skipped"
1824
)
1925

20-
// Payload is a map of enriched fields available to prompt templates.
21-
type Payload map[string]any
26+
// Attrs holds provider-specific fields available to prompt templates.
27+
type Attrs map[string]any
2228

2329
// Event is something that happened from an external source.
30+
// All provider-specific data lives in Attrs — the struct itself
31+
// contains only universal, provider-neutral fields.
2432
type Event struct {
2533
ID string `json:"id"`
2634
DeliveryID string `json:"delivery_id"`
35+
DedupKey string `json:"dedup_key,omitempty"` // reserved for semantic dedup (cron/polling); not yet enforced
2736
Source string `json:"source"`
2837
Type string `json:"type"`
29-
RepoOwner string `json:"repo_owner"`
30-
RepoName string `json:"repo_name"`
31-
Ref string `json:"ref"`
32-
Number int `json:"number"`
33-
Sender string `json:"sender"`
34-
Payload Payload `json:"payload"`
38+
Actor string `json:"actor"`
39+
Subject string `json:"subject"`
40+
Attrs Attrs `json:"attrs"`
3541
Raw json.RawMessage `json:"raw,omitempty"`
3642
State State `json:"state"`
3743
WorkerName string `json:"worker_name,omitempty"`

0 commit comments

Comments
 (0)