Skip to content

Commit 661ad78

Browse files
author
endolinbot
committed
test(driver): mock-garden harness + design-only happy path end-to-end
Three test suites, 41 assertions total: - test_skeleton.sh (20 assertions): argument validation, state-file bootstrap on first run, transcript hashing, and the gardener-inbox section format (lane, PR, state, context, SHA all present; transcript blob retrievable via git cat-file blob). - test_design_only_happy_path.sh (17 assertions): drives the design-only-pr state machine end-to-end through seven ticks: initial -> build -> panel -> verdict -> un-draft -> await-maintainer -> approved+green -> merged. Each tick stubs the gh-pr-view JSON response, runs the driver in DRIVER_ONESHOT=1 mode, and verifies the state-file transition. The un-draft side effect is verified via the UN_DRAFT_STUB call log. - test_trap_fires_on_error.sh (4 assertions): a workflow name the driver does not recognize triggers the escalate path; the test verifies the gardener-inbox section is written and the driver parks rather than crashing. The harness (tests/driver/lib/mock-garden.sh) creates a temporary mock garden with: - A real git repo for the journal (so hash-object and update-ref work). - Copies of roles/driver/ and skills/gardener-inbox-error-reporting/ into the jail (so the driver's GARDEN_ROOT-relative resolution works). - Stub gh and stub claude on a prepended PATH. - Helper functions for setting gh-stub JSON per tick and asserting state-file fields. The harness teardown removes the temp dir, so the tests are hermetic and leave no garden-side state. tests/driver/run.sh runs all test_*.sh suites and reports aggregate pass/fail.
1 parent 6267e3d commit 661ad78

5 files changed

Lines changed: 581 additions & 0 deletions

File tree

‎tests/driver/lib/mock-garden.sh‎

Lines changed: 188 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,188 @@
1+
#!/bin/bash
2+
# mock-garden.sh -- set up a temporary mock garden for driver tests.
3+
#
4+
# Source this file from a test script. After sourcing:
5+
#
6+
# mock_garden_setup # creates a temp dir, exports MOCK_GARDEN_ROOT,
7+
# # MOCK_GARDEN_JOURNAL, and the env the driver expects
8+
# mock_garden_set_pr_json # writes a stubbed gh-pr-view JSON to GH_STUB
9+
# mock_garden_run_driver # invokes driver.sh under the mock env
10+
# mock_garden_teardown # removes the temp dir
11+
#
12+
# All paths are absolute; the harness sets cwd to a known location and
13+
# returns to the caller's cwd on teardown.
14+
15+
# Resolve the real garden root from this script's location: this file
16+
# lives at tests/driver/lib/mock-garden.sh, so .../../.. is the garden.
17+
MOCK_HARNESS_DIR=$(cd "$(dirname "${BASH_SOURCE[0]:-$0}")" && pwd)
18+
REAL_GARDEN_ROOT=$(cd "$MOCK_HARNESS_DIR/../../.." && pwd)
19+
20+
mock_garden_setup() {
21+
MOCK_TMP=$(mktemp -d -t driver-mock-XXXXXX)
22+
MOCK_GARDEN_ROOT="$MOCK_TMP/garden"
23+
MOCK_GARDEN_JOURNAL="$MOCK_GARDEN_ROOT/journal"
24+
MOCK_STUBS="$MOCK_TMP/stubs"
25+
MOCK_BIN="$MOCK_TMP/bin"
26+
MOCK_LOGS="$MOCK_TMP/logs"
27+
28+
mkdir -p "$MOCK_GARDEN_ROOT" "$MOCK_STUBS" "$MOCK_BIN" "$MOCK_LOGS"
29+
30+
# 1. Copy roles/driver/ and the skills the driver invokes into the
31+
# mock garden. We do not copy the entire real garden; that would
32+
# drag in the journal as a git submodule. The mock harness needs
33+
# only the artifacts the driver references at runtime.
34+
cp -r "$REAL_GARDEN_ROOT/roles/driver" "$MOCK_GARDEN_ROOT/" 2>/dev/null || \
35+
cp -r "$REAL_GARDEN_ROOT/roles/driver" "$MOCK_GARDEN_ROOT/roles-driver-tmp"
36+
# The mkdir+cp pattern handles the case where roles/driver did not
37+
# yet exist (first-run); the test fails fast if so. We unify here:
38+
mkdir -p "$MOCK_GARDEN_ROOT/roles"
39+
if [ -d "$MOCK_GARDEN_ROOT/roles-driver-tmp" ]; then
40+
mv "$MOCK_GARDEN_ROOT/roles-driver-tmp" "$MOCK_GARDEN_ROOT/roles/driver"
41+
elif [ -d "$MOCK_GARDEN_ROOT/driver" ]; then
42+
mv "$MOCK_GARDEN_ROOT/driver" "$MOCK_GARDEN_ROOT/roles/driver"
43+
fi
44+
45+
# Copy the skills the driver invokes.
46+
mkdir -p "$MOCK_GARDEN_ROOT/skills"
47+
cp -r "$REAL_GARDEN_ROOT/skills/gardener-inbox-error-reporting" \
48+
"$MOCK_GARDEN_ROOT/skills/" 2>/dev/null || true
49+
cp -r "$REAL_GARDEN_ROOT/skills/job-board" \
50+
"$MOCK_GARDEN_ROOT/skills/" 2>/dev/null || true
51+
52+
# 2. Initialize a real git repo for the mock journal so hash-object
53+
# and update-ref work.
54+
mkdir -p "$MOCK_GARDEN_JOURNAL"
55+
( cd "$MOCK_GARDEN_JOURNAL" && \
56+
git init --quiet --initial-branch=journal && \
57+
git config user.name "mock-bot" && \
58+
git config user.email "mock@example.invalid" && \
59+
mkdir -p inboxes drivers entries jobs/open jobs/claimed jobs/done jobs/abandoned && \
60+
touch inboxes/.gitkeep drivers/.gitkeep entries/.gitkeep \
61+
jobs/open/.gitkeep jobs/claimed/.gitkeep jobs/done/.gitkeep jobs/abandoned/.gitkeep && \
62+
git add -A && \
63+
git commit --quiet -m "mock: initialize journal" )
64+
65+
# 3. Build PATH stubs for gh, claude, and other binaries the driver
66+
# might invoke. The stubs read from MOCK_STUBS/<name>.expected
67+
# and write a transcript line to MOCK_LOGS/<name>.log.
68+
cat > "$MOCK_BIN/gh" <<'EOF'
69+
#!/bin/bash
70+
# gh stub for driver tests.
71+
echo "gh $*" >> "${MOCK_LOGS:-/tmp}/gh.log"
72+
case "$1 $2" in
73+
"pr view")
74+
if [ -n "${GH_STUB:-}" ] && [ -f "$GH_STUB" ]; then
75+
cat "$GH_STUB"
76+
else
77+
echo "{}"
78+
fi
79+
;;
80+
"pr ready")
81+
echo "stub: gh pr ready ${@:3}" >> "${MOCK_LOGS:-/tmp}/gh-ready.log"
82+
exit 0
83+
;;
84+
*)
85+
echo "stub: gh unrecognized: $*" >> "${MOCK_LOGS:-/tmp}/gh-unhandled.log"
86+
exit 0
87+
;;
88+
esac
89+
EOF
90+
chmod +x "$MOCK_BIN/gh"
91+
92+
cat > "$MOCK_BIN/claude" <<'EOF'
93+
#!/bin/bash
94+
# claude stub for driver tests.
95+
echo "claude invoked with $*" >> "${MOCK_LOGS:-/tmp}/claude.log"
96+
# Default: write a "no-op" response.
97+
if [ "${CLAUDE_STUB_RESPONSE:-}" ]; then
98+
printf '%s' "$CLAUDE_STUB_RESPONSE"
99+
else
100+
echo '{"verdict":"escalated","action":"no-op"}'
101+
fi
102+
EOF
103+
chmod +x "$MOCK_BIN/claude"
104+
105+
# 4. Export env for the driver under test.
106+
export GARDEN_ROOT="$MOCK_GARDEN_ROOT"
107+
export GARDEN_JOURNAL="$MOCK_GARDEN_JOURNAL"
108+
export GARDEN_HOST="mock-host"
109+
export MOCK_LOGS
110+
export PATH="$MOCK_BIN:$PATH"
111+
112+
return 0
113+
}
114+
115+
mock_garden_set_pr_json() {
116+
local body=$1
117+
: > "$MOCK_TMP/pr.json"
118+
printf '%s' "$body" > "$MOCK_TMP/pr.json"
119+
export GH_STUB="$MOCK_TMP/pr.json"
120+
}
121+
122+
mock_garden_clear_pr_json() {
123+
unset GH_STUB
124+
rm -f "$MOCK_TMP/pr.json"
125+
}
126+
127+
mock_garden_run_driver() {
128+
local lane=$1
129+
shift
130+
bash "$MOCK_GARDEN_ROOT/roles/driver/driver.sh" "$lane" "$@"
131+
}
132+
133+
mock_garden_teardown() {
134+
if [ -n "${MOCK_TMP:-}" ] && [ -d "$MOCK_TMP" ]; then
135+
rm -rf "$MOCK_TMP"
136+
fi
137+
unset MOCK_TMP MOCK_GARDEN_ROOT MOCK_GARDEN_JOURNAL MOCK_STUBS MOCK_BIN MOCK_LOGS
138+
unset GARDEN_ROOT GARDEN_JOURNAL GARDEN_HOST GH_STUB
139+
}
140+
141+
# Pretty-print helpers used by tests.
142+
mock_garden_assert_eq() {
143+
local label=$1
144+
local expected=$2
145+
local actual=$3
146+
if [ "$expected" = "$actual" ]; then
147+
echo " PASS: $label"
148+
return 0
149+
fi
150+
echo " FAIL: $label"
151+
echo " expected: $expected"
152+
echo " actual: $actual"
153+
return 1
154+
}
155+
156+
mock_garden_assert_contains() {
157+
local label=$1
158+
local needle=$2
159+
local haystack=$3
160+
if printf '%s' "$haystack" | grep -q -F "$needle"; then
161+
echo " PASS: $label"
162+
return 0
163+
fi
164+
echo " FAIL: $label"
165+
echo " expected to contain: $needle"
166+
echo " actual: $haystack"
167+
return 1
168+
}
169+
170+
mock_garden_assert_file_contains() {
171+
local label=$1
172+
local needle=$2
173+
local path=$3
174+
if [ ! -f "$path" ]; then
175+
echo " FAIL: $label"
176+
echo " file does not exist: $path"
177+
return 1
178+
fi
179+
if grep -q -F "$needle" "$path"; then
180+
echo " PASS: $label"
181+
return 0
182+
fi
183+
echo " FAIL: $label"
184+
echo " expected $path to contain: $needle"
185+
echo " actual contents:"
186+
sed 's/^/ /' "$path"
187+
return 1
188+
}

‎tests/driver/run.sh‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
#!/bin/bash
2+
# run.sh -- run all driver tests and report aggregate results.
3+
4+
set -u
5+
6+
HARNESS_DIR=$(cd "$(dirname "$0")" && pwd)
7+
8+
OVERALL_PASS=0
9+
OVERALL_FAIL=0
10+
FAILING_TESTS=()
11+
12+
for test in "$HARNESS_DIR"/test_*.sh; do
13+
[ -f "$test" ] || continue
14+
echo
15+
echo ">>> Running $(basename "$test")"
16+
if bash "$test"; then
17+
OVERALL_PASS=$((OVERALL_PASS+1))
18+
else
19+
OVERALL_FAIL=$((OVERALL_FAIL+1))
20+
FAILING_TESTS+=("$(basename "$test")")
21+
fi
22+
done
23+
24+
echo
25+
echo "================================================================"
26+
echo "Driver tests: $OVERALL_PASS suite(s) passed, $OVERALL_FAIL failed"
27+
if [ "$OVERALL_FAIL" -gt 0 ]; then
28+
echo "Failing suites:"
29+
for t in "${FAILING_TESTS[@]}"; do
30+
echo " - $t"
31+
done
32+
exit 1
33+
fi
34+
exit 0

‎tests/driver/test_design_only_happy_path.sh‎

Lines changed: 144 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,144 @@
1+
#!/bin/bash
2+
# test_design_only_happy_path.sh -- end-to-end happy path for the
3+
# design-only PR workflow.
4+
#
5+
# The driver is invoked once per state transition (DRIVER_ONESHOT=1
6+
# repeatedly) with the gh stub set up to return the appropriate JSON at
7+
# each step. The test verifies the state file advances through:
8+
#
9+
# initial -> build -> panel -> verdict -> un-draft -> await-maintainer
10+
# -> approved+green -> merged
11+
#
12+
# Per-tick stubs:
13+
# - tick 1: initial -> build (DRIVER_PR set, so transition is direct)
14+
# - tick 2: build -> panel (PR JSON says state=OPEN, isDraft=true)
15+
# - tick 3: panel -> verdict (PR JSON includes a kriscendobot review)
16+
# - tick 4: verdict -> un-draft (review body contains "verdict: approve")
17+
# - tick 5: un-draft -> await-maintainer (deterministic; gh pr ready stub no-ops)
18+
# - tick 6: await-maintainer -> approved+green (PR JSON includes APPROVED review)
19+
# - tick 7: approved+green -> merged (PR JSON state=MERGED)
20+
21+
set -uo pipefail
22+
23+
HARNESS_DIR=$(cd "$(dirname "$0")" && pwd)
24+
source "$HARNESS_DIR/lib/mock-garden.sh"
25+
26+
PASS=0
27+
FAIL=0
28+
ok() { PASS=$((PASS+1)); }
29+
ko() { FAIL=$((FAIL+1)); }
30+
run_assert() { if "$@"; then ok; else ko; fi; }
31+
32+
echo "=== test_design_only_happy_path ==="
33+
34+
mock_garden_setup
35+
36+
# Common env for every tick.
37+
export DRIVER_PR="mock/repo#42"
38+
export DRIVER_WORKFLOW="design-only-pr"
39+
export DRIVER_ONESHOT=1
40+
export DRIVER_TICK_SECONDS=0
41+
42+
# Stub post_job so the driver does not try to git push into our mock
43+
# journal (which has no `origin`).
44+
stub_post_job_file="$MOCK_TMP/post-job-stub.sh"
45+
cat > "$stub_post_job_file" <<'EOF'
46+
#!/bin/bash
47+
echo "jobs/open/stubbed--$1--$2.md"
48+
exit 0
49+
EOF
50+
chmod +x "$stub_post_job_file"
51+
export POST_JOB_STUB="$stub_post_job_file"
52+
53+
# Stub the un-draft action so it does not call real gh.
54+
stub_un_draft_file="$MOCK_TMP/un-draft-stub.sh"
55+
cat > "$stub_un_draft_file" <<'EOF'
56+
#!/bin/bash
57+
echo "un-draft stub called for: $1" >> "${MOCK_LOGS:-/tmp}/un-draft.log"
58+
exit 0
59+
EOF
60+
chmod +x "$stub_un_draft_file"
61+
export UN_DRAFT_STUB="$stub_un_draft_file"
62+
63+
run_driver() {
64+
bash "$MOCK_GARDEN_ROOT/roles/driver/driver.sh" 1
65+
}
66+
67+
read_state_field() {
68+
local field=$1
69+
local state_file="$MOCK_GARDEN_JOURNAL/drivers/mock-host/1.md"
70+
# Use sed so we don't split on internal colons (e.g. "solicitor:design-panel").
71+
sed -n "s/^$field: //p" "$state_file" 2>/dev/null | head -1
72+
}
73+
74+
# --- tick 1: initial -> build ----------------------------------
75+
mock_garden_set_pr_json '{"state":"OPEN","isDraft":true,"reviews":[]}'
76+
set +e; run_driver >/dev/null 2>&1; rc=$?; set -e
77+
run_assert mock_garden_assert_eq "tick 1 rc=0" 0 "$rc"
78+
run_assert mock_garden_assert_eq "tick 1 -> state=build" "build" "$(read_state_field state)"
79+
80+
# --- tick 2: build -> panel ------------------------------------
81+
mock_garden_set_pr_json '{"state":"OPEN","isDraft":true,"reviews":[]}'
82+
set +e; run_driver >/dev/null 2>&1; rc=$?; set -e
83+
run_assert mock_garden_assert_eq "tick 2 rc=0" 0 "$rc"
84+
run_assert mock_garden_assert_eq "tick 2 -> state=panel" "panel" "$(read_state_field state)"
85+
run_assert mock_garden_assert_eq "tick 2 -> awaits=solicitor:design-panel" "solicitor:design-panel" "$(read_state_field awaits)"
86+
87+
# --- tick 3: panel -> verdict ----------------------------------
88+
# Need a PR JSON whose grep for "kriscendobot" matches (the predicate
89+
# uses a substring grep on the raw JSON for Phase 2 simplicity).
90+
mock_garden_set_pr_json '{"state":"OPEN","isDraft":true,"reviews":[{"author":{"login":"kriscendobot"},"body":"verdict: pending classification"}]}'
91+
set +e; run_driver >/dev/null 2>&1; rc=$?; set -e
92+
run_assert mock_garden_assert_eq "tick 3 rc=0" 0 "$rc"
93+
run_assert mock_garden_assert_eq "tick 3 -> state=verdict" "verdict" "$(read_state_field state)"
94+
95+
# --- tick 4: verdict -> un-draft -------------------------------
96+
# Review body contains the deterministic "verdict: approve" marker.
97+
mock_garden_set_pr_json '{"state":"OPEN","isDraft":true,"reviews":[{"author":{"login":"kriscendobot"},"body":"verdict: approve"}]}'
98+
set +e; run_driver >/dev/null 2>&1; rc=$?; set -e
99+
run_assert mock_garden_assert_eq "tick 4 rc=0" 0 "$rc"
100+
run_assert mock_garden_assert_eq "tick 4 -> state=un-draft" "un-draft" "$(read_state_field state)"
101+
102+
# --- tick 5: un-draft -> await-maintainer ----------------------
103+
mock_garden_set_pr_json '{"state":"OPEN","isDraft":false,"reviews":[]}'
104+
set +e; run_driver >/dev/null 2>&1; rc=$?; set -e
105+
run_assert mock_garden_assert_eq "tick 5 rc=0" 0 "$rc"
106+
run_assert mock_garden_assert_eq "tick 5 -> state=await-maintainer" "await-maintainer" "$(read_state_field state)"
107+
108+
# Verify un-draft stub was called.
109+
run_assert mock_garden_assert_file_contains "un-draft stub called for PR id" "mock/repo#42" "$MOCK_LOGS/un-draft.log"
110+
111+
# --- tick 6: await-maintainer -> approved+green ----------------
112+
mock_garden_set_pr_json '{"state":"OPEN","isDraft":false,"reviews":[{"author":{"login":"kriskowal"},"state":"APPROVED","body":"lgtm"}]}'
113+
set +e; run_driver >/dev/null 2>&1; rc=$?; set -e
114+
run_assert mock_garden_assert_eq "tick 6 rc=0" 0 "$rc"
115+
run_assert mock_garden_assert_eq "tick 6 -> state=approved+green" "approved+green" "$(read_state_field state)"
116+
117+
# --- tick 7: approved+green -> merged --------------------------
118+
mock_garden_set_pr_json '{"state":"MERGED","isDraft":false,"reviews":[]}'
119+
set +e; run_driver >/dev/null 2>&1; rc=$?; set -e
120+
run_assert mock_garden_assert_eq "tick 7 rc=0 (terminal exit)" 0 "$rc"
121+
run_assert mock_garden_assert_eq "tick 7 -> state=merged" "merged" "$(read_state_field state)"
122+
123+
# Verify no gardener-inbox failure section was written. The whole
124+
# happy path should be silent on the gardener inbox.
125+
inbox_file="$MOCK_GARDEN_JOURNAL/inboxes/mock-host/gardener.md"
126+
if [ -f "$inbox_file" ]; then
127+
if grep -q "driver lane 1" "$inbox_file"; then
128+
echo " FAIL: gardener inbox has a section for lane 1 (happy path should be silent)"
129+
ko
130+
else
131+
ok
132+
fi
133+
else
134+
# File never created: also OK for a clean happy path.
135+
ok
136+
fi
137+
138+
mock_garden_teardown
139+
140+
echo "=== test_design_only_happy_path: $PASS passed, $FAIL failed ==="
141+
if [ "$FAIL" -gt 0 ]; then
142+
exit 1
143+
fi
144+
exit 0

0 commit comments

Comments
 (0)