fix: correct bash installer exit code and isolate live eval auth

install-package.sh reported failure (exit 1) on a successful normal
install or --clean reinstall, because the EXIT trap's cleanup check
used `[ -d "$stage" ] && rm -rf "$stage"` as a bare statement: when
$stage no longer exists (the normal success case), that false test
is exempt from set -e but its own exit status becomes the trap's
return value, which becomes the script's final exit code. Every bash
wrapper script ends by calling install-package.sh as its last
statement, so this silently broke default-mode and --clean install
reporting for all callers. Fixed with explicit if/fi. Caught by a new
scripts/test-installers.sh bash test suite mirroring the existing
PowerShell coverage.

run-behavioral-evals.mjs left CODEX_HOME pointed at the real user's
.codex directory even though HOME/USERPROFILE were overridden to a
disposable sandbox, so live two-pass eval runs could inherit real
skill and session state instead of the isolated fixture copy. Fixed
by pointing CODEX_HOME at the disposable home. A fully empty
disposable home cannot authenticate, so the runner now seeds it with
a copy of the real auth.json only, nothing else. Verified against a
real codex exec run: the failure mode moved from a missing-directory
error, to 401 Unauthorized, to a real account usage-limit response,
confirming both the isolation and the authentication seeding work as
intended. A full live pass/fail verdict for the two named cases is
still blocked by that account-level usage limit, unrelated to this
code.

Author: Shaelz

CHANGELOG.md

@@ -21,7 +21,32 @@
 - Extend the existing behavioral-evaluation runner with the authored
   no-date-only-churn and source-drift two-pass cases, an optional model
   override, and stronger tested-skill isolation evidence.
-- Add installer behavior tests covering both managed packages.
+- Add installer behavior tests covering both managed packages, including a
+  new Bash test suite (`scripts/test-installers.sh`) mirroring the existing
+  PowerShell coverage.
+- Fix a Bash installer defect where a successful normal install or `--clean`
+  reinstall printed a success message but exited with status 1, because the
+  `EXIT` trap's cleanup check used `[ -d "$stage" ] && rm -rf "$stage"` as a
+  bare statement; a false test in that position is exempt from `set -e` but
+  still sets the trap function's own exit status, which becomes the script's
+  final exit code. Every Bash wrapper script (`install-user.sh`,
+  `install-project.sh`, `install-codex-user.sh`, `install-codex-project.sh`,
+  `install-bootstrap-user.sh`) ends by calling `install-package.sh` as its
+  last statement, so this previously caused those default-mode and
+  `--clean` invocations to report failure to callers despite installing
+  correctly. Fixed by using explicit `if`/`fi` instead of `&&` for both the
+  trap cleanup and the post-replace backup removal. Caught by the new Bash
+  test suite.
+- Fix the behavioral-evaluation runner's Codex sandbox isolation: `CODEX_HOME`
+  was left pointing at the real user's `.codex` directory even though `HOME`
+  and `USERPROFILE` were overridden to a disposable sandbox, and the
+  disposable `.codex` directory was never created. Both meant Codex could
+  read real global state during a nominally isolated eval run. Fixed by
+  pointing `CODEX_HOME` at the disposable home and creating that directory
+  before invocation. A fully empty disposable `CODEX_HOME` cannot
+  authenticate, so the runner now seeds it with a copy of the real
+  `auth.json` only - no other real Codex state (skills, session history,
+  config) is copied in.
 
 These changes remain unreleased. They do not select a release version, create a
 tag, or imply publication.

README.md

@@ -224,9 +224,13 @@ representative multi-case subset command is implemented.
 
 The runner keeps disposable fixtures and raw traces outside the repository by
 default. It copies the canonical skill into disposable user-level and
-project-local locations and records content hashes. Runtime skill precedence
-remains observable evidence: if Codex selects another installed copy, the trace
-must be treated as an isolation failure rather than a behavior result.
+project-local locations and records content hashes. It also points `CODEX_HOME`
+at the disposable home rather than the real one, seeding it with a copy of the
+real `auth.json` only so live runs can authenticate without inheriting any
+other real Codex state (registered skills, session history, config). Runtime
+skill precedence remains observable evidence: if Codex selects another
+installed copy, the trace must be treated as an isolation failure rather than
+a behavior result.
 
 Observable limits are intentional and documented:
 

scripts/install-package.sh

@@ -115,7 +115,7 @@ mkdir -p "$parent"
 stage="$(mktemp -d "$parent/.${leaf}.stage.XXXXXX")"
 backup="$parent/.${leaf}.backup.$$"
 moved_existing=false
-cleanup() { [ -d "$stage" ] && rm -rf "$stage"; }
+cleanup() { if [ -d "$stage" ]; then rm -rf "$stage"; fi; }
 trap cleanup EXIT
 
 copy_managed "$SOURCE_DIR" "$stage"
@@ -133,5 +133,5 @@ if ! mv "$stage" "$DEST_DIR" || ! validate_package "$DEST_DIR" "Installed"; then
     exit 1
 fi
 
-[ "$moved_existing" = true ] && rm -rf "$backup"
+if [ "$moved_existing" = true ]; then rm -rf "$backup"; fi
 if [ "$CLEAN" = true ]; then echo "Clean reinstall complete."; else echo "Installation complete."; fi

scripts/run-behavioral-evals.mjs

@@ -276,9 +276,9 @@ async function runLiveCase(args) {
       env_overrides: {
         USERPROFILE: isolatedHomeRoot,
         HOME: isolatedHomeRoot,
-        CODEX_HOME: process.env.CODEX_HOME || path.join(process.env.USERPROFILE || os.homedir(), ".codex"),
+        CODEX_HOME: path.join(isolatedHomeRoot, ".codex"),
       },
-      fallback_prevention: "The runner sets USERPROFILE and HOME to a disposable home containing only the copied codebase-orient skill, and passes --ignore-user-config to codex exec.",
+      fallback_prevention: "The runner sets USERPROFILE, HOME, and CODEX_HOME to a disposable home containing only the copied codebase-orient skill, and passes --ignore-user-config to codex exec. The disposable CODEX_HOME is seeded with a copy of the real auth.json only, so live cases can authenticate without inheriting any other real Codex state such as registered skills or session history.",
     },
     snapshots: {
       initial: initialSnapshots,
@@ -729,6 +729,7 @@ async function writeRunnerFailure({
 
 async function prepareIsolatedSkillHome(homeRoot, fixtureRoot) {
   await resetDirectory(homeRoot);
+  await seedIsolatedCodexHome(homeRoot);
   const isolatedSkillDir = path.join(homeRoot, ".agents", "skills", "codebase-orient");
   await copyDirectory(canonicalSkillDir, isolatedSkillDir);
   const fixtureSkillDir = path.join(fixtureRoot, ".agents", "skills", "codebase-orient");
@@ -751,6 +752,21 @@ async function prepareIsolatedSkillHome(homeRoot, fixtureRoot) {
   };
 }
 
+async function seedIsolatedCodexHome(homeRoot) {
+  const isolatedCodexHome = path.join(homeRoot, ".codex");
+  await fsp.mkdir(isolatedCodexHome, { recursive: true });
+  const realCodexHome = path.join(process.env.USERPROFILE || os.homedir(), ".codex");
+  const realAuthPath = path.join(realCodexHome, "auth.json");
+  try {
+    await fsp.copyFile(realAuthPath, path.join(isolatedCodexHome, "auth.json"));
+  } catch (error) {
+    throw new Error(
+      `Unable to seed isolated Codex auth from ${realAuthPath}: ${error.message}. ` +
+        "Live cases require an authenticated local Codex install; only auth.json is copied, no other real Codex state.",
+    );
+  }
+}
+
 async function createFixtureRepo(fixtureType, fixtureRoot) {
   await resetDirectory(fixtureRoot);
   await runGit(["init"], fixtureRoot);
@@ -791,7 +807,7 @@ async function invokeCodexExec({
     ...process.env,
     USERPROFILE: isolatedHomeRoot,
     HOME: isolatedHomeRoot,
-    CODEX_HOME: process.env.CODEX_HOME || path.join(process.env.USERPROFILE || os.homedir(), ".codex"),
+    CODEX_HOME: path.join(isolatedHomeRoot, ".codex"),
   };
   const commandArgs = [
     "exec",

scripts/test-installers.sh

@@ -0,0 +1,64 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+HELPER="$SCRIPT_DIR/install-package.sh"
+REPO_ROOT="$(dirname "$SCRIPT_DIR")"
+TEST_ROOT="$(mktemp -d)"
+
+cleanup() { rm -rf "$TEST_ROOT"; }
+trap cleanup EXIT
+
+assert_throws() {
+    local message="$1"
+    shift
+    if "$@" >/dev/null 2>&1; then
+        echo "$message" >&2
+        exit 1
+    fi
+}
+
+for package in codebase-orient install-codebase-orient; do
+    source_dir="$REPO_ROOT/skills/$package"
+    dest="$TEST_ROOT/$package-dest"
+
+    bash "$HELPER" "$package" "$source_dir" "$dest" >/dev/null
+    assert_throws "Normal install did not refuse existing $package destination." \
+        bash "$HELPER" "$package" "$source_dir" "$dest"
+
+    printf 'preserve' > "$dest/extra.txt"
+    bash "$HELPER" "$package" "$source_dir" "$dest" --force >/dev/null
+    if [ ! -f "$dest/extra.txt" ]; then
+        echo "Overlay reinstall deleted an extra $package file." >&2
+        exit 1
+    fi
+
+    bash "$HELPER" "$package" "$source_dir" "$dest" --clean >/dev/null
+    if [ -f "$dest/extra.txt" ]; then
+        echo "Clean reinstall retained an extra $package file." >&2
+        exit 1
+    fi
+
+    assert_throws 'Conflicting flags were accepted.' \
+        bash "$HELPER" "$package" "$source_dir" "$dest" --force --clean
+
+    rm -f "$dest/SKILL.md"
+    mkdir -p "$dest/SKILL.md"
+    assert_throws 'Overlay managed type conflict was accepted.' \
+        bash "$HELPER" "$package" "$source_dir" "$dest" --force
+
+    rm -rf "$dest"
+    mkdir -p "$dest"
+    printf 'keep' > "$dest/sentinel.txt"
+    bad_source="$TEST_ROOT/$package-bad-source"
+    cp -r "$source_dir" "$bad_source"
+    printf '\ncorrupt\n' >> "$bad_source/SKILL.md"
+    assert_throws 'Malformed source package was accepted.' \
+        bash "$HELPER" "$package" "$bad_source" "$dest" --clean
+    if [ ! -f "$dest/sentinel.txt" ]; then
+        echo "Source validation failure mutated the destination." >&2
+        exit 1
+    fi
+done
+
+echo "Installer tests passed."