Skip to content

docs(tui): 6/15 canary + flip/rollback runbooks + setup auth-probe guard (PR-C) #138

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dtzp555-max merged 1 commit into from
Jun 10, 2026
Merged

docs(tui): 6/15 canary + flip/rollback runbooks + setup auth-probe guard (PR-C) #138

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -898,6 +898,7 @@ Future `ocp update` invocations sync automatically.
| `OCP_TUI_CWD` | `$HOME/.ocp-tui/work` | (TUI-mode) Scratch working directory where interactive claude sessions run. Transcripts land under `<HOME>/.claude/projects/<encoded-cwd>/`. Created automatically. |
| `OCP_TUI_HOME` | `$HOME` (real home) | (TUI-mode) `HOME` claude runs under. Default is the operator's real home (shared credentials, existing onboarding). Set to a separate path for scratch-home isolation — see ADR 0007 for the credential-fork caveat. |
| `OCP_TUI_ENTRYPOINT` | `cli` | (TUI-mode) Billing-classifier labeling: `cli` (default) pins `cc_entrypoint=cli` deterministically; `auto` lets claude self-classify via TTY detection; `off` leaves the inherited env untouched. Honest only when the spawn is a genuine interactive PTY — see ADR 0007. |
| `OCP_SKIP_AUTH_TEST` | *(unset)* | When `=1`, skip the `claude -p` auth probe during `setup.mjs`. After 2026-06-15 this probe draws from the Agent SDK credit pool; set this to avoid burning a metered credit on re-installs or `ocp update` runs. Auth is validated at the first real request. |
| `OCP_TUI_FULL_TOOLS` | *(unset)* | (TUI-mode, **single-user only**) When `=1`, grant the interactive session the **same tool surface as the `-p` path** — `--allowedTools` (+ optional `--mcp-config` / `--dangerously-skip-permissions`, read from `CLAUDE_ALLOWED_TOOLS` / `CLAUDE_MCP_CONFIG` / `CLAUDE_SKIP_PERMISSIONS`) — instead of the default MCP-walled, built-in-tools-only set. Lets a trusted single-operator TUI deployment run a **tool-using / MCP agent** (e.g. an OpenClaw assistant) on the subscription pool. Safe because TUI **refuses to boot under `AUTH_MODE=multi`** (hard exit) — no guest key can ever reach the TUI path, so this gate cannot expose tools to an untrusted caller. (Under `AUTH_MODE=shared` + `OCP_TUI_ALLOW_LAN=1`, anyone holding the single shared key reaches it — that is the existing TUI trust model, unchanged.) See [Subscription-pool (TUI) mode](#subscription-pool-tui-mode) and ADR 0007. |

### Streaming heartbeat
Expand Down Expand Up @@ -974,6 +975,13 @@ unset CLAUDE_TUI_MODE

The stream-json path is restored immediately. No other change is needed.

### 2026-06-15 operator checklist

Every host serving traffic must be flipped to TUI-mode **and** canary-verified before 2026-06-15, or it will bill the metered Agent SDK credit pool instead of the subscription.

- **[Flip/rollback runbook](docs/runbooks/tui-flip-rollback.md)** — how to set `CLAUDE_TUI_MODE=true` on systemd (Linux) and launchd (macOS) hosts. Covers the `daemon-reload` requirement (systemd) and the `bootout`+`bootstrap` cycle requirement (launchd — `launchctl kickstart -k` does not reload plist env).
- **[615-canary runbook](docs/runbooks/615-canary.md)** — after each flip, run one quiesced request and compare the Agent SDK credit balance before and after. `entrypoint:cli` in the transcript (the `cc_entrypoint` billing classifier) is necessary but not sufficient — only a stable credit balance confirms the subscription pool is being used. Balance check is a manual step (no known programmatic API for the Agent SDK credit pool balance).

### Architecture and design decisions

See [`docs/adr/0007-tui-interactive-mode.md`](docs/adr/0007-tui-interactive-mode.md) for the full rationale, home-strategy options, MCP-disable mechanism, coexistence rules, and the B-path (multi-tenant isolation) roadmap.
Expand Down
151 changes: 151 additions & 0 deletions docs/runbooks/615-canary.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
# 2026-06-15 Canary Runbook

**Purpose:** Confirm that a TUI-mode turn is billed to the **Pro/Max subscription pool** (not the Agent SDK credit pool) after Anthropic's 2026-06-15 billing split activates.

The billing classifier reading `cli` is **necessary but NOT sufficient** proof. (Note the naming: the value is stored in the JSONL transcript under the field name `entrypoint`, and sent to Anthropic on the wire as the `cc_entrypoint` header — they carry the same value after claude's startup classification. The commands below grep the transcript, so they match `entrypoint`.) A `cli` label tells you OCP sent the right classification; it does not tell you Anthropic billed the right pool. The only authoritative test is to observe whether the **Agent SDK credit balance** moves or not before and after the canary turn.

---

## Prerequisites

- `CLAUDE_TUI_MODE=true` already set and OCP restarted (see [TUI-mode setup in README](../../README.md#enabling-tui-mode-opt-in))
- `tmux` installed on the host
- No other OCP traffic during the canary (quiesce — see below)
- Access to your Anthropic account billing page (manual step — see below)

---

## Step 1 — Quiesce the host

Stop any IDE or client that is actively sending requests through this OCP instance.

Confirm the proxy is idle:

```bash
curl -s http://127.0.0.1:3456/health | python3 -m json.tool | grep activeRequests
# Expected: "activeRequests": 0
```

Wait until `activeRequests` is `0` before proceeding. If you cannot quiesce (e.g. family members are actively using it), run the canary on a separate OCP instance or during a quiet window.

---

## Step 2 — Read the Agent SDK credit balance BEFORE the canary

> **Manual step — no programmatic API available.**
>
> OCP's `/usage` endpoint reads `anthropic-ratelimit-unified-*` response headers from the Pro/Max plan quota (5-hour and 7-day subscription windows). These headers report **subscription usage**, not the Agent SDK credit pool balance. There is no known programmatic API to query the Agent SDK credit pool balance from outside the Anthropic web app.

To read the balance:

1. Open [https://claude.ai/settings/billing](https://claude.ai/settings/billing) (or your Anthropic Console billing page) in a browser.
2. Find the **Agent SDK Credits** section (sometimes labeled "API Credits" or "Agent SDK usage").
3. Note the current balance (e.g. `$18.43 remaining of $20.00`).

Write the value down — you will compare it after the canary turn.

---

## Step 3 — Send the canary turn

With TUI-mode on and the host quiesced, send exactly one small request:

```bash
curl -s -X POST http://127.0.0.1:3456/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "claude-haiku-4-5-20251001",
"messages": [{"role": "user", "content": "Reply with the single word: pong"}],
"max_tokens": 10
}' | python3 -m json.tool
```

Use Haiku (the cheapest model) to minimize any hypothetical impact if the canary turns red.

Wait for the response to arrive completely (TUI-mode buffers the full response before returning — you will see a delay of several seconds, then the full reply).

---

## Step 4 — Confirm the transcript shows `entrypoint:"cli"`

After the canary turn completes, inspect the most recent JSONL transcript for the billing-classifier label:

```bash
# The canary was run quiesced (Step 1), so the most recent JSONL across ALL project
# dirs IS the canary turn. We glob every projects subdir instead of recomputing
# claude's cwd-encoding rule (it maps every "/" AND "." to "-", e.g. ~/.ocp-tui/work
# => projects/-home-<user>--ocp-tui-work/; see lib/tui/transcript.mjs encodeCwd) —
# a glob is robust even if that encoding changes in a future claude build.
LATEST=$(ls -t "$HOME"/.claude/projects/*/*.jsonl 2>/dev/null | head -1)
echo "Transcript: $LATEST"
grep -o '"entrypoint":"[^"]*"' "$LATEST" | tail -1
# Expected: "entrypoint":"cli"
```

If the output shows `"entrypoint":"cli"`, the billing-classifier label is correct. If it shows `"entrypoint":"sdk-cli"`, the spawn did not get a real PTY — stop immediately and do not re-enable TUI-mode without investigation. Check `tmux new-session` manually and review ADR 0007 § spawn/PTY gate. (If the grep returns nothing, the transcript may not yet be flushed — re-run after a second, or confirm the turn completed.)

**Reminder: an `entrypoint:cli` label (the `cc_entrypoint=cli` wire header) is necessary but not sufficient.** It tells you OCP sent the right label to Anthropic. You must still check the credit balance in Step 5.

---

## Step 5 — Re-read the Agent SDK credit balance AFTER the canary

Return to [https://claude.ai/settings/billing](https://claude.ai/settings/billing) and reload the page. Note the current balance again.

---

## Step 6 — Green/Red decision

### Green (balance unchanged)

The Agent SDK credit balance did not decrease. The turn billed against the Pro/Max subscription pool as expected. TUI-mode is working correctly.

**Actions:**
- Keep `CLAUDE_TUI_MODE=true` on this host.
- Monitor the balance periodically for the first week to catch any delayed attribution.
- Resume normal traffic.

### Red (Agent SDK credit balance decreased)

The Agent SDK credit balance decreased. The subscription pool is not being used for TUI-mode turns on this host, despite `cc_entrypoint=cli` being set. This may indicate a backend routing change on Anthropic's side, a TTY detection failure, or a policy change.

**Actions — immediate:**
1. Unset `CLAUDE_TUI_MODE` (or set to any value other than `"true"`) in the service unit:
- systemd: edit `/etc/ocp/ocp.env` (or the unit's `Environment=` line), then `sudo systemctl daemon-reload && sudo systemctl restart ocp.service`
- launchd: edit the plist `EnvironmentVariables` section, then `launchctl bootout gui/$(id -u)/dev.ocp.proxy && launchctl bootstrap gui/$(id -u) <plist-path>`
2. Restart OCP and confirm the `/health` response no longer shows TUI-mode active.
3. If you share this OCP with family or other Max users: freeze their access temporarily until you understand the billing impact.
4. Consider pivoting to OLP multi-provider (see [OLP](https://github.com/dtzp555-max/olp)) which can spread load across other providers to avoid the Agent SDK credit drain.

Per ALIGNMENT.md Rule 2 / ADR 0007 § Kill-switch: "Per the constitution, the response is to drop the Anthropic provider rather than escalate spoofing."

---

## Ongoing monitoring — self-classification mini-canary

To detect future drift (e.g. a claude CLI upgrade that changes TTY-detection behavior), you can run a periodic one-liner that sends a tiny TUI turn with `OCP_TUI_ENTRYPOINT=auto` (so claude self-classifies rather than having OCP pin the value) and alerts if the transcript self-classification is not `cli`:

```bash
# Run with OCP temporarily configured OCP_TUI_ENTRYPOINT=auto
# Then check the most recent transcript:
# Glob the most recent transcript across all project dirs (robust to claude's
# cwd-encoding rule; run this right after the auto-mode mini-canary turn).
LATEST=$(ls -t "$HOME"/.claude/projects/*/*.jsonl 2>/dev/null | head -1)
RESULT=$(grep -o '"entrypoint":"[^"]*"' "$LATEST" | tail -1)
echo "Self-classified entrypoint: $RESULT"
if echo "$RESULT" | grep -q '"entrypoint":"cli"'; then
echo "OK — subscription pool"
else
echo "ALERT — not cli; check TTY and billing"
fi
```

Run this after any major `claude` CLI upgrade. The `auto` mode lets the CLI's own `t$A` startup function determine the value from the actual TTY state (see ADR 0007 § Billing-classifier labeling).

---

## Related

- [Flip/rollback runbook](./tui-flip-rollback.md) — how to set and unset `CLAUDE_TUI_MODE` on systemd and launchd hosts
- [ADR 0007](../adr/0007-tui-interactive-mode.md) — TUI-mode architecture and governing rules
- README § [Subscription-pool (TUI) mode](../../README.md#subscription-pool-tui-mode)
180 changes: 180 additions & 0 deletions docs/runbooks/tui-flip-rollback.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
# TUI-Mode Flip and Rollback Runbook

**Purpose:** Step-by-step instructions for enabling (`CLAUDE_TUI_MODE=true`) or disabling TUI-mode on real OCP deployments managed by **systemd** (Linux) or **launchd** (macOS).

Run the [615-canary](./615-canary.md) runbook after any flip to confirm billing pool routing is correct.

---

## Critical pitfalls — read first

### systemd: `daemon-reload` is required after editing the unit

Editing the unit file (or EnvironmentFile) and then doing `systemctl restart ocp.service` **without** `daemon-reload` will restart the process with the **old** environment from the cached unit. Always run `daemon-reload` after editing any unit file.

### launchd: `launchctl kickstart -k` does NOT reload plist env

`launchctl kickstart -k gui/$(id -u)/dev.ocp.proxy` kills the running process and re-launches it, but it **re-uses the launchd-cached environment** — not the current plist file. If you edited the plist's `EnvironmentVariables` section, you must do a full `bootout` + `bootstrap` cycle for the change to take effect. `kickstart` is not sufficient.

---

## Flip — enable TUI-mode

### systemd (Linux, e.g. Raspberry Pi, VPS)

**Option A — EnvironmentFile (recommended for clean separation)**

If your unit uses `EnvironmentFile=/etc/ocp/ocp.env` (or similar):

```bash
# 1. Edit the environment file
sudo nano /etc/ocp/ocp.env
# Add or update:
# CLAUDE_TUI_MODE=true
#
# If OCP binds to 0.0.0.0 AND you trust the network:
# OCP_TUI_ALLOW_LAN=1
# (WARNING: TUI-mode is single-user only — only enable OCP_TUI_ALLOW_LAN=1
# if you fully trust every caller that can reach the OCP port on your network)

# 2. Reload the unit definition and restart
sudo systemctl daemon-reload
sudo systemctl restart ocp.service

# 3. Verify
curl -s http://127.0.0.1:3456/health | python3 -m json.tool | grep -E "tui|version"
# Expected: "tuiMode": true (or similar TUI indicator in the health response)
```

**Option B — inline Environment= in the unit file**

```bash
# 1. Edit the unit file
sudo systemctl edit --full ocp.service
# Add or update in the [Service] section:
# Environment=CLAUDE_TUI_MODE=true

# 2. Reload and restart
sudo systemctl daemon-reload
sudo systemctl restart ocp.service

# 3. Verify
systemctl show ocp.service --property=Environment
# Expected: Environment=CLAUDE_TUI_MODE=true ...
```

### launchd (macOS)

Locate the OCP plist. The standard label is `dev.ocp.proxy`:

```bash
# Find the plist path
ls ~/Library/LaunchAgents/dev.ocp.proxy.plist
```

**Edit the plist:**

```bash
# 1. Stop the service first (bootout)
launchctl bootout gui/$(id -u)/dev.ocp.proxy

# 2. Edit the plist — add CLAUDE_TUI_MODE to EnvironmentVariables
# Use your editor of choice:
nano ~/Library/LaunchAgents/dev.ocp.proxy.plist
```

Inside the plist, in the `<key>EnvironmentVariables</key>` `<dict>` block, add:

```xml
<key>CLAUDE_TUI_MODE</key>
<string>true</string>
```

If `OCP_TUI_ALLOW_LAN=1` is also needed (only if OCP binds to `0.0.0.0` and you trust the network):

```xml
<key>OCP_TUI_ALLOW_LAN</key>
<string>1</string>
```

```bash
# 3. Bootstrap (reload from disk + start)
launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/dev.ocp.proxy.plist

# 4. Verify
curl -s http://127.0.0.1:3456/health | python3 -m json.tool | grep -E "tui|version"
```

**Confirm env was actually loaded** (not just set in your shell):

```bash
ps aux | grep server.mjs | grep -v grep
# Get the PID, then:
# macOS: ps -E -p <PID> | tr ' ' '\n' | grep CLAUDE_TUI_MODE
# Expected: CLAUDE_TUI_MODE=true
```

---

## Rollback — disable TUI-mode

Rollback is the same procedure as flip, but you **remove** `CLAUDE_TUI_MODE` or set it to any value other than `"true"` (e.g. `false`, or simply omit it).

After rollback, OCP returns to the default `callClaude` / `callClaudeStreaming` stream-json path — byte-for-byte identical to the pre-TUI code path. No other change is required.

### systemd rollback

```bash
# Option A — EnvironmentFile
sudo nano /etc/ocp/ocp.env
# Remove or comment out:
# CLAUDE_TUI_MODE=true
# OCP_TUI_ALLOW_LAN=1 (if set)

sudo systemctl daemon-reload
sudo systemctl restart ocp.service

# Verify
curl -s http://127.0.0.1:3456/health | python3 -m json.tool | grep tui
# Expected: "tuiMode": false (or the field absent)
```

### launchd rollback

```bash
# 1. Stop
launchctl bootout gui/$(id -u)/dev.ocp.proxy

# 2. Edit plist — remove the CLAUDE_TUI_MODE and OCP_TUI_ALLOW_LAN entries from EnvironmentVariables

# 3. Bootstrap
launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/dev.ocp.proxy.plist

# 4. Verify
curl -s http://127.0.0.1:3456/health | python3 -m json.tool | grep tui
```

---

## Billing impact of staying on the default (non-TUI) path after 2026-06-15

If you do NOT flip to TUI-mode and keep `CLAUDE_TUI_MODE` unset (the default), OCP continues using `claude -p --output-format stream-json`, which sets `cc_entrypoint=sdk-cli`. After 2026-06-15, every OCP request on the default path will draw from the Agent SDK credit pool (approximately $20/month on a Pro plan, or $100/month on a Max plan) rather than the Pro/Max subscription. The subscription pool usage (5-hour and 7-day windows) will be unaffected, but the Agent SDK credit balance will drain with each request.

If you want to continue using OCP without TUI-mode after 2026-06-15, budget for the Agent SDK credit cost accordingly — or switch to [OLP](https://github.com/dtzp555-max/olp) for multi-provider fallback.

---

## Verify after any flip

1. Check `/health` shows the expected `tuiMode` state.
2. Run the [615-canary](./615-canary.md) to confirm billing pool routing.
3. If TUI-mode is ON: check `ocp logs 10` for any TUI spawn errors (`tui_spawn_failed`, tmux errors).

---

## Related

- [615-canary runbook](./615-canary.md) — how to verify billing pool routing after a flip
- [ADR 0007](../adr/0007-tui-interactive-mode.md) — TUI-mode architecture; Kill-switch section
- README § [Subscription-pool (TUI) mode](../../README.md#subscription-pool-tui-mode)
- README § [Environment Variables](../../README.md#environment-variables) — `CLAUDE_TUI_MODE`, `OCP_TUI_ALLOW_LAN=1`
30 changes: 19 additions & 11 deletions setup.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -141,18 +141,26 @@ try {
}

// Check claude auth (quick test)
try {
const out = execSync('claude -p --output-format text --no-session-persistence -- "ping"', {
encoding: "utf-8",
timeout: 30000,
env: { ...process.env, CLAUDECODE: undefined, ANTHROPIC_API_KEY: undefined, ANTHROPIC_BASE_URL: undefined, ANTHROPIC_AUTH_TOKEN: undefined },
}).trim();
if (out.length > 0) {
log(`Claude CLI authenticated (test response: "${out.slice(0, 40)}...")`);
// NOTE: This probe uses `claude -p` (sdk-cli spawn). After the 2026-06-15 Anthropic billing
// split, every `claude -p` call draws from the Agent SDK credit pool rather than the
// Pro/Max subscription. Re-running setup after 6/15 will consume one metered credit.
// Set OCP_SKIP_AUTH_TEST=1 to skip this probe (auth is still validated at first real request).
if (process.env.OCP_SKIP_AUTH_TEST === "1") {
warn("OCP_SKIP_AUTH_TEST=1 — skipping claude auth probe (will be validated at first request).");
} else {
try {
const out = execSync('claude -p --output-format text --no-session-persistence -- "ping"', {
encoding: "utf-8",
timeout: 30000,
env: { ...process.env, CLAUDECODE: undefined, ANTHROPIC_API_KEY: undefined, ANTHROPIC_BASE_URL: undefined, ANTHROPIC_AUTH_TOKEN: undefined },
}).trim();
if (out.length > 0) {
log(`Claude CLI authenticated (test response: "${out.slice(0, 40)}...")`);
}
} catch (e) {
warn(`Claude CLI auth test failed: ${e.message.slice(0, 100)}`);
warn("Make sure you're logged in: claude login");
}
} catch (e) {
warn(`Claude CLI auth test failed: ${e.message.slice(0, 100)}`);
warn("Make sure you're logged in: claude login");
}

// Check openclaw config (optional — OCP runs standalone without OpenClaw)
Expand Down
Loading