Read the report out of a stream-json transcript

Claude's row passes --output-format stream-json, so its report arrives inside an escaped JSON string with the labels mid-line, where every anchored pattern in classify misses them. The first --e2e run scaffolded and deployed okta-users-claude, reported the deployment id, and was classified as "stopped without reporting". The tenant is what settled it, not the harness.

Expanding \n restores line structure for the labels and, just as importantly, for the blockquoted skill text the next filter strips, so the existing defences against matching our own documentation keep working rather than being bypassed.

The second substitution drops the JSON tail. The report ends the reply text, so the object's remaining fields sit past the closing quote on the same physical line and were being read as part of BLOCKER, turning NONE into a phantom blocker. It anchors on a quote followed by `}` or `]` instead of the first quote, because BLOCKER is asked to quote the CLI error verbatim and cutting there would truncate a real one.

Verified by re-classifying the saved log, which flips to PASS with the app name and deployment id, alongside six plain-text cases covering a clean deploy, a timed-out build, an echoed template, a duplicate app name, a quoted CLI error, and smoke mode.

Author: mraible

test-assistants.sh

@@ -548,7 +548,14 @@ blocker_category() {
 # "sat there doing nothing", and it read a clean timeout as success.
 classify() {
   local log="$1" rc="$2" body status skills raw_cmds raw_blocker cmds detail cat
-  body=$(grep -v '^[[:space:]]*>' "$log" 2>/dev/null)
+  # Claude streams stream-json, so its report arrives inside an escaped JSON string.
+  # Expanding \n puts the labels and the blockquoted skill text back at line start,
+  # where the patterns below expect them. The second sed drops the JSON tail that
+  # follows the closing quote, which would otherwise be read as part of BLOCKER; it
+  # anchors on a quote plus `}` or `]` rather than the first quote, because BLOCKER is
+  # asked to quote the CLI error verbatim. No-ops on plain-text logs.
+  body=$(sed -e 's/\\n/\
+/g' -e 's/"[]}].*$//' "$log" 2>/dev/null | grep -v '^[[:space:]]*>')
 
   grep -qiE "^[[:space:]]*(❌[[:space:]]*)?Error: unknown (flag|command)" <<< "$body" && { echo "FAIL|flag|rejected a CLI flag||"; return; }
   grep -qiE "^[[:space:]]*(❌[[:space:]]*)?Error:.*connection issue|^[[:space:]]*\* connection issue" <<< "$body" && { echo "FAIL|connection|connection issue (denied token-cache write?)||"; return; }