Save failure mode to usage artifact as failure_mode.json

[Copilot]

The conclusion job's handle-failure step writes failure signal data but it wasn't captured in the usage artifact — analytics consumers had no structured record of how a run failed.

Changes

handle_agent_failure.cjs

• Adds FAILURE_MODE_PATH = "/tmp/gh-aw/usage/failure_mode.json" and writes it alongside failure_categories.json with three stages of failure analysis:

  {
    "detected": { "agentConclusion": "failure", "isTimedOut": false, ... },
    "filtered": ["agent_failure"],
    "identified": "agent_failure"
  }

  • detected: all raw boolean signal flags read from env/agent output
  • filtered: the failureCategories array (same source as failure_categories.json)
  • identified: failureCategories[0] or null

notify_comment.go — step ordering fix

• Moves buildUsageArtifactUploadSteps to after agentFailureSteps in buildConclusionJob. Previously the usage artifact was uploaded before handle-failure ran, so any file written there would be missed.
• Adds /tmp/gh-aw/usage/failure_mode.json to the upload path list (if-no-files-found: ignore keeps this safe on success runs where the file is never written).

Tests

• Go: asserts failure_mode.json in upload paths and that "Upload usage artifact" follows "Handle agent failure" in step order.
• JS: asserts FAILURE_MODE_PATH is written with correct detected/filtered/identified structure; fixes missing GH_AW_AGENTIC_ENGINE_TIMEOUT cleanup in afterEach.

[Copilot]

Pull request overview

This PR ensures a structured failure_mode.json file produced by handle_agent_failure is captured in the compact “usage” artifact by (1) writing the file under /tmp/gh-aw/usage/ and (2) reordering/uploading artifact steps so the file exists before upload across generated workflows.

Changes:

• Write /tmp/gh-aw/usage/failure_mode.json from handle_agent_failure.cjs with { detected, filtered, identified } failure-mode data.
• Reorder conclusion-job steps so usage artifact upload occurs after “Handle agent failure”, and include failure_mode.json in the upload paths.
• Add/extend Go + JS tests to cover the new path and step ordering, plus env cleanup.

Show a summary per file

File	Description
pkg/workflow/notify_comment.go	Moves usage artifact upload steps after agent failure handling; adds failure_mode.json to artifact paths.
pkg/workflow/notify_comment_test.go	Asserts failure_mode.json is included and upload ordering is after “Handle agent failure”.
actions/setup/js/handle_agent_failure.cjs	Writes failure_mode.json under /tmp/gh-aw/usage/ and exports FAILURE_MODE_PATH.
actions/setup/js/handle_agent_failure.test.cjs	Adds coverage for failure_mode.json output shape and fixes missing env cleanup.
.github/workflows/workflow-normalizer.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.
.github/workflows/workflow-health-manager.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.
.github/workflows/workflow-generator.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.
.github/workflows/video-analyzer.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.
.github/workflows/update-astro.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.
.github/workflows/typist.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.
.github/workflows/test-workflow.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.
.github/workflows/test-project-url-default.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.
.github/workflows/test-dispatcher.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.
.github/workflows/super-linter.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.
.github/workflows/static-analysis-report.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.
.github/workflows/spec-extractor.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.
.github/workflows/spec-enforcer.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.
.github/workflows/sergo.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.
.github/workflows/security-compliance.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.
.github/workflows/research.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.
.github/workflows/release.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.
.github/workflows/refiner.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.
.github/workflows/org-health-report.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.
.github/workflows/jsweep.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.
.github/workflows/hippo-embed.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.
.github/workflows/gpclean.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.
.github/workflows/go-logger.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.
.github/workflows/go-fan.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.
.github/workflows/firewall.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.
.github/workflows/example-permissions-warning.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.
.github/workflows/dev-hawk.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.
.github/workflows/delight.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.
.github/workflows/deep-report.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.
.github/workflows/daily-news.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.
.github/workflows/copilot-opt.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.
.github/workflows/contribution-check.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.
.github/workflows/codex-github-remote-mcp-test.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.
.github/workflows/changeset.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.
.github/workflows/avenger.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.
.github/workflows/artifacts-summary.lock.yml	Reorders usage artifact collection/upload after handle-failure; adds failure_mode.json path.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

• Files reviewed: 138/249 changed files
• Comments generated: 1