Skip to content

fix(cron): block in-gateway lifecycle jobs - #82079

Closed
SommelierLuc wants to merge 1 commit into
NousResearch:mainfrom
SommelierLuc:fix/cron-gateway-selfkill-30719
Closed

fix(cron): block in-gateway lifecycle jobs#82079
SommelierLuc wants to merge 1 commit into
NousResearch:mainfrom
SommelierLuc:fix/cron-gateway-selfkill-30719

Conversation

@SommelierLuc

@SommelierLuc SommelierLuc commented Aug 8, 2026

Copy link
Copy Markdown

What does this PR do?

Prevents an in-process cron scheduler from scheduling or firing a job that restarts, stops, reloads, or kills its own Hermes gateway. Without this guard, the gateway can terminate before a once-job persists completion; launchd/systemd then revives it and the same job can fire again, creating the respawn loop described in #30719.

The defense is applied at create, update, resume, and fire time. The fire-time backstop runs before the no_agent short-circuit, covers transient extra_prompt context, and fails closed if the guard itself cannot be evaluated.

This is intentionally focused. It does not include the cron ContextVar/approval changes discussed in #76797, Kanban policy, or local deployment policy. It complements the restart-loop breaker in #82039 by preventing the self-targeting job from reaching the gateway lifecycle command in the first place.

Related Issue

Refs #30719

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • 🔒 Security fix
  • ✅ Tests (adding or improving test coverage)

Changes Made

  • Add tools/cron_gateway_guard.py with bounded, cross-platform lifecycle-command detection.
  • Reject unsafe cron jobs during create, update, and resume.
  • Re-check at scheduler fire time before script or agent execution.
  • Normalize shell backslash-newline continuations so wrapped commands cannot evade matching.
  • Add behavioral coverage for create/update/resume/fire-time rejection and non-mutation.

How to Test

  1. Run scripts/run_tests.sh -j 3 tests/cron/test_cron_gateway_runtime_guard.py tests/tools/test_cronjob_tools.py tests/hermes_cli/test_gateway_restart_loop.py -q.
  2. Run .venv/bin/python tools/cron_gateway_guard.py.
  3. Run Ruff, py_compile, and git diff --check on the five changed Python files.

Local result: 195 passed, 0 failed. Ruff, Python compilation, guard self-test, diff check, local-policy scan, and secret-pattern scan passed.

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits
  • I searched for existing PRs and documented related work above
  • My PR contains only changes related to this fix
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for my changes
  • I've tested on macOS 26.5.2, Python 3.11.15

Documentation & Housekeeping

  • Documentation update: N/A; behavior and rationale are documented in module docstrings and errors
  • cli-config.yaml.example: N/A; no config keys changed
  • CONTRIBUTING.md / AGENTS.md: N/A; no workflow contract changed
  • Cross-platform impact considered: launchd and systemd command shapes are covered
  • Tool descriptions/schemas: N/A; schema unchanged

Full-suite attempt

The full repository suite was also attempted on this macOS checkout. It currently reports broad unrelated optional-dependency/platform failures on main (159 failures plus collection/import failures across 52 unrelated files). Neither changed test file failed; the focused 195-test suite above remains green. The only isolated approval failure is the existing macOS /tmp alias issue already covered by #70415; broader macOS suite work is tracked in #82023. The full-suite checkbox therefore remains intentionally unchecked rather than overstating the result.

Reject gateway restart, stop, and kill jobs at create, update, resume, and fire time so an in-process scheduler cannot enter a supervisor respawn loop.\n\nRefs NousResearch#30719
@alt-glitch alt-glitch added type/bug Something isn't working comp/cron Cron scheduler and job management P2 Medium — degraded but workaround exists labels Aug 8, 2026
@spfcraze

spfcraze commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

This was generated by AI during triage.

Summary:
The guard never scans monitor_script, which the scheduler executes in-process at fire time — a monitor job whose monitor_script holds a lifecycle command still restarts the gateway from inside the scheduler, the #30719 respawn loop.

Problems:

  • monitor_script appears nowhere in this diff: the create, update, resume, and fire-time call sites all pass only prompt and script to check_cron_gateway_selfkill.
  • cron/monitor.py _run_monitor_source executes monitor_script through the same _run_job_script as the script field (.sh runs under /bin/bash), and cron/scheduler.py runs it inside run_job via check_monitor after the new fire-time backstop — so a job with a benign prompt and a monitor_script of "launchctl kickstart -k gui/$(id -u)/ai.hermes.gateway" passes every layer and runs the command on its first tick.

Solution:
Scan the monitor source too: pass monitor_script into check_cron_gateway_selfkill at the same four call sites — create, update, resume, and the scheduler fire-time backstop — alongside the existing script scan.


Checked against f77bf14 — the tip of fix/cron-gateway-selfkill-30719 when this was written — and 6243136, main at the same moment.

@SommelierLuc
SommelierLuc deleted the fix/cron-gateway-selfkill-30719 branch August 9, 2026 12:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/cron Cron scheduler and job management P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants