SDN19-01: Domain members observe a deliberate node departure (host model)

[jkenyon-nvidia]

Test statement

IMEX (host model): When the IMEX service on one node is stopped deliberately through the node service manager, the node shuts down cleanly and the remaining domain members report it as unavailable within a bounded time, while the domain stays operational among the surviving nodes.

Source: docs/test-plan.yaml, SDN19-01.

What to assert

• Target node's service stopped gracefully, and a clean shutdown observed.
• A surviving node reports the target as unavailable within a bounded timeout.
• The domain stays operational among the surviving members. Distinguish "peers never noticed" (stale membership) from "domain collapsed" (fragile) in the failure message, since they are different defects.
• Prior state restored on teardown, mandatory rather than best-effort, because this test deliberately leaves the domain degraded.

Step output contract

The check consumes normalized JSON from a provider step. All parsing of vendor
output happens in the provider script; the validation sees only this shape.

{
  "success": true,
  "platform": "provider",
  "target_node": "node-01",
  "operations": {
    "stop": {"requested": true, "clean_exit": true},
    "peer_convergence": {
      "observed_from": "node-02",
      "target_reported": "unavailable",
      "elapsed_seconds": 8,
      "surviving_members_operational": true
    },
    "restore": {"restored_to": "active", "domain_member": true}
  }
}

target_reported is a normalized enum: available, unavailable, or unknown, so
the validation never string-matches vendor output.

Scope

In scope: a deliberate, graceful stop on one node; that node shutting down cleanly; surviving members observing the departure within a bounded time; and the domain remaining operational among survivors.

Out of scope: unexpected termination and automatic recovery, which is a different failure mode with different correct behaviour.

This test is destructive. It removes a node from a live domain and must run only on an idle or drained domain, with a teardown that restores prior state.

Notes

Confirm the actual peer state values the platform emits before coding a matcher, and normalize them in the provider script. The validation must never substring-match raw vendor output.