Cathode heater can show OFF even when output remains energized

[bwalkerMIR]

Cathode heater can show OFF even when output remains energized

Location: power_supply_9104.py:177, cathode_heating.py:2243, cathode_heating.py:2933

What is wrong: Several cathode OFF/abort paths still call set_output("0"), but set_output() first reads preset voltage and validates OVP before sending SOUT0.

Exact scenario: Heater output is on, then OVP readback fails or stored voltage is invalid. The operator toggles output off; set_output("0") returns before sending SOUT0, but the UI still flips to off.

Why it matters: A heater can remain energized while the dashboard says it is off.

Recommended fix: Use disable_output() for every OFF/abort/shutdown path, or make set_output("0") bypass voltage validation. Do not update UI off state unless the OFF command succeeds or the state is explicitly unknown.

[chott24]

Most of this issue has already fixed in #173. All behind-the-scenes variables and states are accurately represented by failed 9104 shutoffs. However, a the UI toggle button still has 3 instances where it gets updated no matter what disable_output() returns. This should be an easy fix, simply adding a conditional that is based off of disable_output(). Here is a codex analysis that was done on the most recent commit - commit c10f225 of PR #180

Summary

The original issue is only partially accurate on the current branch.

Normal cathode OFF/shutdown paths no longer rely on set_output("0"); they mostly use disable_output(), which sends SOUT0 directly and bypasses preset/OVP validation. So the original OVP-readback scenario is largely fixed for production PowerSupply9104 handles.

The remaining issue is narrower: some paths call disable_output() but still mark the UI toggle OFF even if disable_output() fails. In that case the 9104 output state is uncertain, but the dashboard can show OFF.

Faulty remaining paths:

• Manual OFF branch in toggle_output(): logs failed disable_output() but still falls through and sets toggle_states[index] = False.
• update_output_from_current(): if set_current() fails before a voltage ramp, it calls disable_output() and then unconditionally marks the toggle OFF.
• update_output_from_voltage(): if set_voltage() fails before a current ramp, it calls disable_output() and then unconditionally marks the toggle OFF.

There is still one fallback set_output("0") inside turn_off_all_beams() for objects that lack disable_output(), but real PowerSupply9104 handles implement disable_output(), so that fallback should not run in normal production use.

Proposed fix:

• In the manual OFF branch, update the toggle OFF only if disable_output() returns True.
• If disable_output() returns False, log CRITICAL, keep the toggle ON, and treat hardware state as uncertain.
• In the two abort paths, only set toggle_states[index] = False and update the button image if disable_output() succeeds.
• If disable fails in those abort paths, log CRITICAL and leave the toggle ON.

This keeps the UI from claiming the heater is OFF after an unconfirmed shutdown command. It does not require changing set_output().

[chott24]

The above suggested change was implemented and tested in PR #192. See the comment in that PR for more details.

[bwalkerMIR]

I haven't reviewed the latest comments to this issue yet, but the following serial lock behavior described below is also related @chott24 :

The GUI and actual hardware state can diverge, specifically on the OFF path, due to serial lock behavior.

The clearest case is:

1. User clicks output OFF.
2. Code calls set_output("0").
3. Because PR Bugfix: CCS thread bugs #160 made normal 9104 commands use a 0.5s serial-lock timeout, that OFF command can fail to acquire the lock and never send SOUT0.
4. toggle_output() does not check whether set_output("0") succeeded.
5. The GUI still flips the button/state to OFF.

So the user would not reliably know from the main UI. There may be a warning/error in the log, and the next readback may eventually show the real state, but the immediate operator-facing control can say OFF even though the supply may still be physically ON.

For ON commands, the code is better: most ON/setup paths check failures and return before flipping the UI to ON. My blocker is mainly the normal OFF toggle and similar “command failed but UI state advances anyway” situations. The underlying issue is not just “a command might not get sent”; it is “a command might not get sent and at least one caller still updates UI state as if it did.”

[chott24]

The above discrepancy path between the UI toggle button is fixed in #192. The toggle output button only changes after a successful set_ouput() or disable_output().