Avoid getting slashed after a breaking runtime upgrade

[sigurpol]

A runtime upgrade can break the miner and potentially cause slashing if not handled defensively.
E.g.

• The runtime is upgraded.
• The register_score extrinsic remains compatible withthe codegen/types and can be submitted successfully.
• The submit_page extrinsic is changed in a breaking way (e.g., new arguments, different types, semantics).
• The miner, using old codegen/types, submits submit_page extrinsics that are now invalid.

What we are doing today

1. Our CI/CD has a nightly test checking that we are able to get rewarded vs latest polkadot SDK
2. main.rs already has a mechanism for handling runtime upgrades:

• It spawns a runtime_upgrade_task that listens for runtime upgrades using Subxt’s runtime_updates().
• When an upgrade is detected, it:
  • Applies the update to the client.
  • Calls dynamic::update_metadata_constants to refresh all runtime constants and types.

Optionally for legacy (election-multi-phase) path only there is an info separate command that checks codegen compatibility.

How can we improve?

Some ideas:

• Make all mining/submission logic pause while an upgrade is in progress.
• Check codegen compatibility immediately after upgrade and not only via info command
• Bail if the upgrade is not codegen compatible
• Legacy path has a dry-run option to check validity of the extrinsics. We should extend it to EPMB case too and maybe add an option like submit only if dry-run is successful.
• Maybe we can combine the dry-run strategy with the 1st point and try a dry-run of everything which was on-going before the upgrade and if successful we submit it otherwise we bail (hoping that bailing has not introduced a breaking change...)
• Can we get rid of most of the static types and rely almost fully on dynamic API (as in dynamic module)?

[sigurpol]

This issue arises from an insightful comment I received from @Overkillus during our discussion about the miner. I believe it's an important topic to keep on our radar. cc @kianenigma, @Ank4n, @seadanda

[kianenigma]

To be fair, I think

The register_score extrinsic remains compatible withthe codegen/types and can be submitted successfully.
The submit_page extrinsic is changed in a breaking way (e.g., new arguments, different types, semantics).

Is super unlikely, and if it ever happens, it will you or me who will break them, so we can coordinate the miner upgrade :))

It is a good safety feature to have though.