ZetaClient v38 go-live tracker

[lumtis]

This issue centralizes tasks and processes for ZetaClient v38 go-live on testnet and mainnet.

https://github.com/zeta-chain/node/milestone/28

Summary

V38 will mostly contains updates that will not require operations regarding contracts upgrade: new chaos mode, outbounds performance improvements.

The only upgrade will be the Sui upgrade necessary for withdraw and call that was already planned in #4363 but postponed because of some revamp necessary to do.

Sui
The gateway upgrade will break compatibility for withdrawAndCall feature. Currently withdrawAndCall is disabled on mainnet, therefore no pausing will be necessary on Sui, the regular withdraw will still be functional.
After the ZetaClient and gateway upgrade, the ZetaChain EVM will have to be patched to enable Sui withdrawAndCall.

Overview

1. Upgrade ZetaClient binary

2. Upgrade Sui gateway
   The steps are documented in the Upgrading-Sui-Gateway playbook.

3. Enable Sui withdrawAndCall on ZetaChain gateway
   The new Sui chain parameters format is also documented in the Upgrading-Sui-Gateway playbook.

4. Restart the Sui chain observation
   Zetaclient currently cannot automatically pick up new Sui gateway package ID. In order to have zetaclient work with new gateway package (in chain params of step 3), there are two options

   option 1: restart all zetaclientd instances (simple and preferred)
   option 2: disable Sui chain with "is_supported": false,, and then enable it again with "is_supported": true,. This is more like a soft restart.

Testnet

• ZetaClient upgrade
• Sui upgrade

Mainnet

• ZetaClient upgrade
• Sui upgrade
• ZetaClient restart (to pick up Sui new package ID)