Handle rollbacks I

[ch1bo]

What & Why

Currently, rollbacks are not handled by hydra-node. If a rollback occurs in Layer 1 and there are open Hydra Heads after the rolled-back fork point, hydra-node will crash. A basic ("dirt-road") solution ensuring that hydra-node remains operational in such a scenario is needed in order to run hydra-node on a public testnet.

Full support for rollbacks will be implemented at a later stage ("high-way" solution) #185

Requirements

• The hydra-node does not crash when a rollback occurs
• A ServerOutput is emitted to the client when a rollback occurs
• When a not-yet-open head is rolled back, clients can continue opening (i.e. commit, close or abort) the head after the rollback occurred (safety)
• When an already-open head is rolled back "crossing opening point", it is acceptable that the off-chain communication state is reset and we could continue or contest (no liveliness)

Tasks

• Create an ADR to get rid of the TVar in the Direct module #257
• Spike: see how we to test rollbacks and their consequences on the head lifecycle.  #296
• Detect rollbacks, communicate that one happened to the client #306
• Record "points" when a Head was initialized, opened etc. (or just before that) #307
• Update Changelog #314
• Address comments about rollbacks PR #315

[KtorZ]

The solution we discussed for handling rollbacks is a 3-step approach, for which the first step was completed as part of #228 and #221

Step 1

As a first step, we want to untangle the low-level construction and observation of on-chain transactions. We've been carrying around an OnChainHeadState with internals details needed to construct and observe those transactions. However it'd be preferable to keep those details opaque to the higher level of the chain stack such that a state merely becomes some opaque blob of data that needs to be stored but with no way of inspection by upper layers.

To achieve this, we create a State interface to sit between the higher level chain component and the low level transaction construction and observation. That state contains information needed to construct and observe transactions and encode the on-chain transitions of the Hydra protocol. The chain component has no way to peek into the state and can only modify it via the provided interface which is opaque by construction.

With this, the tracking of the on-chain head state within the chain component becomes trivial => some data is stored in a TVar.

Step 2

After step 1, we've made it so that storing the on-chain head state within the chain component itself has become redundant or unnecessary. The chain components evolves around inputs it receives from the head logic (PostChainTx) and communicates back to the chain logic via events (OnChainTx) yielded through a callback.

Thus, it is possible to make the entire approach more stateless by moving the content of the TVar to PostChainTx and OnChainTx however this requires a bit of rework of the various observation function in the Direct.Tx module. In particular, most functions currently require some stateful information to ease the discovery of inputs and scripts but those are in practice unnecessary:

1. For observeInitTx this is "trivial" because there's no previous state anyway. So InitTx is only yielding stuff up.

2. observeCommitTx currently relies on the initials :: [TxIn] to find the input from the inputs list that was distributed during the initTx. This is, unnecessary since (a) we statically know the initial script and (b) have access to all transaction witnesses and redeemers. So we could very much figure this one out of the transaction fully.

3. observeAbortTx weirdly take a UTXO for resolving input based on the head script address in order to find which input of the transaction is the state-machine input! Instead, it could also just look at the witnesses, find the appropriate redeemer for the head script and assert from there.

4. observeCollectComTx does the same weird thing as observeAbortTx with that passed UTXO.

5. observeCloseTx same.

6. observeFanoutTx same.

So, what we need ultimately is get the middle Direct.State layer to yield events that contain new version of the on-chain head state, which remains opaque but can now be stored directly in the head logic and then, be provided back for constructing transactions in PostChainTx

Step 3

Finally, once we've made the chain component stateless and centralized all state representation in the head logic state, we can move the head logic from an aggregated representation to a representation in the form of sequence of events. Done naively, we can always reconstruct the required state by folding over the sequence of events.

After that, handling rollback becomes quite straightforward as it only requires to drop events that happened beyond the point of rollback, and then, replay the application state from it.

[ch1bo]

What does "remaining operational" actually mean? If we are fine by loosing off-chain state, but always want to stay "in control" of the Head, we could do the following for example (as a very simple dirt-road):

• We record the "point" when a Head was initialized (or just before that)
• We detect rollbacks and communicate that one happened to the client
• We provide a way to start a hydra-node given a "point", or do that automatically whenever we see a rollback with the last known pre-initialization point
• As a consequence, we will always re-synchronize the on-chain head state (assuming this is fast enough) from the init point, but maybe loose all the off-chain state on rollbacks.
• Note that you would not be able to contest an already closed Head after this, even when the rollback was not impacting the Head closing transactions, but we don't have contestation until Contest logic & OCV #192 anyways.

[ch1bo]

An (easy) extension to ☝️ is to keep track of two points, an "init" point and an "open" point. That way, we could distinguish the rollbacks impacting the opening of a Head or not. Only when the opening of the head is impacted we would (suggest) to throw away the off-chain head state.

[ch1bo]

Updated the requirements to reflect our current approach.

[ch1bo]

Done as we also merged #310