Network Bridge Refactoring

roadmap/implementers-guide/src/SUMMARY.md

@@ -48,6 +48,7 @@
     - [Runtime](types/runtime.md)
     - [Chain](types/chain.md)
     - [Messages](types/messages.md)
+    - [Network](types/network.md)
 
 [Glossary](glossary.md)
 [Further Reading](further-reading.md)

roadmap/implementers-guide/src/node/availability/availability-distribution.md

@@ -6,24 +6,21 @@ After a candidate is backed, the availability of the PoV block must be confirmed
 
 ## Protocol
 
-`ProtocolId`:`b"avad"`: `PeerSet`: `Validation`
+`PeerSet`: `Validation`
 
 Input:
 
-- NetworkBridgeUpdate(update)
+- NetworkBridgeUpdateV1(update)
 
 Output:
 
-- NetworkBridge::RegisterEventProducer(`ProtocolId`)
-- NetworkBridge::SendMessage(`[PeerId]`, `ProtocolId`, `Bytes`)
+- NetworkBridge::SendValidationMessage(`[PeerId]`, message)
 - NetworkBridge::ReportPeer(PeerId, cost_or_benefit)
 - AvailabilityStore::QueryPoV(candidate_hash, response_channel)
 - AvailabilityStore::StoreChunk(candidate_hash, chunk_index, inclusion_proof, chunk_data)
 
 ## Functionality
 
-Register on startup an event producer with  `NetworkBridge::RegisterEventProducer`.
-
 For each relay-parent in our local view update, look at all backed candidates pending availability. Distribute via gossip all erasure chunks for all candidates that we have to peers.
 
 We define an operation `live_candidates(relay_heads) -> Set<CommittedCandidateReceipt>` which returns a set of [`CommittedCandidateReceipt`s](../../types/candidate.md#committed-candidate-receipt) a given set of relay chain heads that implies a set of candidates whose availability chunks should be currently gossiped. This is defined as all candidates pending availability in any of those relay-chain heads or any of their last `K` ancestors. We assume that state is not pruned within `K` blocks of the chain-head.

roadmap/implementers-guide/src/node/availability/bitfield-distribution.md

@@ -4,22 +4,21 @@ Validators vote on the availability of a backed candidate by issuing signed bitf
 
 ## Protocol
 
-`ProtocolId`: `b"bitd"`: `PeerSet`: `Validation`
+`PeerSet`: `Validation`
 
 Input:
 [`BitfieldDistributionMessage`](../../types/overseer-protocol.md#bitfield-distribution-message) which are gossiped to all peers, no matter if validator or not.
 
 Output:
 
-- `NetworkBridge::RegisterEventProducer(ProtocolId)` in order to register ourself as an event provider for the protocol.
-- `NetworkBridge::SendMessage([PeerId], ProtocolId, Bytes)` gossip a verified incoming bitfield on to interested subsystems within this validator node.
+- `NetworkBridge::SendValidationMessage([PeerId], message)` gossip a verified incoming bitfield on to interested subsystems within this validator node.
 - `NetworkBridge::ReportPeer(PeerId, cost_or_benefit)` improve or penalize the reputation of peers based on the messages that are received relative to the current view.
 - `ProvisionerMessage::ProvisionableData(ProvisionableData::Bitfield(relay_parent, SignedAvailabilityBitfield))` pass
   on the bitfield to the other submodules via the overseer.
 
 ## Functionality
 
-This is implemented as a gossip system. Register a [network bridge](../utility/network-bridge.md) event producer on startup.
+This is implemented as a gossip system.
 
 It is necessary to track peer connection, view change, and disconnection events, in order to maintain an index of which peers are interested in which relay parent bitfields.
 

roadmap/implementers-guide/src/node/backing/pov-distribution.md

@@ -4,15 +4,14 @@ This subsystem is responsible for distributing PoV blocks. For now, unified with
 
 ## Protocol
 
-`ProtocolId`: `b"povd"`, `PeerSet`: `Validation`
+`PeerSet`: `Validation`
 
 Input: [`PoVDistributionMessage`](../../types/overseer-protocol.md#pov-distribution-message)
 
 
 Output:
 
-- NetworkBridge::RegisterEventProducer(`ProtocolId`)
-- NetworkBridge::SendMessage(`[PeerId]`, `ProtocolId`, `Bytes`)
+- NetworkBridge::SendMessage(`[PeerId]`, message)
 - NetworkBridge::ReportPeer(PeerId, cost_or_benefit)
 
 
@@ -56,18 +55,7 @@ struct PeerState {
 }
 ```
 
-We also assume the following network messages, which are sent and received by the [Network Bridge](../utility/network-bridge.md)
-
-```rust
-enum NetworkMessage {
-	/// Notification that we are awaiting the given PoVs (by hash) against a
-	/// specific relay-parent hash.
-	Awaiting(Hash, Vec<Hash>),
-	/// Notification of an awaited PoV, in a given relay-parent context.
-	/// (relay_parent, pov_hash, pov)
-	SendPoV(Hash, Hash, PoV),
-}
-```
+We also use the [`PoVDistributionV1Message`](../../types/network.md#pov-distribution) as our `NetworkMessage`, which are sent and received by the [Network Bridge](../utility/network-bridge.md)
 
 Here is the logic of the state machine:
 

roadmap/implementers-guide/src/node/backing/statement-distribution.md

@@ -4,21 +4,20 @@ The Statement Distribution Subsystem is responsible for distributing statements
 
 ## Protocol
 
-`ProtocolId`: `b"stmd"`, `PeerSet`: `Validation`
+`PeerSet`: `Validation`
 
 Input:
 
 - NetworkBridgeUpdate(update)
 
 Output:
 
-- NetworkBridge::RegisterEventProducer(`ProtocolId`)
-- NetworkBridge::SendMessage(`[PeerId]`, `ProtocolId`, `Bytes`)
+- NetworkBridge::SendMessage(`[PeerId]`, message)
 - NetworkBridge::ReportPeer(PeerId, cost_or_benefit)
 
 ## Functionality
 
-Implemented as a gossip protocol. Register a network event producer on startup. Handle updates to our view and peers' views. Neighbor packets are used to inform peers which chain heads we are interested in data for.
+Implemented as a gossip protocol. Handle updates to our view and peers' views. Neighbor packets are used to inform peers which chain heads we are interested in data for.
 
 Statement Distribution is the only backing subsystem which has any notion of peer nodes, who are any full nodes on the network. Validators will also act as peer nodes.
 

roadmap/implementers-guide/src/node/collators/collator-protocol.md

@@ -24,21 +24,7 @@ Output:
 
 This network protocol uses the `Collation` peer-set of the [`NetworkBridge`][NB].
 
-```rust
-type RequestId = u64;
-
-enum WireMessage {
-	/// Declare the intent to advertise collations under a collator ID.
-	Declare(CollatorId),
-	/// Advertise a collation to a validator. Can only be sent once the peer has declared
-	/// that they are a collator with given ID.
-	AdvertiseCollation(Hash, ParaId),
-	/// Request the advertised collation at that relay-parent.
-	RequestCollation(RequestId, Hash, ParaId),
-	/// A requested collation.
-	Collation(RequestId, CandidateReceipt, PoV),
-}
-```
+It uses the [`CollatorProtocolV1Message`](../../types/network.md#collator-protocol) as its `WireMessage`
 
 Since this protocol functions both for validators and collators, it is easiest to go through the protocol actions for each of them separately.
 

roadmap/implementers-guide/src/node/utility/network-bridge.md

@@ -6,55 +6,118 @@ The most notable challenge is coordinating and eliminating race conditions of pe
 
 One other piece of shared state to track is peer reputation. When peers are found to have provided value or cost, we adjust their reputation accordingly.
 
-So in short, this Subsystem acts as a bridge between an actual network component and a subsystem's protocol.
+So in short, this Subsystem acts as a bridge between an actual network component and a subsystem's protocol. The implementation of the underlying network component is beyond the scope of this module. We make certain assumptions about the network component:
+  * The network allows registering of protocols and multiple versions of each protocol.
+  * The network handles version negotiation of protocols with peers and only connects the peer on the highest version of the protocol.
+  * Each protocol has its own peer-set, although there may be some overlap.
+  * The network provides peer-set management utilities for discovering the peer-IDs of validators and a means of dialing peers with given IDs.
 
-The other component of the network bridge is which peer-set to use. Different peer-sets can be connected for different purposes. The network bridge is not generic over peer-set, but instead exposes two peer-sets that event producers can attach to: `Validation` and `Collation`. More information can be found on the documentation of the [`NetworkBridgeMessage`][NBM].
+
+The network bridge makes use of the peer-set feature, but is not generic over peer-set. Instead, it exposes two peer-sets that event producers can attach to: `Validation` and `Collation`. More information can be found on the documentation of the [`NetworkBridgeMessage`][NBM].
 
 ## Protocol
 
 Input: [`NetworkBridgeMessage`][NBM]
-Output: Varying, based on registered event producers.
+
+
+Output:
+	- [`AvailabilityDistributionMessage`][AvD]`::NetworkBridgeUpdateV1`
+	- [`BitfieldDistributionMessage`][BitD]`::NetworkBridgeUpdateV1`
+	- [`PoVDistributionMessage`][PoVD]`::NetworkBridgeUpdateV1`
+	- [`StatementDistributionMessage`][StmtD]`::NetworkBridgeUpdateV1`
+	- [`CollatorProtocolMessage`][CollP]`::NetworkBridgeUpdateV1`
 
 ## Functionality
 
-Track a set of all Event Producers, each associated with a 4-byte protocol ID and the `PeerSet` it is associated on.
+This network bridge sends messages of these types over the network.
+
+```rust
+enum ProtocolMessage<M> {
+	ProtocolMessage(M),
+	ViewUpdate(View),
+}
+```
+
+and instantiates this type twice, once using the [`ValidationProtocolV1`][VP1] message type, and once with the [`CollationProtocolV1`][CP1] message type.
+
+```rust
+type ValidationV1Message = ProtocolMessage<ValidationProtocolV1>;
+type CollationV1Message = ProtocolMessage<CollationProtocolV1>;
+```
+
+### Startup
+
+On startup, we register two protocols with the underlying network utility. One for validation and one for collation. We register only version 1 of each of these protocols.
+
+### Main Loop
+
+The bulk of the work done by this subsystem is in responding to network events, signals from the overseer, and messages from other subsystems.
+
+Each network event is associated with a particular peer-set.
 
-There are two types of network messages this sends and receives:
+### Overseer Signal: ActiveLeavesUpdate
 
-- ProtocolMessage(ProtocolId, Bytes)
-- ViewUpdate(View)
+The `activated` and `deactivated` lists determine the evolution of our local view over time. A `ProtocolMessage::ViewUpdate` is issued to each connected peer on each peer-set, and a `NetworkBridgeEvent::OurViewChange` is issued to each event handler for each protocol.
 
-Each of these network messages is associated with a particular peer-set. If we are connected to the same peer on both peer-sets, we will receive two `ViewUpdate`s from them every time they change their view.
+If we are connected to the same peer on both peer-sets, we will send the peer two view updates as a result.
 
-`ActiveLeavesUpdate`'s `activated` and `deactivated` lists determine the evolution of our local view over time. A `ViewUpdate` is issued to each connected peer after each update, and a `NetworkBridgeUpdate::OurViewChange` is issued for each registered event producer.
+### Network Event: Peer Connected
 
-On `RegisterEventProducer`:
+Issue a `NetworkBridgeEvent::PeerConnected` for each [Event Handler](#event-handlers) of the peer-set and negotiated protocol version of the peer.
 
-- Add the event producer to the set of event producers. If there is a competing entry, ignore the request.
+### Network Event: Peer Disconnected
 
-On `ProtocolMessage` arrival:
+Issue a `NetworkBridgeEvent::PeerDisconnected` for each [Event Handler](#event-handlers) of the peer-set and negotiated protocol version of the peer.
 
-- If the protocol ID matches an event producer, produce the message from the `NetworkBridgeEvent::PeerMessage(sender, bytes)`, otherwise ignore and reduce peer reputation slightly
-- dispatch message via overseer.
+### Network Event: ProtocolMessage
 
-On `ViewUpdate` arrival:
+Map the message onto the corresponding [Event Handler](#event-handlers) based on the peer-set this message was received on and dispatch via overseer.
 
-- Do validity checks and note the most recent view update of the peer.
-- For each event producer, dispatch the result of a `NetworkBridgeEvent::PeerViewChange(view)` via overseer.
+### Network Event: ViewUpdate
 
-On `ReportPeer` message:
+- Check that the new view is valid and note it as the most recent view update of the peer on this peer-set.
+- Map a `NetworkBridgeEvent::PeerViewChange` onto the corresponding [Event Handler](#event-handlers) based on the peer-set this message was received on and dispatch  via overseer.
+
+### ReportPeer
 
 - Adjust peer reputation according to cost or benefit provided
 
-On `SendMessage` message:
+### SendValidationMessage
 
-- Issue a corresponding `ProtocolMessage` to each listed peer with given protocol ID and bytes.
+- Issue a corresponding `ProtocolMessage` to each listed peer on the validation peer-set.
 
-[NBM]: ../../types/overseer-protocol.md#network-bridge-message
+### SendCollationMessage
+
+- Issue a corresponding `ProtocolMessage` to each listed peer on the collation peer-set.
 
-On `ConnectToValidators` message:
+### ConnectToValidators
 
 - Determine the DHT keys to use for each validator based on the relay-chain state and Runtime API.
 - Recover the Peer IDs of the validators from the DHT. There may be more than one peer ID per validator.
 - Accumulate all `(ValidatorId, PeerId)` pairs and send on the response channel.
-- Feed all Peer IDs to the discovery utility the underlying network provides.
+- Feed all Peer IDs to peer set manager the underlying network provides, indicating the expected peer-set.
+
+## Event Handlers
+
+Network bridge event handlers are the intended recipients of particular network protocol messages. These are each a variant of a message to be sent via the overseer.
+
+### Validation V1
+
+* `StatementDistributionV1Message -> StatementDistributionMessage::NetworkBridgeUpdateV1`
+* `PoVDistributionV1Message -> PoVDistributionMessage::NetworkBridgeUpdateV1`
+* `AvailabilityDistributionV1Message -> AvailabilityDistributionMessage::NetworkBridgeUpdateV1`
+* `BitfieldDistributionV1Message -> BitfieldDistributionMessage::NetworkBridgeUpdateV1`
+
+### Collation V1
+
+* `CollatorProtocolV1Message -> CollatorProtocolMessage::NetworkBridgeUpdateV1`
+
+[NBM]: ../../types/overseer-protocol.md#network-bridge-message
+[AvD]: ../../types/overseer-protocol.md#availability-distribution-message
+[BitD]: ../../types/overseer-protocol.md#bitfield-distribution-message
+[PoVD]: ../../types/overseer-protocol.md#pov-distribution-message
+[StmtD]: ../../types/overseer-protocol.md#statement-distribution-message
+[CollP]: ../../types/overseer-protocol.md#collator-protocol-message
+
+[VP1]: ../../types/network.md#validation-v1
+[CP1]: ../../types/network.md#collation-v1