Mesh synchronisation protocol

[zalmen]

Design and implement the protocol that will allow a node to acquire all blocks from which the current consensus can be proved. This epic sets the requirements only for the protocol that fetches the blocks, requirements for processing the blocks and maintaining the internal state is set here *** (TODO - add link)

References from Ethereum:
Sync protocol - https://github.com/ethereumproject/go-ethereum/wiki/Blockchain-Synchronisation
Fast sync - ethereum/go-ethereum#1889

Based on code inspection -
In the handshake of every new connections the peers exchange basic sync info - genesis block, total difficulty, network id and current block (head). Once the handshake is completed the new peer is registered locally and every new block propagation, announcement and transactions are broadcast to the remote peer. Upon registration the node sends to the new peer all existing transactions

Cases where sync is triggered:

1. 741 @ handler.go: Upon receiving NewBlockMsg, if the block’s src has total difficulty (without counting the block itself) that is larger than the local TD.
2. 143 @ sync.go: Once a connection is established (either inbound or outbound) a sync is triggered with the “best” peer (highest TD) in the list

The sync process -

1. Fetch headers - The best peer fetches a skeleton of the list of all headers (a subset of the chain’s headers with a header every ‘skip’ number of blocks). While the skeleton is fetched from the best peer the rest of the peers are queried for headers to fill the holes in the skeleton until all headers are fetched.
2. Fetch bodies - Fetch bodies for collected headers from all peers
3. Fetch receipts - Not sure why is it needed for full sync

Open questions:

1. In Etheruem blocks are fetched in sections (i.e. in batches of blocks that are a subchain in the blockchain), this way allows the node to verify that the section is valid by validating the hash pointers. What is the equivalent batch in our case?
2. How can we verify that we got all blocks in the most recent layer (where there are no other blocks that points to the blocks in this layer)?
3. How can we validate that a block is syntactically valid without having the full history?
4. What quality of the mesh can a peer export in order to share the "age" of the mesh that it is holding (the equivalent to total difficulty in Ethereum). We need some value with which a node can pick the "best" peer the sync from.