Move libp2p,discv5,backfill to workers

[wemeetagain]

During team retreat we discussed the idea of parallelizing work across multiple threads as makes sense.

If anyone has more notes from that session add here please.

Some open questions:

• how should metrics be handled across workers?

[dapplion]

how should metrics be handled across workers?

I've been playing around with discv5 on a thread on this is the biggest hurdle I found so far.

Requirements:

• Functions can't be called directly, everything must be a message / event
• Metrics server will remain in the main thread, which can have metrics data already available or collect it from somewhere

Solutions:

• On every metrics call (metric.inc()) fire an event to upstream that data to main thread
• Collect metrics on worker, and then on scrape collect all of them with a single event

[dapplion]

Some notes

[wemeetagain]

Collect metrics on worker, and then on scrape collect all of them with a single event

Yeah thats what I was leaning towards. The worker maintains its own Registry. And the worker exposes a async metrics(): Promise<string> that basically looks like return await registry.metrics()

[wemeetagain]

I've been playing around with discv5 on a thread on this is the biggest hurdle I found so far.

Same 😂

The other weird thing I saw is that the threads library doesn't support the event listener pattern, only "observables". You can create an observable per event to listen on, its just a little weird.

[dapplion]

The other weird thing I saw is that the threads library doesn't support the event listener pattern, only "observables". You can create an observable per event to listen on, its just a little weird.

I think we should skip the threads library for now. Or connect to the raw's Worker events for events, and use threads for an easy wrapper for direct calls from the main thread

[wemeetagain]

Discv5 done, see #4988

[wemeetagain]

Looking into moving the network into a thread. Some annoyances/hurdles (that could be treated as refactoring prereqs):

• some consensus objects can't be structuredClone'd. Any BitArray gets converted into an object.
  • convert BitArray methods to functions?
  • build generic ssz type "rehydrator" to recursively convert any BitArray subproperties back into BitArrays?
  • transfer across thread boundary as bytes
    • start with this, only consider ^ solutions if there are problems with this
    • see if bytes can be transferred
• INetworkOptions can't be structuredCloned. Contains functions / instances
  • refactor so it can be structuredCloned
• Network submodules contain refs to chain / config
  • worker can own its own config and clock
  • any chain data required for init can be passed in worker data (initial validator count)
  • gossip validation and req/resp handlers can be performed in main thread
  • push chain.getStatus() into network worker on new head calculation
• libp2p initialized at a higher level and passed into network constructor
  • resolved in Refactor network/libp2p initialization #5131
  • initialize libp2p inside the network module / worker

[dapplion]

• Network worker must deal with ssz serialized bytes only. Handlers for reqresp and gossip must be performed in the main thread.
• Status handler can either be handled on the main thread on a pull basis, or the main thread forkchoice to push updates to network worker
• Config and clock should be cloned

[dapplion]

discv5 and libp2p moved to worker. Open dedicated issue to backfill when we implement it