Structured concurrency: Part 1

[rakanalh]

Description

From task_scope's documentation

What is Strucutured Concurrency?

Structured Concurrency is a programming paradigm that lets asynchronous operations run only within certain scopes so that they form an operation stack like a regular function call stack. As the parent operation waits until all children complete, Structured Concurrency helps local reasoning of concurrent programs.
Out-of-task concurrency considered harmful

Most of the asynchronous programs are composed of async functions and in-task concurrency primitives such as select and join, and this makes such futures automatically well-structured. As "futures do nothing unless polled," executions of inner (child) operations are very explicit (usually at the point of awaits). Moreover, canceling a future is done by dropping it, which will reclaim resources used for the operation, including the inner futures. This drop-chain propagates the cancellation to the very end of the operation stack.

Out-of-task concurrencies such as spawn, however, breaks the structure. They allow us to start a new execution unit that can escape from the parent stack. Although frameworks provide a way to join on the spawned task, they don't propagate cancellation properly. If you drop the spawning task, the spawned task may outlive the parent indefinitely.

Changes in this PR

This PR restructures the ethereum-rpc crate SubscriptionManager so that the list of subscriptions is preserved for as long as they are active, while processing new head and new logs notifications in a single worker which notifies all subscribers about new heads and new logs. That is, replacing the functionality of spawning an unmanaged task per subscription.

The PR also keeps track of the handles for the 3 spawned tasks:

1. new_heads_notifier
2. new_logs_notifier
3. soft_confirmation_event_handler

So that when SubscriptionManager is dropped, the tasks are also stopped. This could be improved in the future where these components can react the OS signals / cancellation tokens.

Initiative

Based on this initiative, i'd like to introduce a no tokio::spawn rule without storing the handle somewhere, where the spawned task is managed and is cancellable.

Linked Issues

• Related Research replacing tokio::spawns with a task manager #923

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 77.6%. Comparing base (94664da) to head (c89e4f4).
Report is 1 commits behind head on nightly.

Additional details and impacted files

Files with missing lines	Coverage Δ
crates/ethereum-rpc/src/lib.rs	88.9% <100.0%> (-0.1%)	⬇️
crates/ethereum-rpc/src/subscription.rs	100.0% <100.0%> (+4.0%)	⬆️
crates/ethereum-rpc/src/trace.rs	78.9% <100.0%> (+0.3%)	⬆️

... and 1 file with indirect coverage changes

[kpp]

That's a nice cleanup. The only thing I question is why you decided to move from broadcast to mpsc.

[rakanalh]

That's a nice cleanup. The only thing I question is why you decided to move from broadcast to mpsc.

broadcast is meant for multiple receivers. In this case, the node (for example sequencer) would send a new soft confirmation event to one receiver which is the new_heads_worker and the worker would notify all subscriptions.

[kpp]

We may use for ... { x.send(y) } in log subscription but new_heads looks better with broadcast.

[yaziciahmet]

LGTM other than a small nit I left above.

[yaziciahmet]

My only concern is that there is a high possibility that the soft_confirmation_tx will need multiple receivers eventually. Only receiver is SubscriptionManager right now, but I see it more fit to be broadcast as that event channel has no specific context and should notify anyone who wants to subscribe.

This comment might be irrelevant if we were to add another EventManager layer for chain-specific events, but it doesn't seem necessary in the near future as almost all the new incoming information is dependent on a new soft confirmation, and the specifics of the requested notification can be derived from that new soft confirmation.

[rakanalh]

My only concern is that there is a high possibility that the soft_confirmation_tx will need multiple receivers eventually. Only receiver is SubscriptionManager right now, but I see it more fit to be broadcast as that event channel has no specific context and should notify anyone who wants to subscribe.

We shouldn't make it use broadcast until we have a second receiver. Why use broadcast for something that has 1 receiver, if that's the case then mpsc would be the way to go for now. If we keep assuming future use cases then we build up tech-debt.

[rakanalh]

I restored using broadcast since pruning is one use case of another receiver.