Overseer: allow subsystems to instruct the overseer to send signals

[rphmeier]

We should create a new message from subsystems to the overseer which informs the overseer to send a specific signal. We might introduce a generic type parameter SubsystemInitiatedSignal which would allow us to prevent subsystems from sending signals such as Conclude, ActiveLeavesUpdate, or other stuff that they shouldn't.

cc @drahnr as the overseer expert

[rphmeier]

My gut instinct says that this can be done against master and not against the asynchronous backing feature branch, even though this is pre-requisite to async backing. SubsystemInitiatedSignal could be enum Void {} in master.

[drahnr]

I think this has two components:

1. receive a message
2. send the message

If there is no logic attached to it, or that logic resides within a small set of subsystems, directly calling fn broadcast_signal might be an option as well.

polkadot/node/overseer/overseer-gen/proc-macro/src/impl_overseer.rs

Line 130 in e721a5a

pub async fn broadcast_signal(&mut self, signal: #signal_ty) -> ::std::result::Result<(), #error_ty > {

Now one would need access to funnel the infor into Overseer::run, which can either be done as a separate stream or integrated with the existing Events, exposed with the SubsystemContext or trait SubsystemSender.

There are a few open questions to be clarified before starting the implementation:

1. do we require mpsc::Senders in those messages that we broadcast? Chances for cyclic dependencies that would not be detected by the planned cycle detection based on incoming/outgoing message types/variants.
2. what's the frequency of these?
3. what nature would those signals be, more events or more messages?
4. what's the synchronicity requirement regarding active leaves between different subsystems?

Effort seems to be reasonably bounded

[rphmeier]

1. do we require mpsc::Senders in those messages that we broadcast? Chances for cyclic dependencies that would not be detected by the planned cycle detection based on incoming/outgoing message types/variants.

No, I think they'd be pure data for any reasonable use-case. But I guess anything Clone can suffice.

2. what's the frequency of these?

Very frequent

3. what nature would those signals be, more events or more messages?

I don't understand the distinction really. specifically, stuff like the ViableCandidateBase from #4962 is the intended use-case.

4. what's the synchronicity requirement regarding active leaves between different subsystems?

The only requirement on signals and messaging is that a message sent after receipt of a signal on one subsystem may only be observed by another subsystem after the receipt of the same signal. This property should be the same.

[drahnr]

No 3 was a fuzzy one, thanks for clarifying the rest!

[rphmeier]

Right now we have

enum FromOverseer<M> {
    Signal(OverseerSignal),
    Communication { msg: M },
}

One way to achieve this would be to change the definition to

enum FromOverseer<M, US> {
    Signal(OverseerSignal),
    UserSignal(US),
    Communication { msg: M },
}

and then have trait Subsystem gain an associated type UserSignal (which is usually Void).

and then have the overseer be generic over US where the subsystems are all bounded on where US: From<Subsystem::UserSignal>

[drahnr]

If we go for the pull based retrieval, we can close this, or are there any remaining usecases?

[rphmeier]

There are possibly other use-cases but none in the near term

[drahnr]

Closing for now