Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use the block import notification fired after the completion of block import on the domain side #1217

Merged
merged 5 commits into from
Mar 14, 2023

Conversation

liuchengxu
Copy link
Contributor

  • The main change in this PR is changing the source of primary block import notification, the notification from sc-consensus-subspace is replaced by the one provided by the native substrate client, which is fired after the entire block import is finished.
  • In addition, unlike reusing the primary block fork choice, now the domain always apply the longest chain fork choice when building the domain block since the new block import notification does not have this info and it's fine to set the longest chain on the domain.
  • Although the new block import stream is used now, the one from sc-consensus-subspace is kept as we still need to pause the primary block import if needed, but the semantic of block import throttling size is slight changed, now when the block import throttling buffer size is 100 blocks, the primary chain may be ahead of the domain chain up to 50 (100/2) blocks.

Close #1143

Code contributor checklist:

Unlike the block import notification from the subspace consensus fired before
the completion of block import, `every_block_import_stream()` in client
will notify the every imported primary block after the completion of block
import pipeline.

Although now domain actually consumes the block import notification from
the client, we still keep the notification from sc-consensus-subspace in
order to pause the block import when needed.

The sematic of block import throttling size is changed but should be
fine, previously the buffer was 100 blocks, which means the primary
chain can be ahead of the domain up to 100 blocks, now the buffer size
of 100 blocks means the primary chain may can only ahead of the domain
up to 50 blocks.
fork_choice is no longer needed in the sc-consensus-subspace block
import notification and domain block info.
Copy link
Member

@vedhavyas vedhavyas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make sense overall

domains/client/domain-executor/src/domain_worker.rs Outdated Show resolved Hide resolved
domains/client/domain-executor/src/domain_worker.rs Outdated Show resolved Hide resolved
NingLin-P
NingLin-P previously approved these changes Mar 13, 2023
Copy link
Member

@NingLin-P NingLin-P left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sense overall!

domains/client/domain-executor/src/domain_worker.rs Outdated Show resolved Hide resolved
@nazar-pc
Copy link
Member

Just to confirm, this still supports starting executor later after primary chain synced to some height, correct? This was a question from SDK team and generally important feature to keep.

@liuchengxu
Copy link
Contributor Author

this still supports starting executor later after primary chain synced to some height

No, and it's never true actually. This PR remains the same executor behavior as before, only the implementation detail is changed. If you want to run executor, it has to be specified when the primary node starts from genesis at present, but we can support this feature if the primary node keeps all the historical blocks. @nazar-pc

@liuchengxu liuchengxu enabled auto-merge March 14, 2023 15:00
Copy link
Member

@nazar-pc nazar-pc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like I didn't submit my review (probably didn't finish reading, but anyway). Non-blocking, just a few things I was curious about.

domains/client/domain-executor/src/domain_worker.rs Outdated Show resolved Hide resolved
// from the corresponding primary block, it's fine to forcibly always use
// the longest chain for simplicity as we manually build all the domain
// branches by literally following the primary chain branches anyway.
let fork_choice = ForkChoiceStrategy::LongestChain;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you remind me: do we finalize on execution chain the same blocks that are finalized on the primary chain?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, we don't follow the finalize event from primary chain right now. We briefly talked about this before, but never made it on the list :P, tracked in #1246 now.

@liuchengxu liuchengxu merged commit f63997f into main Mar 14, 2023
@liuchengxu liuchengxu deleted the switch-to-client-every-block-import-notification branch March 14, 2023 17:40
i1i1 added a commit to autonomys/subspace-pulsar-sdk that referenced this pull request Mar 20, 2023
i1i1 added a commit to autonomys/subspace-pulsar-sdk that referenced this pull request Mar 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants