Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Merged by Bors] - miner: multi signers #5135

Closed
wants to merge 30 commits into from

Conversation

dshulyak
Copy link
Contributor

@dshulyak dshulyak commented Oct 9, 2023

closes: #5087

data is separated into shared data (beacon and active set) and signer specific data. both beacon and activeset are used from shared data, until smeshers generated a reference ballot. once any smesher generated a ballot it will be using data recorded in the reference ballot.

build method now loops over all signers (copied at the start of the layer). there are parts that have to be run once for all signers and parts that makes sense to run in parallel.

serial parts:

  • loading share data (beacon and active set)
  • deciding on mesh hash
  • tally votes & encode votes tortoise calls

parallel parts:

  • loading data (it can be also run serially, but it was convenient to run it in parallel)
  • computing eligibilities (this is done once per node startup)
  • selecting txs
  • publishing proposal. this is the most important to avoid blocking serially in Publish while it runs validation

worker pool (errgroup) is limited by number of cores as there is no network requests during parallel work.

@codecov
Copy link

codecov bot commented Oct 9, 2023

Codecov Report

Merging #5135 (26af5ac) into develop (156d084) will increase coverage by 0.1%.
The diff coverage is 90.0%.

@@            Coverage Diff            @@
##           develop   #5135     +/-   ##
=========================================
+ Coverage     77.6%   77.7%   +0.1%     
=========================================
  Files          259     259             
  Lines        30457   30750    +293     
=========================================
+ Hits         23644   23909    +265     
- Misses        5322    5350     +28     
  Partials      1491    1491             
Files Coverage Δ
miner/metrics.go 100.0% <100.0%> (ø)
proposals/util.go 85.7% <100.0%> (ø)
miner/proposal_builder.go 95.4% <97.5%> (+1.7%) ⬆️
node/node.go 67.3% <84.6%> (+3.4%) ⬆️

... and 3 files with indirect coverage changes

@dshulyak dshulyak marked this pull request as ready for review October 13, 2023 03:53
miner/proposal_builder.go Outdated Show resolved Hide resolved
miner/proposal_builder.go Outdated Show resolved Hide resolved
miner/proposal_builder.go Outdated Show resolved Hide resolved
miner/proposal_builder.go Outdated Show resolved Hide resolved
miner/proposal_builder.go Outdated Show resolved Hide resolved
@dshulyak
Copy link
Contributor Author

bors merge

bors bot pushed a commit that referenced this pull request Oct 16, 2023
closes: #5087

data is separated into shared data (beacon and active set) and signer specific data. both beacon and activeset are used from shared data, until smeshers generated a reference ballot. once any smesher generated a ballot it will be using data recorded in the reference ballot. 

build method now loops over all signers (copied at the start of the layer). there are parts that have to be run once for all signers and parts that makes sense to run in parallel.

serial parts:
- loading share data (beacon and active set)
- deciding on mesh hash
- tally votes & encode votes tortoise calls

parallel parts:
- loading data (it can be also run serially, but it was convenient to run it in parallel)
- computing eligibilities (this is done once per node startup)
- selecting txs
- publishing proposal. this is the most important to avoid blocking serially in Publish while it runs validation

worker pool (errgroup) is limited by number of cores as there is no network requests during parallel work.
@bors
Copy link

bors bot commented Oct 16, 2023

Build failed:

  • ci-status

@dshulyak
Copy link
Contributor Author

bors merge

bors bot pushed a commit that referenced this pull request Oct 16, 2023
closes: #5087

data is separated into shared data (beacon and active set) and signer specific data. both beacon and activeset are used from shared data, until smeshers generated a reference ballot. once any smesher generated a ballot it will be using data recorded in the reference ballot. 

build method now loops over all signers (copied at the start of the layer). there are parts that have to be run once for all signers and parts that makes sense to run in parallel.

serial parts:
- loading share data (beacon and active set)
- deciding on mesh hash
- tally votes & encode votes tortoise calls

parallel parts:
- loading data (it can be also run serially, but it was convenient to run it in parallel)
- computing eligibilities (this is done once per node startup)
- selecting txs
- publishing proposal. this is the most important to avoid blocking serially in Publish while it runs validation

worker pool (errgroup) is limited by number of cores as there is no network requests during parallel work.
@bors
Copy link

bors bot commented Oct 16, 2023

Build failed:

@dshulyak
Copy link
Contributor Author

bors merge

bors bot pushed a commit that referenced this pull request Oct 16, 2023
closes: #5087

data is separated into shared data (beacon and active set) and signer specific data. both beacon and activeset are used from shared data, until smeshers generated a reference ballot. once any smesher generated a ballot it will be using data recorded in the reference ballot. 

build method now loops over all signers (copied at the start of the layer). there are parts that have to be run once for all signers and parts that makes sense to run in parallel.

serial parts:
- loading share data (beacon and active set)
- deciding on mesh hash
- tally votes & encode votes tortoise calls

parallel parts:
- loading data (it can be also run serially, but it was convenient to run it in parallel)
- computing eligibilities (this is done once per node startup)
- selecting txs
- publishing proposal. this is the most important to avoid blocking serially in Publish while it runs validation

worker pool (errgroup) is limited by number of cores as there is no network requests during parallel work.
@bors
Copy link

bors bot commented Oct 16, 2023

Build failed:

@dshulyak
Copy link
Contributor Author

bors merge

bors bot pushed a commit that referenced this pull request Oct 16, 2023
closes: #5087

data is separated into shared data (beacon and active set) and signer specific data. both beacon and activeset are used from shared data, until smeshers generated a reference ballot. once any smesher generated a ballot it will be using data recorded in the reference ballot. 

build method now loops over all signers (copied at the start of the layer). there are parts that have to be run once for all signers and parts that makes sense to run in parallel.

serial parts:
- loading share data (beacon and active set)
- deciding on mesh hash
- tally votes & encode votes tortoise calls

parallel parts:
- loading data (it can be also run serially, but it was convenient to run it in parallel)
- computing eligibilities (this is done once per node startup)
- selecting txs
- publishing proposal. this is the most important to avoid blocking serially in Publish while it runs validation

worker pool (errgroup) is limited by number of cores as there is no network requests during parallel work.
@bors
Copy link

bors bot commented Oct 16, 2023

Build failed:

@dshulyak
Copy link
Contributor Author

bors merge

bors bot pushed a commit that referenced this pull request Oct 16, 2023
closes: #5087

data is separated into shared data (beacon and active set) and signer specific data. both beacon and activeset are used from shared data, until smeshers generated a reference ballot. once any smesher generated a ballot it will be using data recorded in the reference ballot. 

build method now loops over all signers (copied at the start of the layer). there are parts that have to be run once for all signers and parts that makes sense to run in parallel.

serial parts:
- loading share data (beacon and active set)
- deciding on mesh hash
- tally votes & encode votes tortoise calls

parallel parts:
- loading data (it can be also run serially, but it was convenient to run it in parallel)
- computing eligibilities (this is done once per node startup)
- selecting txs
- publishing proposal. this is the most important to avoid blocking serially in Publish while it runs validation

worker pool (errgroup) is limited by number of cores as there is no network requests during parallel work.
@bors
Copy link

bors bot commented Oct 16, 2023

Pull request successfully merged into develop.

Build succeeded!

The publicly hosted instance of bors-ng is deprecated and will go away soon.

If you want to self-host your own instance, instructions are here.
For more help, visit the forum.

If you want to switch to GitHub's built-in merge queue, visit their help page.

@bors bors bot changed the title miner: multi signers [Merged by Bors] - miner: multi signers Oct 16, 2023
@bors bors bot closed this Oct 16, 2023
poszu pushed a commit that referenced this pull request Oct 17, 2023
closes: #5087

data is separated into shared data (beacon and active set) and signer specific data. both beacon and activeset are used from shared data, until smeshers generated a reference ballot. once any smesher generated a ballot it will be using data recorded in the reference ballot.

build method now loops over all signers (copied at the start of the layer). there are parts that have to be run once for all signers and parts that makes sense to run in parallel.

serial parts:
- loading share data (beacon and active set)
- deciding on mesh hash
- tally votes & encode votes tortoise calls

parallel parts:
- loading data (it can be also run serially, but it was convenient to run it in parallel)
- computing eligibilities (this is done once per node startup)
- selecting txs
- publishing proposal. this is the most important to avoid blocking serially in Publish while it runs validation

worker pool (errgroup) is limited by number of cores as there is no network requests during parallel work.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

tortoise: support multiple smeshers
2 participants