Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

optimize PreCommit1 task num #8767

Closed
wants to merge 6 commits into from
Closed

optimize PreCommit1 task num #8767

wants to merge 6 commits into from

Conversation

peter198
Copy link
Contributor

Introduction:
Group task scheduling optimization, adding a limit on the number of PreCommit1 tasks for workers

Statement of needs:
In the original scheduling, there is no limit to the number of workers running PreCommit1 at the same time, and the maximum task configuration size includes the number of CPU cores, available memory, and hard disk space.
There is no reasonable limit. Although workers can execute when they receive too many tasks, they cannot fully exert their computing performance. The parallel PreCommit1 tasks can be reasonably controlled according to the hardware configuration.
Can improve the overall task completion speed.

improve proposals:
When the worker starts, the --parallel-p1-limit parameter sets the maximum PreCommit1 task it can accept. It will be recorded when the miner schedules the PreCommit1 task to the worker
Quantity allocated to it When the number of tasks already allocated reaches the set maximum number of tasks, it will not be allocated to a new task until 1 task is completed, and the allocation will be submitted in advance.

The default value of the --parallel-p1-limit parameter is -1, which means no limit, and if --parallel-p1-limit--parallel-p1-limit--parallel-p1-limit exceeds all the values ​​​​of the hardware configuration of the on-duty worker maximum reception time of
won't work either.
When viewing worker information through lotus-miner sealed workers, the task number information of PreCommit1 will be displayed, as follows:
p1_limit: 15-7, 15 represents the value set by --parallel-p1-limit, and 7 represents the number of PreCommit1 tasks currently running.

peter198 added 2 commits May 31, 2022 11:46
optimize PreCommit1 task
add new change
@peter198 peter198 requested a review from a team as a code owner May 31, 2022 07:19
@peter198 peter198 closed this May 31, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant