scheduler: unify leaderSolver and peerSolver in hot read scheduler #3925

lhy1024 · 2021-07-30T06:09:21Z

Signed-off-by: lhy1024 [email protected]

What problem does this PR solve?

Previously, we gave priority to the transfer leader and only used the move leader when there was no transfer leader, which is not necessarily the optimal solution, so this pr will be compared uniformly and then selected

What is changed and how it works?

Check List

Tests

Unit test

Release note

unify leaderSolver and peerSolver in hot read scheduler

Signed-off-by: lhy1024 <[email protected]>

ti-chi-bot · 2021-07-30T06:09:22Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

Yisaer
rleungx

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

Signed-off-by: lhy1024 <[email protected]>

codecov · 2021-08-01T07:34:25Z

Codecov Report

Merging #3925 (657e9a6) into master (3b920d7) will decrease coverage by 0.06%.
The diff coverage is 72.22%.

@@            Coverage Diff             @@
##           master    #3925      +/-   ##
==========================================
- Coverage   74.71%   74.64%   -0.07%     
==========================================
  Files         246      246              
  Lines       25117    25131      +14     
==========================================
- Hits        18767    18760       -7     
- Misses       4695     4717      +22     
+ Partials     1655     1654       -1

Flag	Coverage Δ
unittests	`74.64% <72.22%> (-0.07%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
server/schedulers/hot_region.go	`85.54% <72.22%> (-1.08%)`	⬇️
server/schedulers/shuffle_hot_region.go	`55.10% <0.00%> (-10.21%)`	⬇️
pkg/dashboard/adapter/manager.go	`82.97% <0.00%> (-7.45%)`	⬇️
server/statistics/hot_cache_task.go	`75.00% <0.00%> (-4.17%)`	⬇️
server/schedulers/random_merge.go	`60.00% <0.00%> (-3.34%)`	⬇️
server/core/storage.go	`67.68% <0.00%> (-1.53%)`	⬇️
server/server.go	`71.12% <0.00%> (-1.22%)`	⬇️
server/schedule/operator/step.go	`68.10% <0.00%> (-1.00%)`	⬇️
pkg/btree/btree.go	`85.30% <0.00%> (-0.97%)`	⬇️
server/grpc_service.go	`46.89% <0.00%> (-0.57%)`	⬇️
... and 10 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3b920d7...657e9a6. Read the comment docs.

Yisaer

rest lgtm

server/schedulers/hot_region.go

rleungx · 2021-08-02T09:02:49Z

server/schedulers/hot_region.go

 	}
+	if len(leaderOps) == 0 && peerSolver.addPendingInfluence() {


Previously, we prefer to balance by the leader. In this PR, it changes?

yes, this pr will compared them and then selected

server/schedulers/hot_region_test.go

Signed-off-by: lhy1024 <[email protected]>

Yisaer

rest lgtm

server/schedulers/hot_region.go

Signed-off-by: lhy1024 <[email protected]>

server/schedulers/hot_region.go

server/schedulers/hot_region_test.go

Signed-off-by: lhy1024 <[email protected]>

server/schedulers/hot_region.go

Signed-off-by: lhy1024 <[email protected]>

rleungx · 2021-08-04T08:22:49Z

server/schedulers/hot_region.go

@@ -393,15 +419,15 @@ func (h *hotScheduler) balanceHotWriteRegions(cluster opt.Cluster) []*operator.O
 	case s < int(schedulePeerPr*100):
 		peerSolver := newBalanceSolver(h, cluster, write, movePeer)
 		ops := peerSolver.solve()
-		if len(ops) > 0 {
+		if len(ops) > 0 && peerSolver.tryAddPendingInfluence() {


Perhaps, we also need to do a comparison for write. The rest LGTM.

Signed-off-by: lhy1024 <[email protected]>

lhy1024 · 2021-08-04T11:53:04Z

/merge

ti-chi-bot · 2021-08-04T11:53:05Z

@lhy1024: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot · 2021-08-04T11:53:06Z

This pull request has been accepted and is ready to merge.

Commit hash: 657e9a6

scheduler: unify leaderSolver and peerSolver in hot read scheduler

06e8157

Signed-off-by: lhy1024 <[email protected]>

ti-chi-bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. labels Jul 30, 2021

ti-chi-bot requested review from nolouch and rleungx July 30, 2021 06:09

fix test

feda187

Signed-off-by: lhy1024 <[email protected]>

lhy1024 marked this pull request as ready for review August 1, 2021 07:29

ti-chi-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 1, 2021

nolouch requested a review from Yisaer August 2, 2021 07:31

Yisaer reviewed Aug 2, 2021

View reviewed changes

server/schedulers/hot_region.go Outdated Show resolved Hide resolved

rleungx reviewed Aug 2, 2021

View reviewed changes

server/schedulers/hot_region_test.go Outdated Show resolved Hide resolved

add comment

b03f2a0

Signed-off-by: lhy1024 <[email protected]>

Yisaer approved these changes Aug 3, 2021

View reviewed changes

server/schedulers/hot_region.go Outdated Show resolved Hide resolved

ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Aug 3, 2021

lhy1024 added 2 commits August 3, 2021 17:23

Merge branch 'master' of github.com:tikv/pd into unify-hot-read

76b8341

address comment

4f76f91

Signed-off-by: lhy1024 <[email protected]>

rleungx reviewed Aug 3, 2021

View reviewed changes

server/schedulers/hot_region.go Outdated Show resolved Hide resolved

rleungx reviewed Aug 3, 2021

View reviewed changes

server/schedulers/hot_region_test.go Outdated Show resolved Hide resolved

server/schedulers/hot_region_test.go Outdated Show resolved Hide resolved

address comments

d9c2430

Signed-off-by: lhy1024 <[email protected]>

rleungx reviewed Aug 4, 2021

View reviewed changes

server/schedulers/hot_region.go Show resolved Hide resolved

address comment

1a538cb

Signed-off-by: lhy1024 <[email protected]>

lhy1024 mentioned this pull request Aug 4, 2021

Adjustable Priority Multidimensional Hot Scheduler Tasklist #3869

Closed

28 tasks

lhy1024 added the priority/P1 The issue has P1 priority. label Aug 4, 2021

rleungx reviewed Aug 4, 2021

View reviewed changes

rleungx approved these changes Aug 4, 2021

View reviewed changes

ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Aug 4, 2021

ti-chi-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 4, 2021

Merge branch 'master' of github.com:tikv/pd into unify-hot-read

186b931

Signed-off-by: lhy1024 <[email protected]>

ti-chi-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 4, 2021

lhy1024 added 2 commits August 4, 2021 18:06

update

6612bb5

Signed-off-by: lhy1024 <[email protected]>

fix test

657e9a6

Signed-off-by: lhy1024 <[email protected]>

HunDunDM approved these changes Aug 4, 2021

View reviewed changes

ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Aug 4, 2021

ti-chi-bot merged commit 9874714 into tikv:master Aug 4, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

scheduler: unify leaderSolver and peerSolver in hot read scheduler #3925

scheduler: unify leaderSolver and peerSolver in hot read scheduler #3925

lhy1024 commented Jul 30, 2021

ti-chi-bot commented Jul 30, 2021 •

edited

Loading

codecov bot commented Aug 1, 2021 •

edited

Loading

Yisaer left a comment

rleungx Aug 2, 2021

lhy1024 Aug 2, 2021

Yisaer left a comment

rleungx Aug 4, 2021

lhy1024 commented Aug 4, 2021

ti-chi-bot commented Aug 4, 2021

ti-chi-bot commented Aug 4, 2021

		}
		if len(leaderOps) == 0 && peerSolver.addPendingInfluence() {

scheduler: unify leaderSolver and peerSolver in hot read scheduler #3925

scheduler: unify leaderSolver and peerSolver in hot read scheduler #3925

Conversation

lhy1024 commented Jul 30, 2021

What problem does this PR solve?

What is changed and how it works?

Check List

Release note

ti-chi-bot commented Jul 30, 2021 • edited Loading

codecov bot commented Aug 1, 2021 • edited Loading

Codecov Report

Yisaer left a comment

Choose a reason for hiding this comment

rleungx Aug 2, 2021

Choose a reason for hiding this comment

lhy1024 Aug 2, 2021

Choose a reason for hiding this comment

Yisaer left a comment

Choose a reason for hiding this comment

rleungx Aug 4, 2021

Choose a reason for hiding this comment

lhy1024 commented Aug 4, 2021

ti-chi-bot commented Aug 4, 2021

ti-chi-bot commented Aug 4, 2021

ti-chi-bot commented Jul 30, 2021 •

edited

Loading

codecov bot commented Aug 1, 2021 •

edited

Loading