placement: improve regoin fit

[disksing]

What problem does this PR solve?

When we fit region's peers into rules, we check rules one by one, and for each one, we pick the best option (if there are multiple best options we simply pick the first one). The problem is that the current choice will affect subsequent rules, so the current optimal option is not necessarily the global optimal option.

For example, a region has 4 peers: [host=h1, host=h2, host=h3, host=h4], and it matches 2 rules: 3 voters + 1 voter on h2. The best choice is [host=h1, host=h3, host=h4] goes to the first rule and [host=h2] goes to the second rule. Without consider globally, the first rule may pick [host=h1, host=h2, host=h3] and left [host=h4] which is not suitable for the second rule.

The problem is originally brought up from #2626

What is changed and how it works?

Recursive all options to choose the global best.

Check List

Tests

• Unit test

Release note

• Fix the issue that when enabling placement rules, sometimes region replicas cannot schedule to optimal

[Yisaer]

I test a case with

"1111,1221,1331,1241", []string{"3/voter/host=host1+host2+host3+host4/rack,host", "1/voter/host=host3/"

The result is that "1111,1221,1331" fit the first rule and no peer fits the second rule. Is the result expected?

[disksing]

I test a case with

"1111,1221,1331,1241", []string{"3/voter/host=host1+host2+host3+host4/rack,host", "1/voter/host=host3/"

The result is that "1111,1221,1331" fit the first rule and no peer fits the second rule. Is the result expected?

It is expedted. for rule1, 1111,1221,1331 is better than 1111,1221,1241.

It is the same situation with my test case {"1111,2211,3111,3112", []string{"3/voter//zone", "1/voter/rack=rack2/"}, "1111,2211,3111//3112"}.

[Yisaer]

Thanks for clarifying. LGTM

[ti-srebot]

@Yisaer,Thanks for your review. However, LGTM is restricted to Reviewers or higher roles.See the corresponding SIG page for more information. Related SIGs: scheduling(slack).

[JmPotato]

LGTM

[ti-srebot]

@JmPotato,Thanks for your review. However, LGTM is restricted to Reviewers or higher roles.See the corresponding SIG page for more information. Related SIGs: scheduling(slack).

[disksing]

/run-all-tests

[disksing]

/run-integration-ddl-test

[rleungx]

/run-cherry-picker

[ti-srebot]

cherry pick to release-4.0 in PR #2720