Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make BrokerSetAwareGoal capable of doing partition coloring #1862

Open
CCisGG opened this issue Jul 14, 2022 · 0 comments · May be fixed by #1864
Open

Make BrokerSetAwareGoal capable of doing partition coloring #1862

CCisGG opened this issue Jul 14, 2022 · 0 comments · May be fixed by #1864
Assignees
Labels
functionality A feature request.

Comments

@CCisGG
Copy link
Contributor

CCisGG commented Jul 14, 2022

Recently AWS has make contributions to CC and create this BrokerSetAwareGoal in: #1809. This goal will ensure each topic (all its partitions and replicas) only lives within a single broker set.

Today we have a need to do partition level coloring, meaning a partition and all its replicas live within a broker set. Different partitions of a topic doesn't need to be in the same color. With the current BrokerSetAwareGoal, it is not allowed. I'm thinking about to add partition coloring by either update the BrokerSetAwareGoal, or make BrokerSetAwareGoal easily extendable to achieve it.

Basically there are 2 things we need to change:

  1. For "ensureBrokerSetAware" method, we need to make it accept partition level coloring.
  2. We need to remove the _mustHaveTopicLeadersPerBroker affect. In topic coloring, the rebalance should ignore those topics; while in partition coloring, the rebalance should not ignore those topics.
@CCisGG CCisGG self-assigned this Jul 14, 2022
@CCisGG CCisGG added the functionality A feature request. label Aug 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
functionality A feature request.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant