Docs: Changes for coordinator improvements #14590
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
writer-jill
suggested changes
Jul 17, 2023
|
|
||
| The `smartSegmentLoading` mode of the Coordinator makes configuring it for segment loading and balancing much easier. | ||
| In this mode, the Coordinator does not require the user to provide values of the following parameters and computes them automatically instead. |
There was a problem hiding this comment.
Suggested change
| In this mode, the Coordinator does not require the user to provide values of the following parameters and computes them automatically instead. | |
| If you enable this mode, do not provide values for the properties in the table below.``` |
Co-authored-by: Jill Osborne <[email protected]>
|
Thanks for the quick review, @writer-jill ! I have incorporated your feedback into this PR. |
| |`replicationThrottleLimit`|The maximum number of segments that can be in the replication queue of a historical tier at any given time.|500| | ||
| |`balancerComputeThreads`|Thread pool size for computing moving cost of segments in segment balancing. Consider increasing this if you have a lot of segments and moving segments starts to get stuck.|1| | ||
| |`replicantLifetime`|The maximum number of Coordinator runs for which a segment can wait in the load queue of a Historical before Druid raises an alert.|15| | ||
| |`replicationThrottleLimit`|The maximum number of segment replicas that can be assigned to a historical tier in a single Coordinator run. This property prevents historicals from becoming overwhelmed when loading extra replicas of segments that are already available in the cluster.|500| |
There was a problem hiding this comment.
(Nit): The maximum number of non-primary segment replicas that can be assigned to a historical tier in a single Coordinator run. This property prevents historicals from becoming overwhelmed when loading extra replicas of segments that are already available in the tier.
There was a problem hiding this comment.
We are trying to get rid of the primary / non-primary terminology, since replica already means a copy. We will be completely removing the other config maxNonPrimaryReplicantsToLoad soon.
AmatyaAvadhanula
approved these changes
Jul 18, 2023
kfaraz
added a commit
to kfaraz/druid
that referenced
this pull request
Jul 18, 2023
This was referenced Jul 18, 2023
kfaraz
added a commit
that referenced
this pull request
Jul 18, 2023
kfaraz
added a commit
that referenced
this pull request
Jul 18, 2023
sergioferragut
pushed a commit
to sergioferragut/druid
that referenced
this pull request
Jul 21, 2023
sergioferragut
pushed a commit
to sergioferragut/druid
that referenced
this pull request
Jul 21, 2023
Changes: - Rephrased the description of `smartSegmentLoading` - Moved detail about value computation outside of quoted block as it is important.
AmatyaAvadhanula
pushed a commit
to AmatyaAvadhanula/druid
that referenced
this pull request
Aug 1, 2023
Changes: - Rephrased the description of `smartSegmentLoading` - Moved detail about value computation outside of quoted block as it is important.
AmatyaAvadhanula
added a commit
that referenced
this pull request
Aug 1, 2023
Changes: - Rephrased the description of `smartSegmentLoading` - Moved detail about value computation outside of quoted block as it is important. Co-authored-by: Kashif Faraz <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes for coordinator improvements done in