Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs(quota plugins): note on quotas for internal users #10803

Merged
merged 6 commits into from
Nov 9, 2024
Merged

docs(quota plugins): note on quotas for internal users #10803

Changes from 3 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
a3c9e5d
docs(quota plugins): note on quotas for internal users
PaulRMellor Nov 5, 2024
b71249b
docs(review): edits to dow from comments by PW
PaulRMellor Nov 5, 2024
5d5ed70
docs(review): edits to doc from review by PM
PaulRMellor Nov 5, 2024
985bc9c
docs(review ): edits to doc from comments by JS
PaulRMellor Nov 6, 2024
d090b87
docs(review): edits to doc from comments by JS and KL
PaulRMellor Nov 7, 2024
ffbe38d
docs(review): edits to doc from comments by JS and KL -- 02
PaulRMellor Nov 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 12 additions & 3 deletions documentation/modules/managing/proc-setting-broker-limits.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,18 @@ Limits can be overridden by xref:con-configuring-client-quotas-str[user-specific
* CPU utilization limits for each client can be set as a percentage of the network threads and I/O threads on a per-broker basis.
* The number of concurrent partition creation and deletion operations (mutations) allowed per second can be set on a per-broker basis.

.Impact on internal users
When using the default Kafka quotas plugin, quotas are applied to all users.
PaulRMellor marked this conversation as resolved.
Show resolved Hide resolved
This includes internal users such as the Topic Operator and Cruise Control, which may impact their operations.

For example, configuring a controller mutation rate quota may result in the Topic Operator being throttled during topic creation or deletion operations.
PaulRMellor marked this conversation as resolved.
Show resolved Hide resolved
Monitoring of relevant controller and broker metrics can help track the rate of operations on topics.
Cruise Control and its metrics reporter also require sufficient produce and fetch rates to conduct rebalances, depending on the scale and configuration of the Kafka cluster.
To prevent issues for Cruise Control, you might start with a rate of at least 1 KB/s for its producers and consumers in small clusters, such as three brokers with moderate traffic, and adjust as needed for larger or more active clusters.

Unlike the Strimzi Quotas plugin, where internal users can be excluded using the `excludedPrincipals` option, the Kafka quotas plugin does not have an exclusion mechanism.
If you want to avoid unduly limiting internal users, consider tuning the quotas effectively or using the Strimzi Quotas plugin where exclusion is possible.
PaulRMellor marked this conversation as resolved.
Show resolved Hide resolved

.Prerequisites

* The Cluster Operator that manages the Kafka cluster is running.
Expand Down Expand Up @@ -104,9 +116,6 @@ spec:

. Apply the changes to the `Kafka` configuration.

NOTE: `minAvailableBytesPerVolume` and `minAvailableRatioPerVolume` are mutually exclusive.
This means that only one of these parameters should be configured.

NOTE: Additional options can be configured in the `spec.kafka.config` section.
The full list of supported options can be found in the https://github.com/strimzi/kafka-quotas-plugin?tab=readme-ov-file#properties-and-their-defaults[plugin documentation].

Expand Down
Loading