fix: Avoid controller crashes when running large number of workflows #9691
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Yuan Tang <[email protected]>
jessesuen
reviewed
Sep 27, 2022
Signed-off-by: Yuan Tang <[email protected]>
terrytangyuan
changed the title
Signed-off-by: Yuan Tang <[email protected]>
jessesuen
reviewed
Sep 27, 2022
| | `INDEX_WORKFLOW_SEMAPHORE_KEYS` | `bool` | `true` | Whether or not to index semaphores. | | ||
| | `LEADER_ELECTION_IDENTITY` | `string` | Controller's `metadata.name` | The ID used for workflow controllers to elect a leader. | | ||
| | `LEADER_ELECTION_DISABLE` | `bool` | `false` | Whether leader election should be disabled. | | ||
| | `LEADER_ELECTION_LEASE_DURATION` | `time.Duration` | `15s` | The duration that non-leader candidates will wait to force acquire leadership. | | ||
| | `LEADER_ELECTION_RENEW_DEADLINE` | `time.Duration` | `10s` | The duration that the acting master will retry refreshing leadership before giving up. | | ||
| | `LEADER_ELECTION_RETRY_PERIOD` | `time.Duration` | `5s` | The duration that the leader election clients should wait between tries of actions. | | ||
| | `LIST_LIMIT` | `int` | `200` | The maximum number of responses to return for a list call on workflows for workflow informer. | |
There was a problem hiding this comment.
This needs to be removed, right?
jessesuen
approved these changes
Sep 27, 2022
jessesuen
reviewed
Sep 27, 2022
| @@ -803,6 +803,7 @@ func (wfc *WorkflowController) tweakListOptions(options *metav1.ListOptions) { | |||
| labelSelector := labels.NewSelector(). | |||
| Add(util.InstanceIDRequirement(wfc.Config.InstanceID)) | |||
| options.LabelSelector = labelSelector.String() | |||
| options.Limit = int64(env.LookupEnvIntOr("LIST_LIMIT", 200)) | |||
There was a problem hiding this comment.
Will this paginate properly? Or will it simply cut it off at 200
chenyangxueHDU
pushed a commit
to chenyangxueHDU/argo
that referenced
this pull request
Sep 29, 2022
…rgoproj#9691) Signed-off-by: Yuan Tang <[email protected]> Signed-off-by: yangxue.chen <[email protected]>
chenyangxueHDU
pushed a commit
to chenyangxueHDU/argo
that referenced
this pull request
Sep 29, 2022
…rgoproj#9691) Signed-off-by: Yuan Tang <[email protected]> Signed-off-by: yangxue.chen <[email protected]>
juchaosong
pushed a commit
to juchaosong/argo-workflows
that referenced
this pull request
Nov 3, 2022
…rgoproj#9691) Signed-off-by: Yuan Tang <[email protected]> Signed-off-by: juchao <[email protected]>
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #8275.
Signed-off-by: Yuan Tang [email protected]