-
Notifications
You must be signed in to change notification settings - Fork 432
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[raycluster controller] Always honor maxReplicas #662
[raycluster controller] Always honor maxReplicas #662
Conversation
Signed-off-by: Dmitri Gekhtman <[email protected]>
cc also @sihanwang41 @architkulkarni |
Signed-off-by: Dmitri Gekhtman <[email protected]>
Will go ahead and add a test. |
Looks good to me! |
Signed-off-by: Dmitri Gekhtman <[email protected]>
Will try it out manually as well. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank @DmitriGekhtman for this contribution!
Confirming that I tested it and it works as expected. I think the behavior change should be fairly uncontroversial. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Let's wait for CI to finish.
* Honor max replicas Signed-off-by: Dmitri Gekhtman <[email protected]> * Tweak message. Signed-off-by: Dmitri Gekhtman <[email protected]> * Add to test logic. Signed-off-by: Dmitri Gekhtman <[email protected]> Signed-off-by: Dmitri Gekhtman <[email protected]>
Signed-off-by: Dmitri Gekhtman [email protected]
Why are these changes needed?
If maxReplicas < Replicas, treat maxReplicas as the target replica count and log the discrepancy.
Partly addresses #560 in which some entity (possibly the Ray autoscaler) set replicas > maxReplicas.
See ray-project/ray#29770 for the corresponding safeguard in the autoscaler code.
Related issue number
Checks