Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The Job Replica Count is Incorrect After Modifying minReplicaCount in ScaledJob #5980

Closed
SpiritZhou opened this issue Jul 25, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@SpiritZhou
Copy link
Contributor

Report

When there are some jobs running , modifying the minReplicaCount of ScaledJob will generate more replica than expected.

Expected Behavior

Generate correct replica according to metric value

Actual Behavior

Generate more replica than not following the metric value

Steps to Reproduce the Problem

  1. Deploy the ScaledJob resources in external_scaler_sj_test.js.
  2. Trigger the ScaledJob scale out. For example, update the metric template with data.MetricValue = 10, which should scale a job.
  3. After the job has scaled up, modify the minReplicaCount of ScaledJob to 1.
  4. The job's replica count should still be 1 since the metric value has not changed, but it will scale up to 2.

Logs from KEDA operator

2024-07-25T06:27:32Z    INFO    scaleexecutor    Scaling Jobs    {"scaledJob.Name": "external-scaler-sj-test-sj", "scaledJob.Namespace": "external-scaler-sj-test-ns", "Number of running Jobs": 1}
2024-07-25T06:27:32Z    INFO    scaleexecutor    Scaling Jobs    {"scaledJob.Name": "external-scaler-sj-test-sj", "scaledJob.Namespace": "external-scaler-sj-test-ns", "Number of pending Jobs": 0}
2024-07-25T06:27:32Z    INFO    scaleexecutor    Creating jobs    {"scaledJob.Name": "external-scaler-sj-test-sj", "scaledJob.Namespace": "external-scaler-sj-test-ns", "Effective number of max jobs": 0}
2024-07-25T06:27:32Z    INFO    scaleexecutor    Creating jobs    {"scaledJob.Name": "external-scaler-sj-test-sj", "scaledJob.Namespace": "external-scaler-sj-test-ns", "Number of jobs": 0}
2024-07-25T06:27:32Z    INFO    scaleexecutor    Created jobs    {"scaledJob.Name": "external-scaler-sj-test-sj", "scaledJob.Namespace": "external-scaler-sj-test-ns", "Number of jobs": 0}
2024-07-25T06:27:37Z    INFO    Reconciling ScaledJob    {"controller": "scaledjob", "controllerGroup": "keda.sh", "controllerKind": "ScaledJob", "ScaledJob": {"name":"external-scaler-sj-test-sj","namespace":"external-scaler-sj-test-ns"}, "namespace": "external-scaler-sj-test-ns", "name": "external-scaler-sj-test-sj", "reconcileID": "b7e24905-cc46-4959-afd8-b3d6397487ec"}
2024-07-25T06:27:37Z    INFO    RolloutStrategy: immediate, Deleting jobs owned by the previous version of the scaledJob    {"controller": "scaledjob", "controllerGroup": "keda.sh", "controllerKind": "ScaledJob", "ScaledJob": {"name":"external-scaler-sj-test-sj","namespace":"external-scaler-sj-test-ns"}, "namespace": "external-scaler-sj-test-ns", "name": "external-scaler-sj-test-sj", "reconcileID": "b7e24905-cc46-4959-afd8-b3d6397487ec", "numJobsToDelete": 1}
2024-07-25T06:27:37Z    INFO    Initializing Scaling logic according to ScaledJob Specification    {"controller": "scaledjob", "controllerGroup": "keda.sh", "controllerKind": "ScaledJob", "ScaledJob": {"name":"external-scaler-sj-test-sj","namespace":"external-scaler-sj-test-ns"}, "namespace": "external-scaler-sj-test-ns", "name": "external-scaler-sj-test-sj", "reconcileID": "b7e24905-cc46-4959-afd8-b3d6397487ec"}
2024-07-25T06:27:37Z    INFO    scaleexecutor    Scaling Jobs    {"scaledJob.Name": "external-scaler-sj-test-sj", "scaledJob.Namespace": "external-scaler-sj-test-ns", "Number of running Jobs": 0}
2024-07-25T06:27:37Z    INFO    scaleexecutor    Scaling Jobs    {"scaledJob.Name": "external-scaler-sj-test-sj", "scaledJob.Namespace": "external-scaler-sj-test-ns", "Number of pending Jobs": 0}
2024-07-25T06:27:37Z    INFO    scaleexecutor    Creating jobs    {"scaledJob.Name": "external-scaler-sj-test-sj", "scaledJob.Namespace": "external-scaler-sj-test-ns", "Effective number of max jobs": 1}
2024-07-25T06:27:37Z    INFO    scaleexecutor    Creating jobs    {"scaledJob.Name": "external-scaler-sj-test-sj", "scaledJob.Namespace": "external-scaler-sj-test-ns", "Number of jobs": 1}
2024-07-25T06:27:37Z    INFO    scaleexecutor    Created jobs    {"scaledJob.Name": "external-scaler-sj-test-sj", "scaledJob.Namespace": "external-scaler-sj-test-ns", "Number of jobs": 1}
2024-07-25T06:27:42Z    INFO    scaleexecutor    Scaling Jobs    {"scaledJob.Name": "external-scaler-sj-test-sj", "scaledJob.Namespace": "external-scaler-sj-test-ns", "Number of running Jobs": 1}
2024-07-25T06:27:42Z    INFO    scaleexecutor    Scaling Jobs    {"scaledJob.Name": "external-scaler-sj-test-sj", "scaledJob.Namespace": "external-scaler-sj-test-ns", "Number of pending Jobs": 0}
2024-07-25T06:27:42Z    INFO    scaleexecutor    Creating jobs    {"scaledJob.Name": "external-scaler-sj-test-sj", "scaledJob.Namespace": "external-scaler-sj-test-ns", "Effective number of max jobs": 1}
2024-07-25T06:27:42Z    INFO    scaleexecutor    Creating jobs    {"scaledJob.Name": "external-scaler-sj-test-sj", "scaledJob.Namespace": "external-scaler-sj-test-ns", "Number of jobs": 1}
2024-07-25T06:27:42Z    INFO    scaleexecutor    Created jobs    {"scaledJob.Name": "external-scaler-sj-test-sj", "scaledJob.Namespace": "external-scaler-sj-test-ns", "Number of jobs": 1}
2024-07-25T06:27:47Z    INFO    scaleexecutor    Scaling Jobs    {"scaledJob.Name": "external-scaler-sj-test-sj", "scaledJob.Namespace": "external-scaler-sj-test-ns", "Number of running Jobs": 2}
2024-07-25T06:27:47Z    INFO    scaleexecutor    Scaling Jobs    {"scaledJob.Name": "external-scaler-sj-test-sj", "scaledJob.Namespace": "external-scaler-sj-test-ns", "Number of pending Jobs": 0}
2024-07-25T06:27:47Z    INFO    scaleexecutor    Creating jobs    {"scaledJob.Name": "external-scaler-sj-test-sj", "scaledJob.Namespace": "external-scaler-sj-test-ns", "Effective number of max jobs": 0}
2024-07-25T06:27:47Z    INFO    scaleexecutor    Creating jobs    {"scaledJob.Name": "external-scaler-sj-test-sj", "scaledJob.Namespace": "external-scaler-sj-test-ns", "Number of jobs": 0}
2024-07-25T06:27:47Z    INFO    scaleexecutor    Created jobs    {"scaledJob.Name": "external-scaler-sj-test-sj", "scaledJob.Namespace": "external-scaler-sj-test-ns", "Number of jobs": 0}
2024-07-25T06:27:52Z    INFO    scaleexecutor    Scaling Jobs    {"scaledJob.Name": "external-scaler-sj-test-sj", "scaledJob.Namespace": "external-scaler-sj-test-ns", "Number of running Jobs": 2}
2024-07-25T06:27:52Z    INFO    scaleexecutor    Scaling Jobs    {"scaledJob.Name": "external-scaler-sj-test-sj", "scaledJob.Namespace": "external-scaler-sj-test-ns", "Number of pending Jobs": 0}
2024-07-25T06:27:52Z    INFO    scaleexecutor    Creating jobs    {"scaledJob.Name": "external-scaler-sj-test-sj", "scaledJob.Namespace": "external-scaler-sj-test-ns", "Effective number of max jobs": 0}
2024-07-25T06:27:52Z    INFO    scaleexecutor    Creating jobs    {"scaledJob.Name": "external-scaler-sj-test-sj", "scaledJob.Namespace": "external-scaler-sj-test-ns", "Number of jobs": 0}
2024-07-25T06:27:52Z    INFO    scaleexecutor    Created jobs    {"scaledJob.Name": "external-scaler-sj-test-sj", "scaledJob.Namespace": "external-scaler-sj-test-ns", "Number of jobs": 0}
2024-07-25T06:27:57Z    INFO    scaleexecutor    Scaling Jobs    {"scaledJob.Name": "external-scaler-sj-test-sj", "scaledJob.Namespace": "external-scaler-sj-test-ns", "Number of running Jobs": 2}
2024-07-25T06:27:57Z    INFO    scaleexecutor    Scaling Jobs    {"scaledJob.Name": "external-scaler-sj-test-sj", "scaledJob.Namespace": "external-scaler-sj-test-ns", "Number of pending Jobs": 0}
2024-07-25T06:27:57Z    INFO    scaleexecutor    Creating jobs    {"scaledJob.Name": "external-scaler-sj-test-sj", "scaledJob.Namespace": "external-scaler-sj-test-ns", "Effective number of max jobs": 0}
2024-07-25T06:27:57Z    INFO    scaleexecutor    Creating jobs    {"scaledJob.Name": "external-scaler-sj-test-sj", "scaledJob.Namespace": "external-scaler-sj-test-ns", "Number of jobs": 0}
2024-07-25T06:27:57Z    INFO    scaleexecutor    Created jobs    {"scaledJob.Name": "external-scaler-sj-test-sj", "scaledJob.Namespace": "external-scaler-sj-test-ns", "Number of jobs": 0}

KEDA Version

2.14.0

Kubernetes Version

None

Platform

None

Scaler Details

No response

Anything else?

No response

@SpiritZhou SpiritZhou added the bug Something isn't working label Jul 25, 2024
@SpiritZhou SpiritZhou closed this as not planned Won't fix, can't repro, duplicate, stale Aug 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant