Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue with Prometheus ScaledObject - ERROR failed to ensure HPA is correctly created for ScaledObject #5857

Closed
Shyamsundar45 opened this issue Jun 4, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@Shyamsundar45
Copy link

Report

I have a scaledobject with Prometheus trigger, I face below error and the error is similar to the issue mentioned here

2024-06-04T06:29:17Z	ERROR	Reconciler error	{"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "ScaledObject": {"name":"prom-scaledobject","namespace":"default"}, "namespace": "default", "name": "prom-scaledobject", "reconcileID": "b3c9e90e-cf16-484b-b275-04f67ebfcaff", "error": "HorizontalPodAutoscaler.autoscaling \"keda-hpa-prom-scaledobject\" is invalid: spec.metrics[0].external.target.averageValue: Invalid value: resource.Quantity{i:resource.int64Amount{value:0, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:\"0\", Format:\"DecimalSI\"}: must be positive"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
	/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:329
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:266
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
	/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227

Below is my manifest for keda scaledobject

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: prom-scaledobject
  namespace: default
spec:
  # This is the interval to check each trigger on. By default KEDA will check each trigger source on every ScaledObject every 30 seconds.
  pollingInterval: 30  # Optional. Default: 30 seconds
  cooldownPeriod: 120  # Optional. Default: 300 seconds

  minReplicaCount: 1     # Optional. Default: 0
  maxReplicaCount: 100
  idleReplicaCount: 0   # cannot be changed

  fallback:                                          # Optional. Section to specify fallback options
    failureThreshold: 3                              # Mandatory if fallback section is included
    replicas: 1

  advanced:
    restoreToOriginalReplicaCount: true

  scaleTargetRef:
    name: "{{ .Values.app }}-deployment"
    envSourceContainerName: "{{ .Values.app }}" # Optional. container-name

  triggers:
    - type: prometheus
      metadata:
        serverAddress: https://promurl/.test
        threshold: '1'
        activationThreshold: '5'
        query: sum(avg_over_time(rq_jobs{queue="default",namespace=“default”,status="queued”}[10m]))
        authModes: "basic"
      authenticationRef:
        name: keda-prom-creds

Can someone pls confirm if the scaled object type Prometheus also needs a fix , or Am I doing something wrong in setting the values?

NOTE: All the variables from values file are also passed correctly.

Expected Behavior

The scaledobject should be able to create the HPA for prometheus trigger since all the values are already passed.

Actual Behavior

The HPA is not created so the scaledobject is not working as expected.

Steps to Reproduce the Problem

  1. Create yaml manifest file for the ScaledObeject with trigger type Prometheus
  2. Set the values as mentioned in the description
  3. Run the manifest file to create the scaledobject

Logs from KEDA operator

2024-06-04T06:29:17Z	ERROR	Reconciler error	{"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "ScaledObject": {"name":"prom-scaledobject","namespace":"default"}, "namespace": "default", "name": "prom-scaledobject", "reconcileID": "b3c9e90e-cf16-484b-b275-04f67ebfcaff", "error": "HorizontalPodAutoscaler.autoscaling \"keda-hpa-prom-scaledobject\" is invalid: spec.metrics[0].external.target.averageValue: Invalid value: resource.Quantity{i:resource.int64Amount{value:0, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:\"0\", Format:\"DecimalSI\"}: must be positive"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
	/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:329
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:266
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
	/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227

KEDA Version

2.13.1

Kubernetes Version

1.27

Platform

Google Cloud

Scaler Details

Prometheus

Anything else?

No response

@Shyamsundar45 Shyamsundar45 added the bug Something isn't working label Jun 4, 2024
@Shyamsundar45
Copy link
Author

I apologize for directly opening a bug, I missed the discussion part. I am closing this as I have created a discussion here
#5858

@Shyamsundar45 Shyamsundar45 closed this as not planned Won't fix, can't repro, duplicate, stale Jun 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant