-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Keda Operator - invalid memory address or nil pointer dereference during large scaler failures #6176
Comments
today is a community bi-weekly call, I will add it to the agenda, you are welcome to attend as well :) |
I encountered the similar issue in 2.13.1
|
this was briefly discussed last week, possibly an issue with trigger cache invalidation. It may take some time before it gets fixed. |
This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions. |
This issue has been automatically closed due to inactivity. |
Report
KEDA Version: 2.15.1
Running on Kubernetes 1.29+ {"version": "v1.29.7-eks-a18cd3a"}
I am running into a transient issue where i have multiple scaledObjects (464) in many namespaces. We use
metricsAPI
scaler to poll against an api pod in each namespace that dynamically scales off an endpoint.This works extremely well and at scale. However, what we noticed is if there is an issue where multiple scaledObjects are failing at once (such as a networking or control plane issue), the Operator will run into a null pointer reference when it runs into many variations of:
Can replace
namespace
with any of the 150 namespaces etc. This results in the Keda Operator restarting and prior to a restart the following nil pointer reference logs are shown:Expected Behavior
I'd expect the Operator to not run into null pointers when Scalers fail at a large amount or run into transient issues.
Actual Behavior
Operator cannot get a scaling decision, spread across 500 scaledObjects and runs into nill pointer panic and restarts
Steps to Reproduce the Problem
Logs from KEDA operator
KEDA Version
2.15.1
Kubernetes Version
1.29
Platform
Amazon Web Services
Scaler Details
MetricsAPI
Anything else?
No response
The text was updated successfully, but these errors were encountered: