It's not working in China Azure

[heavenwing]

Expected Behavior

Should scale pod when queue have messages

Actual Behavior

Don't scale pod.

Steps to Reproduce the Problem

1. Create a aks cluster in China Azure Cloud
2. Follow steps in https://github.com/kedacore/sample-hello-world-azure-functions

Specifications

• KEDA Version: 1.4.1
• Platform & Version: China Azure Cloud
• Kubernetes Version: 1.16.7
• Scaler(s): azure-queue

BTW

If I use Global Azure Cloud's AKS, it's ok.
Global AKS k8s version is 1.15.11
I am not sure that root cause is Cloud region or K8S version.

[tomkerkhove]

Can you provide logs from KEDA please?

[heavenwing]

keda-operator's log is here
keda-keda-operator-5895ff46b9-fjp8b-1592271782284779400.log

It don't parse connectionstring of China Azure Cloud azure storage correctly.

And in this process , I found another issue: #880

[vinothav]

Hi @tomkerkhove:

I face the same issue in AWS EKS.

Environment: AWS EKS
Region: Ireland
EKS Version: 1.19

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: xxxx
namespace: xxx
spec:
scaleTargetRef:
name: xxxx
pollingInterval: 30
cooldownPeriod: 300
minReplicaCount: 1
maxReplicaCount: 15
triggers:
- type: rabbitmq
metadata:
host: amqp://xxx:xxx@rabbitmq:5672//
protocol: amqp
mode: QueueLength
value: "50"
queueName: events.save_event

my virtual host is : "/"

rabbitmq in the host parameter is the service name of rabbitmq.
Note: Rabbitmq is running as stateful set in kubernetes

The connection string is not parsed and I get the below message from the KEDA controller:

2021-05-10T13:18:22.323Z ERROR controller Reconciler error {"reconcilerGroup": "keda.sh", "reconcilerKind": "ScaledObject", "controller": "scaledobject", "name": "sentry-save-event", "namespace": "sentry", "error": "error getting scaler for trigger #0: error establishing rabbitmq connection: dial tcp: lookup rabbitmq on 10.100.0.10:53: no such host"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:218
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker
/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:197
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155
k8s.io/apimachinery/pkg/util/wait.BackoffUntil
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156
k8s.io/apimachinery/pkg/util/wait.JitterUntil
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133
k8s.io/apimachinery/pkg/util/wait.Until
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:90

[vinothav]

@heavenwing: Did you manage to fix this issue with a workaround?

[tomkerkhove]

What version of KEDA are you running? Also v1.4.1?

[vinothav]

Hi @tomkerkhove

I am deploying the operator and metrics-api-server using helm chart version 2.2.1 and the application version is 2.2.0
Image: ghcr.io/kedacore/keda:2.2.0

[tomkerkhove]

I think this is a seperate issue where it's not able to connect to your Rabbit MQ instance, no?

[vinothav]

Yes it does not connect to my rabbitmq instance, and I also get similar output
error establishing rabbitmq connection: dial tcp: lookup rabbitmq on 10.100.0.10:53: no such host

But I am not sure if its a different issue, if so can you guide in that direction?

[jeje-ch]

Hello,

having the same issue, I was able to fix/workaround it by adding the namespace to the service name (with dot in the middle)... I mean:

• rabbitmq service name = my_rabbit
• namespace = my_namespace
• then, connection string => amqp://guest:guest@my_rabbit.my_namespace:5672/

Hope it will help ;-)

[tomkerkhove]

That's a good tip from @jeje-ch, is it running in the same Kubernetes namespace?

Sorry I lost track of this one.

[tomkerkhove]

@vinothav Can you open a discussion please? The real issue report here is related to Azure China and queues, not Kafka and we seem to be getting away from that report so want to separate it.

[jeje-ch]

@tomkerkhove keda's yaml applied in keda namespace, and my scaledJob and rabbitmq in my_namespace; so yes, scaledJob and rabbit in the same namespace.

I didn't try if it works if they are on different namespaces, but with the appended ns I supposed it will => but I will let anyone being in such config to try and let us know :-)

[tomkerkhove]

Yes, it has tobe configured from the KEDA namespace so namespace is required.

[jeje-ch]

I forgot to mention that I'm neither on Azure China nor AWS EKS, but Jelastic... but pretty sure it doesn't matter the cloud for that issue

[amirschw]

The original issue was fixed in KEDA v2.4. Adding cloud: AzureChinaCloud to the ScaledObject's metadata as described in the docs should work.

[tomkerkhove]

Thanks for chiming in @amirschw - i lost track of this issue.

Can you please give this a go @heavenwing ?

[heavenwing]

Thanks @amirschw . I will close this issue.