Rabbitmq scaler be used with regexes with amqps protocol

[matan-legit]

Proposal

similar to 743 i want to be able to use regex on queue names, this was implemented only on http protocol. As Keda doesn't support https in this scaler i can't use this protocol. (this might be another good feature request)

Use-Case

use regex in queue name with amqps protocol

Is this a feature you are interested in implementing yourself?

No

Anything else?

No response

[JorTurFer]

Hi,
Sadly, regex feature can't be ported to AMQP because AMQP doesn't support it. From management pov, amqp is poorer than http.

As Keda doesn't support https in this scaler

What do you mean? You can just set a URL with HTTPS and it will work if the certificate is valid. In case of self-signed certificates, you still can add them to KEDA's trusted store: https://keda.sh/docs/2.11/operate/security/#register-your-own-ca-in-keda-operator-trusted-store

[matan-legit]

@JorTurFer when i put a url with https it didn't work for me but i might've done something wrong
also i saw this pr closed: #1073

[JorTurFer]

Could you share the https ScaledObject that you are using?
I'm sure that HTTPS has to work because in my previous company they used a hosted rabbitmq service, exposed by HTTPS.
It could be also because the RabbitMQ server doesn't have the management plugin enabled: https://www.rabbitmq.com/management.html
Could you check it too?

[matan-legit]

@JorTurFer thanks for the quick response!

im using a clustertrigerauthentication to authenticate
scaledobject:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  annotations:
    meta.helm.sh/release-name: code-scanners
    meta.helm.sh/release-namespace: default
  creationTimestamp: "2023-07-31T15:48:02Z"
  finalizers:
  - finalizer.keda.sh
  generation: 3
  labels:
    app.kubernetes.io/managed-by: Helm
    scaledobject.keda.sh/name: code-scanners-scaledobject
  name: code-scanners-scaledobject
  namespace: default
  resourceVersion: "184205450"
  uid: e6195b5c-4710-4c0a-9a77-4fa3eddd3938
spec:
  cooldownPeriod: 300
  maxReplicaCount: 4
  minReplicaCount: 2
  pollingInterval: 30
  scaleTargetRef:
    name: code-scanners
  triggers:
  - authenticationRef:
      kind: ClusterTriggerAuthentication
      name: rabbitmq-url
    metadata:
      mode: QueueLength
      queueName: testqueue
      value: "10"
    type: rabbitmq
status:
  conditions:
  - message: ScaledObject is defined correctly and is ready for scaling
    reason: ScaledObjectReady
    status: "True"
    type: Ready
  - message: Scaling is not performed because triggers are not active
    reason: ScalerNotActive
    status: "False"
    type: Active
  - message: No fallbacks are active on this scaled object
    reason: NoFallbackFound
    status: "False"
    type: Fallback
  - status: Unknown
    type: Paused
  externalMetricNames:
  - ***
  hpaName: keda-hpa-code-scanners-scaledobject
  originalReplicaCount: 2
  scaleTargetGVKR:
    group: apps
    kind: Deployment
    resource: deployments
    version: v1
  scaleTargetKind: apps/v1.Deployment

CTA:

apiVersion: keda.sh/v1alpha1
kind: ClusterTriggerAuthentication
metadata:
  annotations:
    meta.helm.sh/release-name: keda
    meta.helm.sh/release-namespace: keda
  creationTimestamp: "2023-07-30T14:19:24Z"
  finalizers:
  - finalizer.keda.sh
  generation: 2
  labels:
    app.kubernetes.io/managed-by: Helm
  name: rabbitmq-url
  resourceVersion: "182731986"
  uid: 701c462c-4511-4017-a6d9-4a6e73f3f5d8
spec:
  secretTargetRef:
  - key: rabbitmq-url
    name: rabbitmq-connection
    parameter: host

url structure: https://user:password@b-87ds8765-abcd-4123-sdb9-d187172cf294.mq.us-east-1.amazonaws.com:15671
port is 15671 as per this doc: https://docs.aws.amazon.com/amazon-mq/latest/developer-guide/rabbitmq-basic-elements-broker.html
but i also configured without a port with no success

another unrelated issue is, when i edit the secret for the connection url i need to redeploy the scaled object for the configuration to apply, it would be nice if it applied automatically.

regarding plugins we're using amazon elasticmq so im not sure if i can even change something in this regard.

[JorTurFer]

Just recapping:

• You have tried using https://user:password@b-87ds8765-abcd-4123-sdb9-d187172cf294.mq.us-east-1.amazonaws.com:15671/ (with the https and port)
• You have tried using https://user:password@b-87ds8765-abcd-4123-sdb9-d187172cf294.mq.us-east-1.amazonaws.com/ (with the https and without port)

Could you give another try an share KEDA error on those cases? In my personal experience, I have had issues with some AWS certificates, maybe it's related with that 🤔

[JorTurFer]

another unrelated issue is, when i edit the secret for the connection url i need to redeploy the scaled object for the configuration to apply, it would be nice if it applied automatically.

In theory, the ScaledObject is rebuilt on each error, so the change in secrets should be read once the scaler returns an error. In normal operation we don't watch the secrets because watching all the configuration sources could be a problem to manage them, but it's a good idea. If you open an issue requesting it, we can discuss about the topic 😄

[matan-legit]

│ 2023-08-01T09:03:32Z    ERROR    scale_handler    error getting scale decision    {"scaledObject.Namespace": "default", "scaledObject.Name": "code-scanners-scaledobject", "scaler": "rabbi │
│ tMQScaler", "error": "error inspecting rabbitMQ: Get \"https://user:password@b-87ds8765-abcd-4123-sdb9-d187172cf294.mq.us-east-1.amazonaws.com:15671/api/queues/%!F(MISSING)/Legit.Common.Q │
│ ueue.QueueMessage%5B%!B(MISSING)asopdkaposd.Shared.djoaisjdo.asidjiojio%!C(MISSING)+CodeScanners.Shared%!D(MISSING)%!D(MISSING)%!C(MISSING)+Legit.Common_OfflineConsumers\": │
│  context deadline exceeded (Client.Timeout exceeded while awaiting headers)"}                                                                                                               │
│ github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).getScaledObjectState                                                                                                                │
│     /workspace/pkg/scaling/scale_handler.go:588                                                                                                                                             │
│ github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).checkScalers                                                                                                                        │
│     /workspace/pkg/scaling/scale_handler.go:236                                                                                                                                             │
│ github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).startScaleLoop                                                                                                                      │
│     /workspace/pkg/scaling/scale_handler.go:175                                                                                                                                             │
│                                                                                                                                                                                             │

[matan-legit]

actually when im doing no port it sends: error inspecting rabbitMQ: error requesting rabbitMQ API status: 404 Not Found

[matan-legit]

ok no port config actually worked for me 🥳 i had misconfiguration on the queuename side, looks like regex is also working.
thanks for all the help @JorTurFer :) closing the issue

[JorTurFer]

Nice! Happy to help :)