Add option to the Datadog scaler to use the Cluster Agent as proxy #5496

arapulido · 2024-02-09T14:44:40Z

Add option to the Datadog scaler to use the Datadog Cluster Agent as proxy to obtain the metrics, instead of calling the REST API directly.

This avoids duplication and also improves the rate limiting issue that the Datadog scaler currently has, as the Cluster Agent gets the metric values in batches.

Implements: #5355

Checklist

Tests have been added
Changelog has been updated and is aligned with our changelog requirements
A PR is opened to update the documentation on (repo) (if applicable)
Commits are signed with Developer Certificate of Origin (DCO - learn more)

Fixes #5355

Docs: kedacore/keda-docs#1310

pkg/scalers/datadog_scaler.go

stale · 2024-04-09T23:32:32Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

zroubalik · 2024-04-10T15:41:57Z

/run-e2e datadog
Update: You can check the progress here

stale · 2024-06-09T22:57:26Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

zroubalik

@arapulido any update on this, please?

arapulido · 2024-06-11T13:04:09Z

@zroubalik I will now continue this work. I need to rebase and I will ask for reviews soon

arapulido · 2024-06-19T09:10:57Z

@zroubalik this is ready to be reviewed. Thanks!

zroubalik · 2024-06-25T21:47:35Z

/run-e2e datadog
Update: You can check the progress here

arapulido · 2024-07-01T11:58:29Z

@zroubalik I have made some commits that fix the tests. They are currently working correctly in my local environment. Can you kick off running them in CI, please? Thanks!

arapulido · 2024-07-10T07:23:16Z

@zroubalik @JorTurFer hello! can you kick off the E2E tests for this PR, please? This one should be ready for review. Thanks!

arapulido · 2024-07-19T10:03:48Z

@zroubalik @tomkerkhove @JorTurFer Can we please rerun the e2e tests here, please? I would love to include this feature in the next KEDA release

JorTurFer · 2024-07-24T09:45:19Z

/run-e2e datadog
Update: You can check the progress here

JorTurFer

Nice job! 💪
Some nits inline :)

pkg/scalers/datadog_scaler.go

Signed-off-by: Ara Pulido <[email protected]>

…ice IP Signed-off-by: Ara Pulido <[email protected]>

Signed-off-by: Ara Pulido <[email protected]>

JorTurFer · 2024-07-24T12:56:08Z

/run-e2e datadog
Update: You can check the progress here

JorTurFer

LGTM! PTAL @zroubalik

zroubalik

LGTM, great job @arapulido !

zroubalik

@arapulido @JorTurFer is there anything we need to update in the docs? kedacore/keda-docs#1310

arapulido · 2024-07-25T12:20:55Z

@arapulido @JorTurFer is there anything we need to update in the docs? kedacore/keda-docs#1310

Nothing, that one is good to go as well. Thanks!

JorTurFer · 2024-07-29T09:08:14Z

/run-e2e datadog
Update: You can check the progress here

…edacore#5496) * Add new Datadog External scaler to talk to the DCA Signed-off-by: Ara Pulido <[email protected]> * Add ability to retrieve a metric value from the DCA Signed-off-by: Ara Pulido <[email protected]> * Use datadogmetric naming convention Signed-off-by: Ara Pulido <[email protected]> * Merge both Datadog scalers into one Signed-off-by: Ara Pulido <[email protected]> * Add authMode to the TriggerAuthentication Signed-off-by: Ara Pulido <[email protected]> * Add unit tests for Datadog scaler with Cluster Agent proxy Signed-off-by: Ara Pulido <[email protected]> * Fix activation Signed-off-by: Ara Pulido <[email protected]> * Add E2E tests for the Datadog scaler using the Cluster Agent proxy Signed-off-by: Ara Pulido <[email protected]> * Fixes after rebase Signed-off-by: Ara Pulido <[email protected]> * Rearrange Datadog tests Signed-off-by: Ara Pulido <[email protected]> * Fix linting errors Signed-off-by: Ara Pulido <[email protected]> * Keep token only auth Signed-off-by: Ara Pulido <[email protected]> * Remove references to ca metadata Signed-off-by: Ara Pulido <[email protected]> * remove trailing space Signed-off-by: Ara Pulido <[email protected]> * Add changelog entry for cluster agent as proxy feature Signed-off-by: Ara Pulido <[email protected]> * Fix activation parameter for Datadog API Signed-off-by: Ara Pulido <[email protected]> * Adjust Datadog test values Signed-off-by: Ara Pulido <[email protected]> * Fix typo in Datadog test Signed-off-by: Ara Pulido <[email protected]> * Make cluster agent metric server parameter mandatory Signed-off-by: Ara Pulido <[email protected]> * Use only service name and namespace to resolve the Cluster Agent service IP Signed-off-by: Ara Pulido <[email protected]> * Fix linting issues Signed-off-by: Ara Pulido <[email protected]> --------- Signed-off-by: Ara Pulido <[email protected]> Co-authored-by: Jorge Turrado Ferrero <[email protected]> Signed-off-by: Jorge Turrado <[email protected]>

arapulido changed the title ~~Cluster agent proxy~~ Add option to the Datadog scaler to use the Cluster Agent as proxy Feb 9, 2024

github-advanced-security bot found potential problems Feb 9, 2024

View reviewed changes

pkg/scalers/datadog_scaler.go Fixed Show fixed Hide fixed

arapulido force-pushed the cluster_agent_proxy branch from 76f8249 to 76a58a9 Compare February 9, 2024 16:19

stale bot added the stale All issues that are marked as stale due to inactivity label Apr 9, 2024

stale bot removed the stale All issues that are marked as stale due to inactivity label Apr 10, 2024

stale bot added the stale All issues that are marked as stale due to inactivity label Jun 9, 2024

zroubalik reviewed Jun 10, 2024

View reviewed changes

stale bot removed the stale All issues that are marked as stale due to inactivity label Jun 10, 2024

arapulido force-pushed the cluster_agent_proxy branch from 76a58a9 to d3984a6 Compare June 11, 2024 14:13

arapulido marked this pull request as ready for review June 19, 2024 07:54

arapulido requested a review from a team as a code owner June 19, 2024 07:54

arapulido force-pushed the cluster_agent_proxy branch from 20e7540 to 66a1b57 Compare June 19, 2024 07:56

JorTurFer reviewed Jul 24, 2024

View reviewed changes

pkg/scalers/datadog_scaler.go Show resolved Hide resolved

pkg/scalers/datadog_scaler.go Outdated Show resolved Hide resolved

arapulido added 6 commits July 24, 2024 14:50

Add new Datadog External scaler to talk to the DCA

cc1dd27

Signed-off-by: Ara Pulido <[email protected]>

Add ability to retrieve a metric value from the DCA

9b6d897

Signed-off-by: Ara Pulido <[email protected]>

Use datadogmetric naming convention

aa8d501

Signed-off-by: Ara Pulido <[email protected]>

Merge both Datadog scalers into one

842a85f

Signed-off-by: Ara Pulido <[email protected]>

Add authMode to the TriggerAuthentication

f9522cf

Signed-off-by: Ara Pulido <[email protected]>

Add unit tests for Datadog scaler with Cluster Agent proxy

6a5801e

Signed-off-by: Ara Pulido <[email protected]>

arapulido added 15 commits July 24, 2024 14:50

Fix activation

cb5fd0a

Signed-off-by: Ara Pulido <[email protected]>

Add E2E tests for the Datadog scaler using the Cluster Agent proxy

1879b03

Signed-off-by: Ara Pulido <[email protected]>

Fixes after rebase

d949fce

Signed-off-by: Ara Pulido <[email protected]>

Rearrange Datadog tests

69a7121

Signed-off-by: Ara Pulido <[email protected]>

Fix linting errors

03186a2

Signed-off-by: Ara Pulido <[email protected]>

Keep token only auth

2d0c98b

Signed-off-by: Ara Pulido <[email protected]>

Remove references to ca metadata

8eda0e7

Signed-off-by: Ara Pulido <[email protected]>

remove trailing space

785d389

Signed-off-by: Ara Pulido <[email protected]>

Add changelog entry for cluster agent as proxy feature

3ad042c

Signed-off-by: Ara Pulido <[email protected]>

Fix activation parameter for Datadog API

4fd53c3

Signed-off-by: Ara Pulido <[email protected]>

Adjust Datadog test values

639853a

Signed-off-by: Ara Pulido <[email protected]>

Fix typo in Datadog test

ade33d0

Signed-off-by: Ara Pulido <[email protected]>

Make cluster agent metric server parameter mandatory

6f66434

Signed-off-by: Ara Pulido <[email protected]>

Use only service name and namespace to resolve the Cluster Agent serv…

a52057b

…ice IP Signed-off-by: Ara Pulido <[email protected]>

Fix linting issues

a9250ea

Signed-off-by: Ara Pulido <[email protected]>

arapulido force-pushed the cluster_agent_proxy branch from acd1bdc to a9250ea Compare July 24, 2024 12:51

JorTurFer approved these changes Jul 24, 2024

View reviewed changes

zroubalik approved these changes Jul 25, 2024

View reviewed changes

zroubalik reviewed Jul 25, 2024

View reviewed changes

Merge branch 'main' into cluster_agent_proxy

178abb8

JorTurFer merged commit 00baf28 into kedacore:main Jul 29, 2024
18 checks passed

arapulido deleted the cluster_agent_proxy branch July 29, 2024 09:50

arapulido restored the cluster_agent_proxy branch July 29, 2024 09:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add option to the Datadog scaler to use the Cluster Agent as proxy #5496

Add option to the Datadog scaler to use the Cluster Agent as proxy #5496

arapulido commented Feb 9, 2024 •

edited by JorTurFer

Loading

stale bot commented Apr 9, 2024

zroubalik commented Apr 10, 2024 •

edited by github-actions bot

Loading

stale bot commented Jun 9, 2024

zroubalik left a comment

arapulido commented Jun 11, 2024

arapulido commented Jun 19, 2024

zroubalik commented Jun 25, 2024 •

edited by github-actions bot

Loading

arapulido commented Jul 1, 2024

arapulido commented Jul 10, 2024

arapulido commented Jul 19, 2024

JorTurFer commented Jul 24, 2024 •

edited by github-actions bot

Loading

JorTurFer left a comment

JorTurFer commented Jul 24, 2024 •

edited by github-actions bot

Loading

JorTurFer left a comment

zroubalik left a comment

zroubalik left a comment

arapulido commented Jul 25, 2024

JorTurFer commented Jul 29, 2024 •

edited by github-actions bot

Loading

Add option to the Datadog scaler to use the Cluster Agent as proxy #5496

Add option to the Datadog scaler to use the Cluster Agent as proxy #5496

Conversation

arapulido commented Feb 9, 2024 • edited by JorTurFer Loading

Checklist

stale bot commented Apr 9, 2024

zroubalik commented Apr 10, 2024 • edited by github-actions bot Loading

stale bot commented Jun 9, 2024

zroubalik left a comment

Choose a reason for hiding this comment

arapulido commented Jun 11, 2024

arapulido commented Jun 19, 2024

zroubalik commented Jun 25, 2024 • edited by github-actions bot Loading

arapulido commented Jul 1, 2024

arapulido commented Jul 10, 2024

arapulido commented Jul 19, 2024

JorTurFer commented Jul 24, 2024 • edited by github-actions bot Loading

JorTurFer left a comment

Choose a reason for hiding this comment

JorTurFer commented Jul 24, 2024 • edited by github-actions bot Loading

JorTurFer left a comment

Choose a reason for hiding this comment

zroubalik left a comment

Choose a reason for hiding this comment

zroubalik left a comment

Choose a reason for hiding this comment

arapulido commented Jul 25, 2024

JorTurFer commented Jul 29, 2024 • edited by github-actions bot Loading

arapulido commented Feb 9, 2024 •

edited by JorTurFer

Loading

zroubalik commented Apr 10, 2024 •

edited by github-actions bot

Loading

zroubalik commented Jun 25, 2024 •

edited by github-actions bot

Loading

JorTurFer commented Jul 24, 2024 •

edited by github-actions bot

Loading

JorTurFer commented Jul 24, 2024 •

edited by github-actions bot

Loading

JorTurFer commented Jul 29, 2024 •

edited by github-actions bot

Loading