AWS SDK V2 Upgrade #4939

jeevanragula · 2023-09-02T17:51:16Z

In the Keda project, we are currently using below AWS Go SDK version.

github.com/aws/aws-sdk-go v1.44.287

But AWS released a V2 version long back which has few advantages over V1.
This PR is to update the code to use AWS SDK V2.

Checklist

When introducing a new scaler, I agree with the scaling governance policy
I have verified that my change is according to the deprecations & breaking changes policy
Tests have been added
Changelog has been updated and is aligned with our changelog requirements
A PR is opened to update our Helm chart (repo) (if applicable, ie. when deployment manifests are modified)
A PR is opened to update the documentation on (repo) (if applicable)
Commits are signed with Developer Certificate of Origin (DCO - learn more)

Fixes #4905

Relates to #4905

Signed-off-by: Jeevan Reddy Ragula <[email protected]>

github-actions · 2023-09-02T17:51:28Z

Thank you for your contribution! 🙏 We will review your PR as soon as possible.

🏖️ Over the summer, the response time will be longer than usual due to maintainers taking time off so please bear with us.

While you are waiting, make sure to:

Add an entry in our changelog in alphabetical order and link related issue
Update the documentation, if needed
Add unit & e2e tests for your changes
GitHub checks are passing
Is the DCO check failing? Here is how you can fix DCO issues

Learn more about:

Our contribution guide

semgrep-app · 2023-09-02T17:52:11Z

Semgrep found 6 context-todo findings:

pkg/scalers/aws_sqs_queue_scaler.go: L232
pkg/scalers/aws_kinesis_stream_scaler.go: L177
pkg/scalers/aws_dynamodb_streams_scaler.go: L203
pkg/scalers/aws_dynamodb_scaler.go: L232
pkg/scalers/aws_common.go: L34
pkg/scalers/aws_cloudwatch_scaler.go: L370

Consider to use well-defined context

_{Ignore this finding from context-todo.}

Signed-off-by: Jeevan Reddy Ragula <[email protected]>

JorTurFer · 2023-09-02T18:32:57Z

/run-e2e aws
Update: You can check the progress here

Signed-off-by: Jeevan Reddy Ragula <[email protected]>

CHANGELOG.md

pkg/metricsservice/api/metrics.pb.go

pkg/metricsservice/api/metrics_grpc.pb.go

pkg/scalers/aws_cloudwatch_scaler.go

pkg/scalers/aws_common.go

pkg/scalers/aws_kinesis_stream_scaler.go

pkg/scalers/aws_sqs_queue_scaler.go

pkg/scalers/externalscaler/externalscaler.pb.go

tests/scalers/aws/aws_cloudwatch/aws_cloudwatch_test.go

Signed-off-by: Jeevan Reddy Ragula <[email protected]>

JorTurFer · 2023-09-03T18:55:07Z

pkg/scalers/aws_dynamodb_scaler.go

+	return attributeValues, nil
+}
+
+func attributeValueFromInterface(value interface{}) (types.AttributeValue, error) {


Does the SDK have something like this internally?

@JorTurFer Found this package. But few test cases are failing when using that.

"github.com/aws/aws-sdk-go-v2/feature/dynamodb/attributevalue"

Do they fail because the behavior has changed?

TestParseDynamoMetadata test case was failing.
May be I am not using this "attributevalue" package correctly. But with my code, all the test cases are working fine.
If any one suggest easy way to replace this method, happy to do that.

JorTurFer · 2023-09-03T18:58:38Z

/run-e2e aws
Update: You can check the progress here

Signed-off-by: Jeevan Reddy Ragula <[email protected]>

zroubalik · 2023-09-05T14:27:09Z

/run-e2e aws
Update: You can check the progress here

JorTurFer · 2023-09-05T19:45:39Z

/run-e2e aws
Update: You can check the progress here

tests/scalers/aws/aws_dynamodb_streams_pod_identity/aws_dynamodb_streams_pod_identity_test.go

…db_streams_pod_identity_test.go Co-authored-by: Jorge Turrado Ferrero <[email protected]> Signed-off-by: Jeevan Reddy Ragula <[email protected]>

Signed-off-by: Jeevan Reddy Ragula <[email protected]>

semgrep-app · 2023-09-06T20:01:43Z

Semgrep found 6 context-todo findings:

pkg/scalers/aws_sqs_queue_scaler.go: L241
pkg/scalers/aws_kinesis_stream_scaler.go: L177
pkg/scalers/aws_dynamodb_streams_scaler.go: L203
pkg/scalers/aws_dynamodb_scaler.go: L232
pkg/scalers/aws_common.go: L35
pkg/scalers/aws_cloudwatch_scaler.go: L371

Consider to use well-defined context

_{Ignore this finding from context-todo.}

Signed-off-by: Jeevan Reddy Ragula <[email protected]>

…#4922) Co-authored-by: Jorge Turrado Ferrero <[email protected]>

…ring (kedacore#4584) * fix RabbitMQ Scaler, allow subpaths along with vhost in connection string (kedacore#4584) Signed-off-by: Roman Bielyi <[email protected]> * Fix CHANGELOG.md, remove redundant/wrong pieces that were made during rebase, this fix is aimed to resolve failed workflow's Static Check (validate-changelog) test Signed-off-by: Roman Bielyi <[email protected]> --------- Signed-off-by: Roman Bielyi <[email protected]>

Signed-off-by: Jorge Turrado <[email protected]> Signed-off-by: Jorge Turrado <[email protected]>

Signed-off-by: Jeevan Reddy Ragula <[email protected]>

jeevanragula · 2023-09-07T07:47:56Z

@JorTurFer I have cancelled it and raised a fresh review.
#4953

jeevanragula added 3 commits September 2, 2023 23:13

Updating AWS SDK to V2

6181021

Signed-off-by: Jeevan Reddy Ragula <[email protected]>

Updating AWS SDK to V2

1b316db

Signed-off-by: Jeevan Reddy Ragula <[email protected]>

Updating Changelog

bc5927e

Signed-off-by: Jeevan Reddy Ragula <[email protected]>

jeevanragula requested a review from a team as a code owner September 2, 2023 17:51

jeevanragula added 3 commits September 2, 2023 23:24

Merge branch 'main' into aws_version_update

a95aec5

Signed-off-by: Jeevan Reddy Ragula <[email protected]>

update go mod file and vendor modules text

01f25f3

Signed-off-by: Jeevan Reddy Ragula <[email protected]>

committing vendor folder

dab2106

Signed-off-by: Jeevan Reddy Ragula <[email protected]>

jeevanragula added 2 commits September 3, 2023 00:10

fixing pre-commit issues

c998166

Signed-off-by: Jeevan Reddy Ragula <[email protected]>

fixing pre-commit issues

4bca057

Signed-off-by: Jeevan Reddy Ragula <[email protected]>

JorTurFer reviewed Sep 2, 2023

View reviewed changes

jeevanragula added 2 commits September 3, 2023 01:37

passing context to aws methods

18c5059

Signed-off-by: Jeevan Reddy Ragula <[email protected]>

revert protoc changes

a9a8e39

Signed-off-by: Jeevan Reddy Ragula <[email protected]>

JorTurFer reviewed Sep 3, 2023

View reviewed changes

jeevanragula added 2 commits September 4, 2023 23:51

Merge branch 'main' into aws_version_update

f6bf9de

adding aws endpoint to client

ab94c06

Signed-off-by: Jeevan Reddy Ragula <[email protected]>

Merge branch 'main' into aws_version_update

179aa98

JorTurFer reviewed Sep 5, 2023

View reviewed changes

tests/scalers/aws/aws_dynamodb_streams_pod_identity/aws_dynamodb_streams_pod_identity_test.go Show resolved Hide resolved

jeevanragula and others added 4 commits September 7, 2023 00:38

Update tests/scalers/aws/aws_dynamodb_streams_pod_identity/aws_dynamo…

2fd0008

…db_streams_pod_identity_test.go Co-authored-by: Jorge Turrado Ferrero <[email protected]> Signed-off-by: Jeevan Reddy Ragula <[email protected]>

Updating AWS SDK to V2

96597bc

Signed-off-by: Jeevan Reddy Ragula <[email protected]>

fixing pre-commit issues

f9bd71b

Signed-off-by: Jeevan Reddy Ragula <[email protected]>

go mod update

59e5bf4

Signed-off-by: Jeevan Reddy Ragula <[email protected]>

jeevanragula added 2 commits September 7, 2023 01:36

Updating AWS SDK to V2

9ba65f0

Signed-off-by: Jeevan Reddy Ragula <[email protected]>

resolving conflicts and revert protoc version

4a21f1a

Signed-off-by: Jeevan Reddy Ragula <[email protected]>

Wolfe1 and others added 7 commits September 7, 2023 02:13

Selenium Grid | Fix latest browser version use case scaling (kedacore…

e0c8e0d

…#4922) Co-authored-by: Jorge Turrado Ferrero <[email protected]>

chore: bump deps (kedacore#4942)

782fa54

Signed-off-by: Jorge Turrado <[email protected]> Signed-off-by: Jorge Turrado <[email protected]>

updating AWS SDK to V2

9fbb58c

Signed-off-by: Jeevan Reddy Ragula <[email protected]>

resolving conflicts and revert protoc version

223068c

Signed-off-by: Jeevan Reddy Ragula <[email protected]>

resolve conflicts

6c2d055

Signed-off-by: Jeevan Reddy Ragula <[email protected]>

resolve conflicts

f2557d1

Signed-off-by: Jeevan Reddy Ragula <[email protected]>

jeevanragula closed this Sep 7, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

AWS SDK V2 Upgrade #4939

AWS SDK V2 Upgrade #4939

jeevanragula commented Sep 2, 2023 •

edited

Loading

github-actions bot commented Sep 2, 2023

semgrep-app bot commented Sep 2, 2023

JorTurFer commented Sep 2, 2023 •

edited by github-actions bot

Loading

JorTurFer Sep 3, 2023

jeevanragula Sep 4, 2023

JorTurFer Sep 5, 2023

jeevanragula Sep 6, 2023

JorTurFer commented Sep 3, 2023 •

edited by github-actions bot

Loading

zroubalik commented Sep 5, 2023 •

edited by github-actions bot

Loading

JorTurFer commented Sep 5, 2023 •

edited by github-actions bot

Loading

semgrep-app bot commented Sep 6, 2023

jeevanragula commented Sep 7, 2023

AWS SDK V2 Upgrade #4939

AWS SDK V2 Upgrade #4939

Conversation

jeevanragula commented Sep 2, 2023 • edited Loading

Checklist

github-actions bot commented Sep 2, 2023

semgrep-app bot commented Sep 2, 2023

JorTurFer commented Sep 2, 2023 • edited by github-actions bot Loading

JorTurFer Sep 3, 2023

Choose a reason for hiding this comment

jeevanragula Sep 4, 2023

Choose a reason for hiding this comment

JorTurFer Sep 5, 2023

Choose a reason for hiding this comment

jeevanragula Sep 6, 2023

Choose a reason for hiding this comment

JorTurFer commented Sep 3, 2023 • edited by github-actions bot Loading

zroubalik commented Sep 5, 2023 • edited by github-actions bot Loading

JorTurFer commented Sep 5, 2023 • edited by github-actions bot Loading

semgrep-app bot commented Sep 6, 2023

jeevanragula commented Sep 7, 2023

jeevanragula commented Sep 2, 2023 •

edited

Loading

JorTurFer commented Sep 2, 2023 •

edited by github-actions bot

Loading

JorTurFer commented Sep 3, 2023 •

edited by github-actions bot

Loading

zroubalik commented Sep 5, 2023 •

edited by github-actions bot

Loading

JorTurFer commented Sep 5, 2023 •

edited by github-actions bot

Loading