test: fix searchable migration test on code build #13046

sobolk · 2023-08-02T04:09:42Z

Description of changes

This PR:

Adds credential rotator inspired by Add credential refresh logic for long running tests amplify-category-api#1541
Refreshes account credentials in searchable migration test so that test running over 1hr always has access.

Out of scope:
This mechanism can be further extended to support child test account credential rotations if such need occurs.
However, today every other test manages to finish every attempt below 1hr threshold (scripts assume test role per retry, see

amplify-cli/.circleci/local_publish_helpers.sh

Line 233 in 9793be6

setAwsAccountCredentials

).

Issue #, if available

Description of how you validated changes

Please migrate your code to use AWS SDK for JavaScript (v3).
--
103 | For more information, check the migration guide at https://a.co/7PzMCcy
104 | (Use `node --trace-warnings ...` to show where the warning was created)
105 | console.log
106 | Test profile credentials refresh was scheduled
107 |  
108 | at tryScheduleCredentialRefresh (../amplify-e2e-core/src/utils/credentials-rotator.ts:56:11)
109 |  
110 | console.log
111 | CLI_REGION set to: us-east-2. Overwriting AWS_REGION and AWS_DEFAULT_REGION
112 |  
113 | at src/setup-tests.ts:24:13
114 |  
115 | (node:531) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023.
116 |  
117 | Please migrate your code to use AWS SDK for JavaScript (v3).
118 | For more information, check the migration guide at https://a.co/7PzMCcy
119 | (Use `node --trace-warnings ...` to show where the warning was created)
120 | console.log
121 | /tmp/amplify-e2e-tests/ad91ed979b5745deb278_859814
122 |  
123 | at createNewProjectDir (../amplify-e2e-core/src/index.ts:126:11)
124 |  
125 | console.log
126 | Test profile credentials refreshed
127 |  
128 | at refreshCredentials (../amplify-e2e-core/src/utils/credentials-rotator.ts:32:13)
129 |  
130 | console.log
131 | Test profile credentials refreshed
132 |  
133 | at refreshCredentials (../amplify-e2e-core/src/utils/credentials-rotator.ts:32:13)
134 |  
135 | console.log
136 | Test profile credentials refreshed
137 |  
138 | at refreshCredentials (../amplify-e2e-core/src/utils/credentials-rotator.ts:32:13)
139 |  
140 | console.log
141 | Skipping cloud deletion since we are in CI, and cleanup script will delete this stack in cleanup step.
142 |  
143 | at src/__tests__/transformer-migrations/searchable-migration.test.ts:43:15
144 |  
145 | PASS src/__tests__/transformer-migrations/searchable-migration.test.ts (3469.739 s)
146 | transformer model searchable migration test
147 | ✓ migration of searchable directive - search should return expected results (3442393 ms)

Checklist

PR description included
yarn test passes
Tests are changed or added
Relevant documentation is changed or added (and PR referenced)
New AWS SDK calls or CloudFormation actions have been added to relevant test and service IAM policies
Pull request labels are added

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

This reverts commit 8d620d9.

codecov-commenter · 2023-08-02T04:30:02Z

Codecov Report

Merging #13046 (87ae928) into dev (ec9a2ba) will increase coverage by 48.45%.
Report is 284 commits behind head on dev.
The diff coverage is 59.37%.

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

@@             Coverage Diff             @@
##              dev   #13046       +/-   ##
===========================================
+ Coverage    0.00%   48.45%   +48.45%     
===========================================
  Files        1296      842      -454     
  Lines      149743    38038   -111705     
  Branches     1296     7789     +6493     
===========================================
+ Hits            0    18432    +18432     
+ Misses     148447    18016   -130431     
- Partials     1296     1590      +294

Files Changed	Coverage Δ
...rmation/auth-stack-builder/auth-stack-transform.ts	`91.30% <0.00%> (+91.30%)`	⬆️
...tack-builder/auth-user-pool-group-stack-builder.ts	`51.72% <ø> (+51.72%)`	⬆️
...h-stack-builder/user-pool-group-stack-transform.ts	`94.59% <0.00%> (+94.59%)`	⬆️
.../src/provider-utils/awscloudformation/constants.ts	`100.00% <ø> (+100.00%)`	⬆️
...e-walkthrough-types/awsCognito-user-input-types.ts	`100.00% <ø> (+100.00%)`	⬆️
...vice-walkthrough-types/cognito-user-input-types.ts	`100.00% <ø> (+100.00%)`	⬆️
...y-category-function/src/commands/function/build.ts	`0.00% <0.00%> (ø)`
...ify-category-function/src/events/prePushHandler.ts	`33.33% <0.00%> (+33.33%)`	⬆️
...ider-utils/awscloudformation/utils/layerHelpers.ts	`21.80% <0.00%> (+21.80%)`	⬆️
...er-utils/awscloudformation/utils/storeResources.ts	`30.38% <ø> (+30.38%)`	⬆️
... and 58 more

... and 1243 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

edwardfoyle · 2023-08-02T15:37:34Z

packages/amplify-e2e-core/src/utils/credentials-rotator.ts

+    console.error('Test profile credentials request failed');
+    console.error(e);


do we intend to let execution continue here?

It's a background task. We don't await it anywhere in tests. But we want to know if it fails for some reason.

edwardfoyle · 2023-08-02T15:47:25Z

packages/amplify-e2e-core/src/utils/credentials-rotator.ts

+  }
+
+  setInterval(() => {
+    void refreshCredentials(process.env.TEST_ACCOUNT_ROLE);


should this callback set isRotationScheduled back to false?

No. setInterval calls refresh every 15 minutes. It's a background job.

I pushed an update, I hope the code is more self-explanatory now.

I'm dumb and was reading this as setTimeout not setInterval. makes sense now

edwardfoyle · 2023-08-02T15:47:53Z

packages/amplify-e2e-core/src/utils/credentials-rotator.ts

+let isRotationScheduled = false;
+
+export const tryScheduleCredentialRefresh = () => {
+  if (!process.env.IS_AMPLIFY_CI || !process.env.TEST_ACCOUNT_ROLE || isRotationScheduled) {


isn't this condition always false in CI?

No. Logs here

Please migrate your code to use AWS SDK for JavaScript (v3). -- 103 | For more information, check the migration guide at https://a.co/7PzMCcy 104 | (Use `node --trace-warnings ...` to show where the warning was created) 105 | console.log 106 | Test profile credentials refresh was scheduled 107 | 108 | at tryScheduleCredentialRefresh (../amplify-e2e-core/src/utils/credentials-rotator.ts:56:11) 109 | 110 | console.log 111 | CLI_REGION set to: us-east-2. Overwriting AWS_REGION and AWS_DEFAULT_REGION 112 | 113 | at src/setup-tests.ts:24:13 114 | 115 | (node:531) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 116 | 117 | Please migrate your code to use AWS SDK for JavaScript (v3). 118 | For more information, check the migration guide at https://a.co/7PzMCcy 119 | (Use `node --trace-warnings ...` to show where the warning was created) 120 | console.log 121 | /tmp/amplify-e2e-tests/ad91ed979b5745deb278_859814 122 | 123 | at createNewProjectDir (../amplify-e2e-core/src/index.ts:126:11) 124 | 125 | console.log 126 | Test profile credentials refreshed 127 | 128 | at refreshCredentials (../amplify-e2e-core/src/utils/credentials-rotator.ts:32:13) 129 | 130 | console.log 131 | Test profile credentials refreshed 132 | 133 | at refreshCredentials (../amplify-e2e-core/src/utils/credentials-rotator.ts:32:13) 134 | 135 | console.log 136 | Test profile credentials refreshed 137 | 138 | at refreshCredentials (../amplify-e2e-core/src/utils/credentials-rotator.ts:32:13) 139 | 140 | console.log 141 | Skipping cloud deletion since we are in CI, and cleanup script will delete this stack in cleanup step. 142 | 143 | at src/__tests__/transformer-migrations/searchable-migration.test.ts:43:15 144 | 145 | PASS src/__tests__/transformer-migrations/searchable-migration.test.ts (3469.739 s) 146 | transformer model searchable migration test 147 | ✓ migration of searchable directive - search should return expected results (3442393 ms)

https://us-east-1.console.aws.amazon.com/codesuite/codebuild/671107461633/projects/AmplifyCLI-E2E-Testing/build/AmplifyCLI-E2E-Testing%3A554fff6b-76b0-4ad8-8d99-b65ad2f37934?region=us-east-1

sobolk added 8 commits August 1, 2023 12:16

chore: increase token validity

8d620d9

Revert "chore: increase token validity"

c2515cc

This reverts commit 8d620d9.

chore: try credential rotator

59a3918

chore: try that

412f982

chore: ehh

a86d31b

chore: try this

b0241d8

chore: try this

d7f3eb9

chore: fortify logic a bit

a440bbc

sobolk requested a review from a team as a code owner August 2, 2023 04:09

chore: fix order

bbf690c

edwardfoyle reviewed Aug 2, 2023

View reviewed changes

sobolk added 2 commits August 2, 2023 09:02

chore: massaging code

fe4c2b8

chore: massaging code

87ae928

edwardfoyle approved these changes Aug 2, 2023

View reviewed changes

jhockett approved these changes Aug 2, 2023

View reviewed changes

sobolk merged commit 390bf8a into aws-amplify:dev Aug 2, 2023

sobolk deleted the fix-searchable-cb branch August 2, 2023 16:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test: fix searchable migration test on code build #13046

test: fix searchable migration test on code build #13046

sobolk commented Aug 2, 2023 •

edited

Loading

codecov-commenter commented Aug 2, 2023 •

edited

Loading

edwardfoyle Aug 2, 2023

sobolk Aug 2, 2023

edwardfoyle Aug 2, 2023

sobolk Aug 2, 2023

sobolk Aug 2, 2023

edwardfoyle Aug 2, 2023

edwardfoyle Aug 2, 2023

sobolk Aug 2, 2023

		console.error('Test profile credentials request failed');
		console.error(e);

test: fix searchable migration test on code build #13046

test: fix searchable migration test on code build #13046

Conversation

sobolk commented Aug 2, 2023 • edited Loading

Description of changes

Issue #, if available

Description of how you validated changes

Checklist

codecov-commenter commented Aug 2, 2023 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sobolk commented Aug 2, 2023 •

edited

Loading

codecov-commenter commented Aug 2, 2023 •

edited

Loading