Skip to content

Commit

Permalink
Document timeout when testing (#3178)
Browse files Browse the repository at this point in the history
* Document timeout when testing

* Fix broken links
  • Loading branch information
theunrepentantgeek authored Aug 3, 2023
1 parent f967954 commit 21171f2
Show file tree
Hide file tree
Showing 3 changed files with 32 additions and 11 deletions.
14 changes: 11 additions & 3 deletions docs/hugo/content/contributing/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,9 +39,16 @@ The task `controller:test-integration-envtest` runs the tests in a record/replay

To do this, delete the recordings for the failing tests (under `{test-dir}/recordings/{test-name}.yml`), and re-run `controller:test-integration-envtest`. If the test passes, a new recording will be saved, which you can commit to include with your change. All authentication and subscription information is removed from the recording.

To run the test and produce a new recording you will also need to have set the required authentication environment variables for an Azure Service Principal: `AZURE_SUBSCRIPTION_ID`, `AZURE_TENANT_ID`, `AZURE_CLIENT_ID`, and `AZURE_CLIENT_SECRET`. This Service Principal will need access to the subscription to create and delete resources.
To run the test and produce a new recording you will need to have set the required authentication environment variables for an Azure Service Principal: `AZURE_SUBSCRIPTION_ID`, `AZURE_TENANT_ID`, `AZURE_CLIENT_ID`, and `AZURE_CLIENT_SECRET`. This Service Principal will need access to the subscription to create and delete resources.

A few tests also need the `TEST_BILLING_ID` variable set to a valid Azure Billing ID when running in record mode. In replay mode this variable is never required. Note that the billing ID is redacted from all recording files so that the resulting file can be replayed by anybody, even somebody who does not know the Billing ID the test was recorded with.

Some Azure resources take longer to provision or delete than the default test timeout of 15m. To change the timeout, set `TIMEOUT` to a suitable value when running task. For example, to give your test a 60m timeout, use:

``` bash
TIMEOUT=60m task controller:test-integration-envtest
```

If you need to create a new Azure Service Principal, run the following commands:

```console
Expand All @@ -66,9 +73,10 @@ The output contains `appId` (`AZURE_CLIENT_ID`), `password` (`AZURE_CLIENT_SECRE
If you want to skip all recordings and run all tests directly against live Azure resources, you can use the `controller:test-integration-envtest-live` task. This will also require you to set the authentication environment variables, as detailed above.

### Running a single test
By default `task controller:test-integration-envtest` and its variants run all tests. This is often undesirable as you may just be working on a single feature or test. In order to run a subset of tests, use:
By default `task controller:test-integration-envtest` and its variants run all tests. This is often undesirable as you may just be working on a single feature or test. In order to run a subset of tests, use the `TEST_FILTER`:

```bash
TEST_FILTER=test_name_regex task controller:test-integration-envtest
TEST_FILTER=<test_name_regex> task controller:test-integration-envtest
```

## Running the operator locally
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -262,13 +262,26 @@ See [the code generator README](../#running-integration-tests) for how to run re
The samples are located in the [samples directory](https://github.com/Azure/azure-service-operator/blob/main/v2/samples). There should be at least one sample for each kind of supported resource. These currently need to be added manually. It's possible in the future we will automatically generate samples similar to how we automatically generate CRDs and types, but that doesn't happen today.

## Run test for added sample and commit the recording
The added new sample needs to be tested and recorded. To perform that, follow the steps below:
1. If a recording for the test already exists, delete it.
Look in the [recordings directory](https://github.com/Azure/azure-service-operator/blob/main/v2/internal/controllers/recordings/Test_Samples_CreationAndDeletion) for a file with the same name as your new test.
Typically these are named `Test_<GROUP>_<VERSION_PREFIX>_CreationAndDeletion.yaml`.
For example, If we're adding sample for NetworkSecurityGroup resource, check for `Test_Network_v1beta_CreationAndDeletion.yaml`
2. if recording exists, delete it. Else move to next step
3. run `TEST_FILTER=Test_Samples_CreationAndDeletion task controller:test-integration-envtest` to re-record the test.
The added new sample needs to be tested and recorded.

If a recording for the test already exists, delete it.
Look in the [recordings directory](https://github.com/Azure/azure-service-operator/blob/main/v2/internal/controllers/recordings/Test_Samples_CreationAndDeletion) for a file with the same name as your new test.
Typically these are named `Test_<GROUP>_<VERSION_PREFIX>_CreationAndDeletion.yaml`.
For example, if we're adding sample for NetworkSecurityGroup resource, check for `Test_Network_v1beta_CreationAndDeletion.yaml`

Run the test and record it:

``` bash
$ TEST_FILTER=Test_Samples_CreationAndDeletion task controller:test-integration-envtest
```

Some Azure resources take longer to provision or delete than the default test timeout of 15m, so you may need to add the `TIMEOUT` environment variable to the command above. For example, to give your test a 60m timeout, use:

``` bash
$ TIMEOUT=60m TEST_FILTER=Test_Samples_CreationAndDeletion task controller:test-integration-envtest
```


## Send a PR

You're all done!
2 changes: 1 addition & 1 deletion docs/hugo/content/design/secrets.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ Below is a table containing a sampling of resources with secrets that ASO alread
| MySQL FlexibleServer | ✔️ | ❌ | ✔️ | `AdministratorLogin` and `AdministratorLoginPassword`. Must have even if using AAD. Can be modified by subsequent PUT. |
| StorageAccount | ❌ | ✔️ | ✔️ | [List Keys API](https://docs.microsoft.com/rest/api/storagerp/storage-accounts/list-keys) and [Regenerate Keys API](https://docs.microsoft.com/en-us/rest/api/storagerp/storage-accounts/regenerate-key). AAD+RBAC (blob/table only?) [Authorizing Access with Active Directory](https://docs.microsoft.com/en-us/azure/storage/blobs/authorize-access-azure-active-directory). |
| CosmosDB DatabaseAccount | ❌ | ✔️ | ✔️ | [List Keys API](https://docs.microsoft.com/en-us/rest/api/cosmos-db-resource-provider/2021-11-15-preview/database-accounts/list-keys), [List Read Only Keys](https://docs.microsoft.com/en-us/rest/api/cosmos-db-resource-provider/2021-11-15-preview/database-accounts/list-read-only-keys) and [Regenerate Key API](https://docs.microsoft.com/en-us/rest/api/cosmos-db-resource-provider/2021-11-15-preview/database-accounts/regenerate-key). For AAD+RBAC (supported by SQL only?), see [Disabling Local Auth](https://docs.microsoft.com/en-us/azure/cosmos-db/how-to-setup-rbac#disable-local-auth), [Create Role Assignment API](https://learn.microsoft.com/en-us/rest/api/cosmos-db-resource-provider/2023-03-15/sqlresources2/create-update-sql-role-assignment), [Create Role Definition API](https://learn.microsoft.com/en-us/rest/api/cosmos-db-resource-provider/2023-03-15/sqlresources2/create-update-sql-role-definition). [Built-in Role Definitions](https://docs.microsoft.com/en-us/azure/cosmos-db/how-to-setup-rbac#built-in-role-definitions). |
| EventHubAuthorizationRules | ❌ | ✔️ | ❌ | [List Keys API](https://docs.microsoft.com/en-us/rest/api/eventhub/stable/authorization-rules-event-hubs/list-keys). There are default authorization rules created, such as `RootManageSharedAccessKey`. Supports [regeneration](https://docs.microsoft.com/en-us/rest/api/eventhub/preview/event-hubs-authorization-rules/regenerate-keys). |
| EventHubAuthorizationRules | ❌ | ✔️ | ❌ | [List Keys API](https://learn.microsoft.com/rest/api/eventhub/controlplane-stable/authorization-rules-event-hubs/list-keys). There are default authorization rules created, such as `RootManageSharedAccessKey`. Supports [regeneration](https://learn.microsoft.com/en-us/rest/api/eventhub/controlplane-preview/event-hubs-authorization-rules/regenerate-keys). |
| Redis | ❌ | ✔️ | ❌ | [List Keys API](https://docs.microsoft.com/en-us/rest/api/redis/redis/list-keys). [Regenerate Key API](https://docs.microsoft.com/en-us/rest/api/redis/redis/regenerate-key). <!-- AAD? --> |

### Other kinds of secrets in Azure:
Expand Down

0 comments on commit 21171f2

Please sign in to comment.