Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

{KeyVault} Modify random vault name prefix #12076

Merged
merged 7 commits into from
Feb 7, 2020

Conversation

bim-msft
Copy link
Contributor

@bim-msft bim-msft commented Feb 6, 2020


This checklist is used to make sure that common guidelines for a pull request are followed.

@bim-msft bim-msft requested a review from jiasli February 6, 2020 09:57
@bim-msft
Copy link
Contributor Author

bim-msft commented Feb 6, 2020

@jiasli Reduced the name prefix length to meet the rule and reran all tests according to your advice, please review it again.

@bim-msft
Copy link
Contributor Author

bim-msft commented Feb 6, 2020

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@jiasli
Copy link
Member

jiasli commented Feb 7, 2020

After live run KeyVaultCertificateDefaultPolicyScenarioTest, the recording yaml is removed. CI reports failure:

  File "/opt/hostedtoolcache/Python/3.6.10/x64/lib/python3.6/site-packages/azure/cli
    /command_modules/resource/custom.py", line 736, in create_resource_group
  ...
knack.util.CLIError: Please run 'az login' to setup account.

By looking into ResourceGroupPreparer then live_only_execute, see this line returns True:

if self.test_class_instance.in_recording:

Then find usage of in_recording reveals this line in azure_devtools.

self.in_recording = self.is_live or not os.path.exists(self.recording_file)

How it fails:

  1. Because group create/delete commands are not recorded and there is no network traffic in the test body, after live run the yaml is not created
  2. When running as recording, ReplayableTest gets is_live == False, but still thinks this is a live test because the yaml doesn't exists. In other words, the current design requires ResourceGroupPreparer to be paired with a recording yaml for the test to be treated as recording. Even if no --live or AZURE_TEST_RUN_LIVE is specified, lacking of yaml will still turn this test into a live test
  3. The test is ran as live. This doesn't cause problem locally because az has been logged in, but in CI it has not.

To solve it, simply remove:

@ResourceGroupPreparer(name_prefix='cli_test_kv_cert_default_policy')

@yonzhan yonzhan added this to the S165 milestone Feb 7, 2020
@bim-msft bim-msft merged commit 601dfc0 into Azure:dev Feb 7, 2020
@bim-msft bim-msft deleted the bim_kv_fix_test branch February 7, 2020 15:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants