Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

initial VMware by Virtustream preview swagger spec #6653

Closed
wants to merge 46 commits into from

Conversation

ctaggart
Copy link
Contributor

@ctaggart ctaggart commented Jul 17, 2019

This is the a initial publish of our preview of our swagger spec. It matches the currently deployed service. The CLI published yesterday at https://github.com/virtustream/azure-vmware-virtustream-cli-extension/releases/tag/0.2.0 used this spec.

.

Latest improvements:

MSFT employees can try out our new experience at OpenAPI Hub - one location for using our validation tools and finding your workflow.

Contribution checklist:

  • I have reviewed the documentation for the workflow.
  • Validation tools were run on swagger spec(s) and have all been fixed in this PR.
  • The OpenAPI Hub was used for checking validation status and next steps.

ARM API Review Checklist

  • Service team MUST add the "WaitForARMFeedback" label if the management plane API changes fall into one of the below categories.
  • adding/removing APIs.
  • adding/removing properties.
  • adding/removing API-version.
  • adding a new service in Azure.

Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs.

  • If you are blocked on ARM review and want to get the PR merged urgently, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.
    Please follow the link to find more details on API review process.

@AutorestCI
Copy link

AutorestCI commented Jul 17, 2019

Automation for azure-sdk-for-go

Encountered an unknown error: (azure-sdk-for-go)

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 33, in exception_to_github
    yield context
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github.py", line 170, in rest_handle_action
    return rest_pull_close(body, restapi_repo, sdk_pr_target_repo, sdkbase, sdk_tag)
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github.py", line 185, in rest_pull_close
    rest_pr_management(rest_pr, sdk_pr_target_repo, sdk_tag, sdk_default_base)
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github_handler.py", line 151, in rest_pr_management
    sdk_tag=sdk_tag
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/SwaggerToSdkNewCLI.py", line 254, in generate_sdk_from_git_object
    with manage_git_folder(gh_token, Path(temp_dir) / Path("rest"), branched_rest_api_id, pr_number=pr_number) as restapi_git_folder, \
  File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 272, in manage_git_folder
    clone_to_path(gh_token, temp_dir, split_git_id[0], branch_or_commit=branch, pr_number=pr_number)
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 212, in clone_to_path
    repo.git.checkout(branch_or_commit)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 548, in <lambda>
    return lambda *args, **kwargs: self._call_process(name, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 1014, in _call_process
    return self.execute(call, **exec_kwargs)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 825, in execute
    raise GitCommandError(command, status, stderr_value, stdout_value)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
  cmdline: git checkout ee9825c58bb9b0677a89122c6e28d1e41ab69fe6
  stderr: 'fatal: reference is not a tree: ee9825c58bb9b0677a89122c6e28d1e41ab69fe6'

@AutorestCI
Copy link

AutorestCI commented Jul 17, 2019

Automation for azure-sdk-for-python

Encountered an unknown error: (azure-sdk-for-python)

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 33, in exception_to_github
    yield context
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github.py", line 170, in rest_handle_action
    return rest_pull_close(body, restapi_repo, sdk_pr_target_repo, sdkbase, sdk_tag)
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github.py", line 185, in rest_pull_close
    rest_pr_management(rest_pr, sdk_pr_target_repo, sdk_tag, sdk_default_base)
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github_handler.py", line 151, in rest_pr_management
    sdk_tag=sdk_tag
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/SwaggerToSdkNewCLI.py", line 254, in generate_sdk_from_git_object
    with manage_git_folder(gh_token, Path(temp_dir) / Path("rest"), branched_rest_api_id, pr_number=pr_number) as restapi_git_folder, \
  File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 272, in manage_git_folder
    clone_to_path(gh_token, temp_dir, split_git_id[0], branch_or_commit=branch, pr_number=pr_number)
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 212, in clone_to_path
    repo.git.checkout(branch_or_commit)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 548, in <lambda>
    return lambda *args, **kwargs: self._call_process(name, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 1014, in _call_process
    return self.execute(call, **exec_kwargs)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 825, in execute
    raise GitCommandError(command, status, stderr_value, stdout_value)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
  cmdline: git checkout ee9825c58bb9b0677a89122c6e28d1e41ab69fe6
  stderr: 'fatal: reference is not a tree: ee9825c58bb9b0677a89122c6e28d1e41ab69fe6'

@AutorestCI
Copy link

AutorestCI commented Jul 17, 2019

Automation for azure-sdk-for-java

Nothing to generate for azure-sdk-for-java

@azuresdkci
Copy link
Contributor

Can one of the admins verify this patch?

@ctaggart
Copy link
Contributor Author

ctaggart commented Jul 17, 2019

I ran the latest autorest validator and I just fixed one of the seven errors so far. The remaining six are:

  • ERROR (PutGetPatchResponseSchema/R3007/ARMViolation): /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.VMwareVirtustream/privateClouds/{privateCloudName} has different responses for PUT/GET/PATCH operations. The PUT/GET/PATCH operations must have same schema response.

    • file:///src/vmwarevirtustream.json:167:4 ($.paths["/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.VMwareVirtustream/privateClouds/{privateCloudName}"])
  • ERROR (OperationsAPIImplementation/R3023/ARMViolation): Operations API must be implemented for '/providers/Microsoft.VMwareVirtustream/operations'.

    • file:///src/vmwarevirtustream.json:78:2 ($.paths)
  • ERROR (XmsResourceInPutResponse/R2062/ARMViolation): The 200 response model for an ARM PUT operation must have x-ms-azure-resource extension set to true in its hierarchy. Operation: 'PrivateCloud_Create' Model: 'AzureClusterResponse'.

    • file:///src/vmwarevirtustream.json:201:6 ($.paths["/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.VMwareVirtustream/privateClouds/{privateCloudName}"].put)
  • ERROR (XmsResourceInPutResponse/R2062/ARMViolation): The 200 response model for an ARM PUT operation must have x-ms-azure-resource extension set to true in its hierarchy. Operation: 'PrivateCloud_Create' Model: 'AzureClusterResponse'.

    • file:///src/vmwarevirtustream.json:201:6 ($.paths["/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.VMwareVirtustream/privateClouds/{privateCloudName}"].put)
  • ERROR (XmsExamplesRequired/D5001/Documentation): Please provide x-ms-examples describing minimum/maximum property set for response/request payloads for operations.

    • file:///src/vmwarevirtustream.json:78:2 ($.paths)
  • ERROR (RequiredPropertiesMissingInResourceModel/R2020/ARMViolation): Model definition 'AzureClusterResponse' must have the properties 'name', 'id' and 'type' in its hierarchy and these properties must be marked as readonly.

    • file:///src/vmwarevirtustream.json:1386:4 ($.definitions.AzureClusterResponse)

ctaggart added a commit to Azure/az-vmware-cli that referenced this pull request Jul 17, 2019
@jhendrixMSFT jhendrixMSFT added the WaitForARMFeedback <valid label in PR review process> add this label when ARM review is required label Jul 17, 2019
@openapi-sdkautomation
Copy link

openapi-sdkautomation bot commented Jul 17, 2019

In Testing, Please Ignore

[Logs] (Generated from 285c483, Iteration 33)

Warning JavaScript: test-repo-billy/azure-sdk-for-js [Logs] [Diff]
  • Warning @azure/arm-vmwarevirtustream [Logs]
Warning Python: test-repo-billy/azure-sdk-for-python [Logs] [Diff]
  • No packages generated.
Warning .NET: test-repo-billy/azure-sdk-for-net [Logs] [Diff]
  • No packages generated.

@jhendrixMSFT
Copy link
Member

The README.md file is missing, please see here for an example. If you want to automatically build SDKs for this swagger please also include the README.*.md files for the various languages and I will tag the language owners to review.

@ctaggart
Copy link
Contributor Author

Errors from the autorest validator have all been fixed except for providing x-ms-examples. Can we add those for the next preview?

Copy link
Member

@jhendrixMSFT jhendrixMSFT left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add missing README.md file.

@jhendrixMSFT
Copy link
Member

Please also provide examples for each operation as they are used for validation and doc generation. See this for an example.

@jhendrixMSFT
Copy link
Member

Looks like our comments crossed regarding examples. For new swaggers, it's a requirement that they include examples before merging, unless this is a time-critical release.

@KrisBash
Copy link
Contributor

@ctaggart Has this spec undergone ARM review (in meeting or a prior swagger version)?

@ctaggart
Copy link
Contributor Author

The README.md file is missing, please see here for an example. If you want to automatically build SDKs for this swagger please also include the README.*.md files for the various languages and I will tag the language owners to review.

@jhendrixMSFT, I added a readme.md. We use TypeScript, Python, and .NET, so those are are initial targets.

@ctaggart
Copy link
Contributor Author

ctaggart commented Jul 17, 2019

Has this spec undergone ARM review (in meeting or a prior swagger version)?

@KrisBash, it has had some review. Shoot me an email and I'll give you more details. My email is on these commits. It is the standard [email protected].

Please also provide examples for each operation as they are used for validation and doc generation. See this for an example.

Okay, I'll begin putting that together tomorrow.

@anthony-c-martin anthony-c-martin added the ARMChangesRequested <valid label in PR review process>add this label when require changes after ARM review label Jul 26, 2019
@daviwil
Copy link
Contributor

daviwil commented Jul 26, 2019

@OpenAPIBot sdkautomation rebuild

@ctaggart
Copy link
Contributor Author

ctaggart commented Jul 26, 2019

@antmarti-microsoft, thanks for the ARM review. I've made all of the swagger updates that I can make and still remain compatible with the version 2016-09-01-preview that is deployed.

There are several changes that you would like to see that would need to go into a new version of the API. May be we can call it 2019-08-09-preview, which is two weeks out. You've requested changes to these operations:

  • PrivateClouds_List
  • Clusters_List

They need to be pageable.

  • PrivateClouds_GetAdminCredentials

https://github.com/Azure/azure-rest-api-specs/pull/6653/files/7aafcf289d83ba332baab86553f52b6c272ac65b#r307617878

The standard for getting credentials is a POST action prefixed with "list" - i.e. "listAdminCredentials". This means this functionality can be utilized during template deployments (for example if you wanted to create a private cloud, fetch credentials, and then pass the credentials to some other resource like a keyvault).

  • PrivateClouds_AddIdentitySource
  • PrivateClouds_DeleteIdentitySource
  • PrivateClouds_AddAuthorization
  • PrivateClouds_DeleteAuthorization

https://github.com/Azure/azure-rest-api-specs/pull/6653/files/7aafcf289d83ba332baab86553f52b6c272ac65b#r307618510

Standard guidance would be to either implement this as a POST action directly on the private cloud resource (i.e. /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.VMwareVirtustream/privateClouds/{privateCloudName}/addAuthorization where the body includes the {authorizationName}), or to implement "authorizations" as an actual proxy resource and support CRUD on it - ARM is quite specific about how it interprets key-value pairs in resource URIs.

https://github.com/Azure/azure-rest-api-specs/pull/6653/files/7aafcf289d83ba332baab86553f52b6c272ac65b#r307618654

Same as above comment - standard guidance would be to either implement this as a POST action directly on the private cloud resource (i.e. /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.VMwareVirtustream/privateClouds/{privateCloudName}/deleteAuthorization where the body includes the {authorizationName}), or to implement "authorizations" as an actual proxy resource and support CRUD on it.


I'm hoping we can get this merged, which matches the API currently deployed, then address those changes in version 2019-08-09-preview.

@ctaggart
Copy link
Contributor Author

@antmarti-microsoft What do you think? Can we get this merged?

Copy link
Contributor

@ravbhatnagar ravbhatnagar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ctaggart - few comments. Please take a look.

"$ref": "#/definitions/PrivateCloudResponse"
}
},
"202": {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just wanted to make sure you want to use the 202 + location header pattern for the long running operations by choice. The pattern we recommend is 201 + non-terminal provisioning State + Azure Async operation header. Details here - https://github.com/Azure/azure-resource-manager-rpc/blob/master/v1.0/Addendum.md#creating-or-updating-resources

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll use 201 and follow the recommended pattern for the recommended pattern for the next API version.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#6878 contains this change in the new API version.

],
"responses": {
"200": {
"description": "successful operation",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It MUST be wrapped in a "value" array even if paging is not supported in v1.

}
}
},
"/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.VMwareVirtustream/privateClouds/{privateCloudName}/deleteAuthorization/{authorizationName}": {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This API is updating the property on the privateClouds resource. It should be enabled via a PATCH on privateClouds rather than modeling it as a POST action.

@ctaggart
Copy link
Contributor Author

ctaggart commented Aug 5, 2019

This API spec matches the deployed version. Any remaining ARM review issues are breaking changes and will be in a version 2019-08-09-preview due out soon. Are you able to merge this pull request as is? If not, what else is needed?

{
"parameters": {
"api-version": "2016-09-01-preview",
"subscriptionId": "{subscription-id}",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] find+replace {subscription-id} in these examples with something that resembles real data (random guid)

],
"responses": {
"200": {
"description": "successful operation",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ctaggart are you planning on addressing this comment (re wrapping in a "value")?

}
}
},
"/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.VMwareVirtustream/privateClouds/{privateCloudName}/addAuthorization/{authorizationName}": {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As per these comments ^ the URL shouldn't include the /{authorizationName} at the end, or ARM will misinterpret this action. https://armwiki.azurewebsites.net/faq/armbasics.html has some information.

}
}
},
"/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.VMwareVirtustream/privateClouds/{privateCloudName}/deleteAuthorization/{authorizationName}": {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was a decision reached on PATCH vs POST?

@ctaggart
Copy link
Contributor Author

This API version did not pass ARM review. @anthony-c-martin, please review #6878 instead, which is a new version of the API to address all issues brought up in the ARM review so far.

@ctaggart ctaggart closed this Aug 12, 2019
ctaggart added a commit to Azure/az-vmware-cli that referenced this pull request Aug 21, 2019
* swagger spec validation

* test latest spec from Azure/azure-rest-api-specs#6653

* update to latest spec in progress

* remove operation status and results

* easier changes from ARM review

* private cloud list will be paged

* added default ApiError

* remove cluster location

* remove privatecloud update

* update to 2019-08-09-preview API

* no longer gen'd

* additional api changes
@ctaggart ctaggart deleted the 2016-09-01-preview branch February 4, 2020 19:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ARMChangesRequested <valid label in PR review process>add this label when require changes after ARM review WaitForARMFeedback <valid label in PR review process> add this label when ARM review is required
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants