Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AzureDataShare: Adding version 2020-09-01 swagger. #12745

Merged
merged 6 commits into from
Feb 5, 2021
Merged

AzureDataShare: Adding version 2020-09-01 swagger. #12745

merged 6 commits into from
Feb 5, 2021

Conversation

ahermas
Copy link
Contributor

@ahermas ahermas commented Jan 30, 2021

Just duplicating the files of the previous stable version 2019 into the newly created folder for version 2020-09-01.

MSFT employees can try out our new experience at OpenAPI Hub - one location for using our validation tools and finding your workflow.

Changelog

Please ensure to add changelog with this PR by answering the following questions.

  1. What's the purpose of the update?
    • new service onboarding
    • new API version
    • update existing version for new feature
    • update existing version to fix swagger quality issue in s360
    • Other, please clarify
  2. When you are targeting to deploy new service/feature to public regions? Please provide date, or month to public if date is not available yet.
  3. When you expect to publish swagger? Please provide date, or month to public if date is not available yet.
  4. If it's an update to existing version, please select SDKs of specific language and CLIs that require refresh after swagger is published.
    • SDK of .NET (need service team to ensure code readiness)
    • SDK of Python
    • SDK of Java
    • SDK of Js
    • SDK of Go
    • PowerShell
    • CLI
    • Terraform
    • No, no need to refresh for updates in this PR

Contribution checklist:

If any further question about AME onboarding or validation tools, please view the FAQ.

ARM API Review Checklist

  • Ensure to check this box if one of the following scenarios meet updates in the PR, so that label “WaitForARMFeedback” will be added automatically to involve ARM API Review. Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs, all “removals” and “adding a new property” no more require ARM API review.

    • Adding new API(s)
    • Adding a new API version
    • Adding a new service
  • Please ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

  • If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist

If there are following updates in the PR, ensure to request an approval from API Review Board as defined in the Breaking Change Policy.

  • Removing API(s) in stable version
  • Removing properties in stable version
  • Removing API version(s) in stable version
  • Updating API in stable version with Breaking Change Validation errors
  • Updating API(s) in preview over 1 year

Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Addition details on the process and office hours are on the Breaking change Wiki.

Please follow the link to find more details on PR review process.

…he newly created folder for version 2020-09-01
@openapi-workflow-bot
Copy link

Hi, @ahermas Thanks for your PR. I am workflow bot for review process. Here are some small tips.

  • Please ensure to do self-check against checklists in first PR comment.
  • PR assignee is the person auto-assigned and responsible for your current PR reviewing and merging.
  • For specs comparison cross API versions, Use API Specs Comparison Report Generator
  • If there is CI failure(s), to fix CI error(s) is mandatory for PR merging; or you need to provide justification in PR comment for explanation. How to fix?

  • Any feedback about review process or workflow bot, pls contact swagger and tools team. [email protected]

    @openapi-pipeline-app
    Copy link

    openapi-pipeline-app bot commented Jan 30, 2021

    Swagger Validation Report

    ️️✔️BreakingChange succeeded [Detail] [Expand]
    There are no breaking changes.
    ️⚠️LintDiff: 1 Warnings warning [Detail]

    Rule Message
    ⚠️ R3018 - EnumInsteadOfBoolean Booleans are not descriptive and make them hard to use. Consider using string enums with allowed set of values defined. Property: fillGapWithZero
    Location: Microsoft.DataShare/stable/2020-09-01/DataShare.json#L5147
    ️️✔️Avocado succeeded [Detail] [Expand]
    Validation passes for Avocado.
    ️️✔️ModelValidation succeeded [Detail] [Expand]
    Validation passes for ModelValidation.
    ️️✔️SemanticValidation succeeded [Detail] [Expand]
    Validation passes for SemanticValidation.
    ️⚠️[Staging] Cross Version BreakingChange (Base on preview version): 8 Warnings warning [Detail]

    Rule Message
    ⚠️ 1005 - RemovedPath The new version is missing a path that was found in the old version. Was path '/providers/Microsoft.DataShare/ListInvitations' removed or restructured?
    Old: Microsoft.DataShare/preview/2018-11-01-preview/DataShare.json#L314:5
    ⚠️ 1005 - RemovedPath The new version is missing a path that was found in the old version. Was path '/providers/Microsoft.DataShare/locations/{location}/RejectInvitation' removed or restructured?
    Old: Microsoft.DataShare/preview/2018-11-01-preview/DataShare.json#L406:5
    ⚠️ 1005 - RemovedPath The new version is missing a path that was found in the old version. Was path '/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.DataShare/accounts/{accountName}/shareSubscriptions/{shareSubscriptionName}/ConsumerSourceDataSets' removed or restructured?
    Old: Microsoft.DataShare/preview/2018-11-01-preview/DataShare.json#L1988:5
    ⚠️ 1005 - RemovedPath The new version is missing a path that was found in the old version. Was path '/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.DataShare/accounts/{accountName}/shareSubscriptions/{shareSubscriptionName}/Synchronize' removed or restructured?
    Old: Microsoft.DataShare/preview/2018-11-01-preview/DataShare.json#L2265:5
    ⚠️ 1010 - AddingRequiredParameter The required parameter 'providerShareSubscription' was added in the new version.
    New: Microsoft.DataShare/stable/2020-09-01/DataShare.json#L1550:11
    ⚠️ 1034 - AddedRequiredProperty The new version has new required property 'sourceShareLocation' that was not found in the old version.
    New: Microsoft.DataShare/stable/2020-09-01/DataShare.json#L5798:7
    Old: Microsoft.DataShare/preview/2018-11-01-preview/DataShare.json#L5674:7
    ⚠️ 1034 - AddedRequiredProperty The new version has new required property 'sourceShareLocation' that was not found in the old version.
    New: Microsoft.DataShare/stable/2020-09-01/DataShare.json#L5763:9
    Old: Microsoft.DataShare/preview/2018-11-01-preview/DataShare.json#L5640:9
    ⚠️ 1034 - AddedRequiredProperty The new version has new required property 'sourceShareLocation' that was not found in the old version.
    New: Microsoft.DataShare/stable/2020-09-01/DataShare.json#L5791:5
    Old: Microsoft.DataShare/preview/2018-11-01-preview/DataShare.json#L5668:5
    ️⚠️[Staging] Cross Version BreakingChange (Base on stable version): 5 Warnings warning [Detail]

    Rule Message
    ⚠️ 1005 - RemovedPath The new version is missing a path that was found in the old version. Was path '/providers/Microsoft.DataShare/ListInvitations' removed or restructured?
    Old: Microsoft.DataShare/stable/2019-11-01/DataShare.json#L314:5
    ⚠️ 1005 - RemovedPath The new version is missing a path that was found in the old version. Was path '/providers/Microsoft.DataShare/locations/{location}/RejectInvitation' removed or restructured?
    Old: Microsoft.DataShare/stable/2019-11-01/DataShare.json#L406:5
    ⚠️ 1005 - RemovedPath The new version is missing a path that was found in the old version. Was path '/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.DataShare/accounts/{accountName}/shareSubscriptions/{shareSubscriptionName}/ConsumerSourceDataSets' removed or restructured?
    Old: Microsoft.DataShare/stable/2019-11-01/DataShare.json#L1988:5
    ⚠️ 1005 - RemovedPath The new version is missing a path that was found in the old version. Was path '/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.DataShare/accounts/{accountName}/shareSubscriptions/{shareSubscriptionName}/Synchronize' removed or restructured?
    Old: Microsoft.DataShare/stable/2019-11-01/DataShare.json#L2265:5
    ⚠️ 1010 - AddingRequiredParameter The required parameter 'providerShareSubscription' was added in the new version.
    New: Microsoft.DataShare/stable/2020-09-01/DataShare.json#L1550:11
    ️️✔️CredScan succeeded [Detail] [Expand]
    There is no credential detected.
    ️⚠️[Staging] SDK Track2 Validation: 3 Warnings warning [Detail]

    Rule Message
    ⚠️ PreCheck/SchemaMissingType "readme":"datashare/resource-manager/readme.md",
    "tag":"package-2020-09-01",
    "details":"The schema 'DefaultDto-systemData' with an undefined type and 'allOf'/'anyOf'/'oneOf' is a bit ambigious. This has been auto-corrected to 'type:object'"
    ⚠️ PreCheck/SchemaMissingType "readme":"datashare/resource-manager/readme.md",
    "tag":"package-2020-09-01",
    "details":"The schema 'ProxyDto-systemData' with an undefined type and 'allOf'/'anyOf'/'oneOf' is a bit ambigious. This has been auto-corrected to 'type:object'"
    ⚠️ PreCheck/CheckDuplicateSchemas "readme":"datashare/resource-manager/readme.md",
    "tag":"package-2020-09-01",
    "details":"Checking for duplicate schemas,
    this could take a (long) while. Run with --verbose for more detail."
    ️️✔️[Staging] PrettierCheck succeeded [Detail] [Expand]
    Validation passes for PrettierCheck.
    ️️✔️[Staging] SpellCheck succeeded [Detail] [Expand]
    Validation passes for SpellCheck.
    Posted by Swagger Pipeline | How to fix these errors?

    @openapi-pipeline-app
    Copy link

    openapi-pipeline-app bot commented Jan 30, 2021

    Swagger Generation Artifacts

    ️️✔️ azure-sdk-for-net succeeded [Detail] [Expand]
    • ️✔️Succeeded [Logs]Release - Generate from e1515e7. SDK Automation 14.0.0
      warn	Skip initScript due to not configured
      command	sudo apt-get install -y dotnet-sdk-5.0
      command	autorest --version=V2 --csharp --reflect-api-versions --license-header=MICROSOFT_MIT_NO_VERSION [email protected]/[email protected] --csharp-sdks-folder=/home/vsts/work/1/s/azure-sdk-for-net/sdk ../azure-rest-api-specs/specification/datashare/resource-manager/readme.md
      cmderr	[Autorest] realpath(): Permission denied
      cmderr	[Autorest] realpath(): Permission denied
      cmderr	[Autorest] realpath(): Permission denied
      cmderr	[Autorest] realpath(): Permission denied
      cmderr	[Autorest] realpath(): Permission denied
      cmderr	[Autorest] realpath(): Permission denied
    • ️✔️Microsoft.Azure.Management.DataShare [View full logs]  [Release SDK Changes]
    ️⚠️ azure-sdk-for-python warning [Detail]
    • ⚠️Warning [Logs]Release - Generate from e1515e7. SDK Automation 14.0.0
      command	sh scripts/automation_init.sh ../azure-sdk-for-python_tmp/initInput.json ../azure-sdk-for-python_tmp/initOutput.json
      cmderr	[automation_init.sh] Ignoring mock: markers 'python_version <= "2.7"' don't match your environment
      cmderr	[automation_init.sh]   Failed building wheel for ConfigArgParse
      cmderr	[automation_init.sh]   Failed building wheel for wrapt
      cmderr	[automation_init.sh]   Failed building wheel for yarl
      cmderr	[automation_init.sh]   Failed building wheel for multidict
      cmderr	[automation_init.sh] Cannot uninstall requirement azure-nspkg, not installed
      cmderr	[automation_init.sh] Command '['/home/vsts/work/1/s/azure-sdk-for-python_tmp/venv-sdk/bin/python', '-m', 'pip', 'uninstall', '-y', 'azure-nspkg']' returned non-zero exit status 1.
      command	sh scripts/automation_generate.sh ../azure-sdk-for-python_tmp/generateInput.json ../azure-sdk-for-python_tmp/generateOutput.json
    • ️✔️azure-mgmt-datashare [View full logs]  [Release SDK Changes] Breaking Change Detected
      info	[Changelog] **Breaking changes**
      info	[Changelog]
      info	[Changelog]   - Operation DataSetMappingsOperations.list_by_share_subscription has a new signature
      info	[Changelog]   - Operation DataSetMappingsOperations.list_by_share_subscription has a new signature
      info	[Changelog]   - Operation DataSetsOperations.list_by_share has a new signature
      info	[Changelog]   - Operation DataSetsOperations.list_by_share has a new signature
      info	[Changelog]   - Operation InvitationsOperations.list_by_share has a new signature
      info	[Changelog]   - Operation InvitationsOperations.list_by_share has a new signature
      info	[Changelog]   - Operation ShareSubscriptionsOperations.list_synchronization_details has a new signature
      info	[Changelog]   - Operation ShareSubscriptionsOperations.list_synchronization_details has a new signature
      info	[Changelog]   - Operation ShareSubscriptionsOperations.list_synchronizations has a new signature
      info	[Changelog]   - Operation ShareSubscriptionsOperations.list_synchronizations has a new signature
      info	[Changelog]   - Operation SharesOperations.list_synchronization_details has a new signature
      info	[Changelog]   - Operation SharesOperations.list_synchronization_details has a new signature
      info	[Changelog]   - Operation SharesOperations.list_synchronizations has a new signature
      info	[Changelog]   - Operation SharesOperations.list_synchronizations has a new signature
      info	[Changelog]   - Operation ShareSubscriptionsOperations.list_by_account has a new signature
      info	[Changelog]   - Operation SharesOperations.list_by_account has a new signature
    ️️✔️ azure-sdk-for-go succeeded [Detail] [Expand]
    • ️✔️Succeeded [Logs]Release - Generate from e1515e7. SDK Automation 14.0.0
      command	sh ./initScript.sh ../../../../../azure-sdk-for-go_tmp/initInput.json ../../../../../azure-sdk-for-go_tmp/initOutput.json
      command	go run ./tools/generator/main.go ../../../../../azure-sdk-for-go_tmp/generateInput.json ../../../../../azure-sdk-for-go_tmp/generateOutput.json
    • ️✔️datashare/mgmt/2019-11-01/datashare [View full logs]  [Release SDK Changes]
      info	[Changelog] No exported changes
    • ️✔️preview/datashare/mgmt/2018-11-01-preview/datashare [View full logs]  [Release SDK Changes]
      info	[Changelog] No exported changes
    ️❌ azure-cli-extensions failed [Detail]
    • Failed [Logs]Release - Generate from e1515e7. Azure CLI Automation 14.0.0
      warn	Skip initScript due to not configured
      command	npx n 12
      command	python3 -m venv venv
      command	pip install --upgrade setuptools wheel pip
      command	pip install pathlib jinja2 msrestazure
      command	autorest --az --use=@autorest/az@latest --version=3.0.6271 --clear-output-folder=true --azure-cli-extension-folder=/home/vsts/work/1/s/azure-cli-extensions ../azure-rest-api-specs/specification/datashare/resource-manager/readme.md
      cmdout	[Autorest] WARNING (PreCheck/CheckDuplicateSchemas): Checking for duplicate schemas, this could take a (long) while.  Run with --verbose for more detail.
      cmderr	[Autorest] FATAL: TypeError: schema.addProperty is not a function
      cmderr	[Autorest]   Error: Plugin flattener reported failure.
      error	Script return with result [failed] code [1] signal [null] cwd [azure-cli-extensions]: autorest
      warn	Skip package processing as generation is failed
    ️️✔️[Staging] ApiDocPreview succeeded [Detail] [Expand]
     Please click here to preview with your @microsoft account. 
    Posted by Swagger Pipeline | How to fix these errors?

    @openapi-workflow-bot
    Copy link

    Hi @ahermas, Your PR has some issues. Please fix the CI sequentially by following the order of Avocado, semantic validation, model validation, breaking change, lintDiff.

    TaskHow to fixPrioritySupport (Microsoft alias)
    AvocadoFix-AvocadoHighruowan
    Semantic validationFix-SemanticValidation-ErrorHighraychen, jianyxi
    Model validationFix-ModelValidation-ErrorHighraychen,jianyxi
    LintDiffFix-LintDiffhighjianyxi, ruoxuan
    If you need further help, please feedback via swagger feedback."

    @ahermas
    Copy link
    Contributor Author

    ahermas commented Jan 30, 2021

    AzureDataShare: First Commit : Just duplicating the files of the previous stable version 2019 into the newly created folder for version 2020-09-01.

    The upcoming second commit will be the real files for the new 2020-09-01. It will include the systemData property of azure resources.

    @ahermas ahermas changed the title AzureDataShare: Just duplicating the files of the previous stable version 2019 into the newly created folder for version 2020-09-01 AzureDataShare: Adding version 2020-09-01 swagger. Jan 30, 2021
    @ahermas
    Copy link
    Contributor Author

    ahermas commented Jan 30, 2021

    The second commit is the that actually add the 2020-09-01 files.

    This is a branch new PR doine what was supposed to be done in this PR.
    @pilor @j5lim @varunkumta Kindly review :)

    @erich-wang erich-wang added the WaitForARMFeedback <valid label in PR review process> add this label when ARM review is required label Feb 1, 2021
    @erich-wang
    Copy link
    Member

    erich-wang commented Feb 1, 2021

    Adding label WaitForARMFeedback as there's new API added for new version.
    @ahermas , please fix CI error first, thanks.

    @ahermas
    Copy link
    Contributor Author

    ahermas commented Feb 1, 2021

    Adding label WaitForARMFeedback as there's new API added for new version.
    @ahermas , please fix CI error first, thanks.

    Hi @erich-wang ,

    Thanks for your comment. I fixed the CI issue and now all checks have passed.

    Thanks

    @jorgecotillo
    Copy link
    Contributor

    Please mark the appropriate checkboxes from above, without marking them we cannot proceed to review your PR, we need to ensure that you reviewed our documentation repository and are following the standard.

    @jorgecotillo jorgecotillo self-assigned this Feb 2, 2021
    @openapi-workflow-bot
    Copy link

    Hi, @ahermas your PR are labelled with WaitForARMFeedback. A notification email will be sent out shortly afterwards to notify ARM review board([email protected]). cc @jorgecotillo

    @jorgecotillo
    Copy link
    Contributor

    jorgecotillo commented Feb 2, 2021

    "SystemData": {
    

    If you are going to use your own definition please follow the property types from common-type:


    Refers to: specification/datashare/resource-manager/Microsoft.DataShare/stable/2020-09-01/DataShare.json:6665 in 1128b45. [](commit_id = 1128b45, deletion_comment = False)

    @ahermas
    Copy link
    Contributor Author

    ahermas commented Feb 2, 2021

        "fillGapWithZero": {
    

    Is this a boolean?

    Refers to: specification/datashare/resource-manager/Microsoft.DataShare/stable/2020-09-01/DataShare.json:5142 in 1128b45. [](commit_id = 1128b45, deletion_comment = False)

    @jorgecotillo fillGapWithZero is a string but we set it to "true" in our code for eg.

    @jorgecotillo jorgecotillo added ARMChangesRequested <valid label in PR review process>add this label when require changes after ARM review and removed WaitForARMFeedback <valid label in PR review process> add this label when ARM review is required labels Feb 2, 2021
    @ahermas
    Copy link
    Contributor Author

    ahermas commented Feb 3, 2021

    @jorgecotillo I updated the PR to address your comment about systemdata to follow the property types from common-type.

    @sarajang
    Copy link
    Member

    sarajang commented Feb 3, 2021

      "get": {
    

    Not a blocker since the endpoint seems that already exists for the previous version, but can it be called 'invitations' instead of listInvitations? list* conflicts with our guidance around returning secrets:

    https://armwiki.azurewebsites.net/rp_onboarding/process/property_design_best_practices.html#secrets

    Refers to: specification/datashare/resource-manager/Microsoft.DataShare/stable/2020-09-01/DataShare.json:315 in 1128b45. [](commit_id = 1128b45, deletion_comment = False)

    @jorgecotillo this is a tenant level fan out query that returns a list of items, we had clarified this in our initial review with ARM and went with their suggestion.

    @sarajang
    Copy link
    Member

    sarajang commented Feb 3, 2021

        "fillGapWithZero": {
    

    Is this a boolean?

    Refers to: specification/datashare/resource-manager/Microsoft.DataShare/stable/2020-09-01/DataShare.json:5142 in 1128b45. [](commit_id = 1128b45, deletion_comment = False)

    @jorgecotillo this is the schema to be integrated with diagnostics logging team.

    @ahermas
    Copy link
    Contributor Author

    ahermas commented Feb 4, 2021

    @jorgecotillo I know that a check is still in progress but please let me know if you're ok with the recent changes to system data and Santosh's comments.

    @jorgecotillo
    Copy link
    Contributor

      "post": {
    

    just making sure, this is not an async operation right?


    Refers to: specification/datashare/resource-manager/Microsoft.DataShare/stable/2020-09-01/DataShare.json:1450 in c334ab5. [](commit_id = c334ab5, deletion_comment = False)

    @jorgecotillo
    Copy link
    Contributor

        "fillGapWithZero": {
    

    If this is a string with a value "true" I assume it can be set to "false" which makes this a boolean. Please update the type to ensure type consistency.


    In reply to: 772095284 [](ancestors = 772095284)


    Refers to: specification/datashare/resource-manager/Microsoft.DataShare/stable/2020-09-01/DataShare.json:5142 in 1128b45. [](commit_id = 1128b45, deletion_comment = False)

    @jorgecotillo
    Copy link
    Contributor

    Address fillGapWithZero type inconsistency please.

    @sarajang
    Copy link
    Member

    sarajang commented Feb 4, 2021

      "post": {
    

    just making sure, this is not an async operation right?

    Refers to: specification/datashare/resource-manager/Microsoft.DataShare/stable/2020-09-01/DataShare.json:1450 in c334ab5. [](commit_id = c334ab5, deletion_comment = False)

    No, this is not an async operation. It returns 200.

    @ahermas
    Copy link
    Contributor Author

    ahermas commented Feb 4, 2021

    Address fillGapWithZero type inconsistency please.

    @jorgecotillo I've addressed the "fillGapWithZero type inconsistency"
    Thanks

    @jorgecotillo jorgecotillo removed their assignment Feb 5, 2021
    @jorgecotillo jorgecotillo added ARMSignedOff <valid label in PR review process>add this label when ARM approve updates after review and removed ARMChangesRequested <valid label in PR review process>add this label when require changes after ARM review labels Feb 5, 2021
    @erich-wang erich-wang merged commit e1515e7 into Azure:master Feb 5, 2021
    mkarmark pushed a commit to mkarmark/azure-rest-api-specs that referenced this pull request Jul 21, 2021
    * Just duplicating the files of the previous stable version 2019 into the newly created folder for version 2020-09-01
    
    * Second Commit : Adding the actual 2020-09-01 swagger files
    
    * Fix CI Avocado errors by editing readme files
    
    * Add UserAssigned enum to 2020-09-01
    
    * Updated systemdata in DataShare according to types.json
    
    * Change fillGapWithZero to boolean instead of string
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    ARMSignedOff <valid label in PR review process>add this label when ARM approve updates after review CI-BreakingChange-Python
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    4 participants