Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add entity query templates with examples #15306

Merged
merged 4 commits into from
Aug 31, 2021
Merged

add entity query templates with examples #15306

merged 4 commits into from
Aug 31, 2021

Conversation

ori-licht
Copy link
Contributor

@ori-licht ori-licht commented Jul 22, 2021 •
edited
Loading

MSFT employees can try out our new experience at OpenAPI Hub - one location for using our validation tools and finding your workflow.

Changelog

Please ensure to add changelog with this PR by answering the following questions.

  1. What's the purpose of the update?
    • new service onboarding
    • new API version
    • update existing version for new feature
    • update existing version to fix swagger quality issue in s360
    • Other, please clarify
  2. When you are targeting to deploy new service/feature to public regions? Please provide date, or month to public if date is not available yet.
  3. When you expect to publish swagger? Please provide date, or month to public if date is not available yet.
  4. If it's an update to existing version, please select SDKs of specific language and CLIs that require refresh after swagger is published.
    • SDK of .NET (need service team to ensure code readiness)
    • SDK of Python
    • SDK of Java
    • SDK of Js
    • SDK of Go
    • PowerShell
    • CLI
    • Terraform
    • No, no need to refresh for updates in this PR

Contribution checklist:

If any further question about AME onboarding or validation tools, please view the FAQ.

ARM API Review Checklist

  • Ensure to check this box if one of the following scenarios meet updates in the PR, so that label “WaitForARMFeedback” will be added automatically to involve ARM API Review. Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs, all “removals” and “adding a new property” no more require ARM API review.

    • Adding new API(s)
    • Adding a new API version
    • Ensure to copy the existing version into new directory structure for first commit (including refactoring) and then push new changes including version updates in separate commits. This is required to review the changes efficiently.
    • Adding a new service

  • Please ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

  • If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist

If there are following updates in the PR, ensure to request an approval from Breaking Change Review Board as defined in the Breaking Change Policy.

  • Removing API(s) in stable version
  • Removing properties in stable version
  • Removing API version(s) in stable version
  • Updating API in stable or public preview version with Breaking Change Validation errors
  • Updating API(s) in public preview over 1 year (refer to Retirement of Previews)

Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Addition details on the process and office hours are on the Breaking change Wiki.

Please follow the link to find more details on PR review process.

@openapi-workflow-bot
Copy link

Hi, @ori-licht Thanks for your PR. I am workflow bot for review process. Here are some small tips.

  • Please ensure to do self-check against checklists in first PR comment.
  • PR assignee is the person auto-assigned and responsible for your current PR reviewing and merging.
  • For specs comparison cross API versions, Use API Specs Comparison Report Generator
  • If there is CI failure(s), to fix CI error(s) is mandatory for PR merging; or you need to provide justification in PR comment for explanation. How to fix?

    • Any feedback about review process or workflow bot, pls contact swagger and tools team. [email protected]

    @openapi-workflow-bot
    Copy link

    [Call for Action] To better understand Azure service dev/test scenario, and support Azure service developer better on Swagger and REST API related tests in early phase, please help to fill in with this survey https://aka.ms/SurveyForEarlyPhase. It will take 5 to 10 minutes. If you already complete survey, please neglect this comment. Thanks.

    @openapi-pipeline-app
    Copy link

    openapi-pipeline-app bot commented Jul 22, 2021 •
    edited
    Loading

    Swagger Validation Report

    ️❌BreakingChange: 2 Errors, 0 Warnings failed [Detail]
    Rule Message
    ❌ 1038 - AddedPath The new version is adding a path that was not found in the old version.
    New: Microsoft.SecurityInsights/preview/2021-03-01-preview/EntityQueryTemplates.json#L37:5
    ❌ 1038 - AddedPath The new version is adding a path that was not found in the old version.
    New: Microsoft.SecurityInsights/preview/2021-03-01-preview/EntityQueryTemplates.json#L88:5
    ️❌LintDiff: 1 Errors, 0 Warnings failed [Detail]
    The following errors/warnings are introduced by current PR:
    Rule Message
    ❌ R4017 - TopLevelResourcesListBySubscription The top-level resource 'EntityQueryTemplate' does not have list by subscription operation, please add it.
    Location: Microsoft.SecurityInsights/preview/2021-03-01-preview/EntityQueryTemplates.json#L175


    The following errors/warnings exist before current PR submission:

    Only 30 items are listed, please refer to log for more details.

    Rule Message
    ❌ R4018 - OperationsApiResponseSchema The response schema of operations API '/providers/Microsoft.SecurityInsights/operations' does not match the ARM specification. Please standardize the schema.
    Location: Microsoft.SecurityInsights/preview/2021-03-01-preview/operations.json#L37
    ❌ R4037 - MissingTypeObject The schema 'Resource' is considered an object but without a 'type:object', please add the missing 'type:object'.
    Location: resource-manager/common/1.0/types.json#L52
    ❌ R4037 - MissingTypeObject The schema 'ResourceWithEtag' is considered an object but without a 'type:object', please add the missing 'type:object'.
    Location: resource-manager/common/1.0/types.json#L38
    ❌ R4037 - MissingTypeObject The schema 'SettingList' is considered an object but without a 'type:object', please add the missing 'type:object'.
    Location: Microsoft.SecurityInsights/preview/2021-03-01-preview/Settings.json#L227
    ❌ R4037 - MissingTypeObject The schema 'UebaProperties' is considered an object but without a 'type:object', please add the missing 'type:object'.
    Location: Microsoft.SecurityInsights/preview/2021-03-01-preview/Settings.json#L372
    ❌ R4037 - MissingTypeObject The schema 'Sku' is considered an object but without a 'type:object', please add the missing 'type:object'.
    Location: Microsoft.SecurityInsights/preview/2021-03-01-preview/Settings.json#L398
    ❌ R4037 - MissingTypeObject The schema 'Resource' is considered an object but without a 'type:object', please add the missing 'type:object'.
    Location: resource-manager/common/1.0/types.json#L52
    ❌ R4037 - MissingTypeObject The schema 'ResourceWithEtag' is considered an object but without a 'type:object', please add the missing 'type:object'.
    Location: resource-manager/common/1.0/types.json#L38
    ❌ R4037 - MissingTypeObject The schema 'SentinelOnboardingStatesList' is considered an object but without a 'type:object', please add the missing 'type:object'.
    Location: Microsoft.SecurityInsights/preview/2021-03-01-preview/OnboardingStates.json#L259
    ❌ R4037 - MissingTypeObject The schema 'Resource' is considered an object but without a 'type:object', please add the missing 'type:object'.
    Location: resource-manager/common/1.0/types.json#L52
    ❌ R4037 - MissingTypeObject The schema 'ResourceWithEtag' is considered an object but without a 'type:object', please add the missing 'type:object'.
    Location: resource-manager/common/1.0/types.json#L38
    ❌ R4037 - MissingTypeObject The schema 'OperationsList' is considered an object but without a 'type:object', please add the missing 'type:object'.
    Location: Microsoft.SecurityInsights/preview/2021-03-01-preview/operations.json#L70
    ❌ R4037 - MissingTypeObject The schema 'Operation' is considered an object but without a 'type:object', please add the missing 'type:object'.
    Location: Microsoft.SecurityInsights/preview/2021-03-01-preview/operations.json#L90
    ❌ R4037 - MissingTypeObject The schema 'RepoList' is considered an object but without a 'type:object', please add the missing 'type:object'.
    Location: Microsoft.SecurityInsights/preview/2021-03-01-preview/SourceControls.json#L287
    ❌ R4037 - MissingTypeObject The schema 'Resource' is considered an object but without a 'type:object', please add the missing 'type:object'.
    Location: resource-manager/common/1.0/types.json#L52
    ❌ R4037 - MissingTypeObject The schema 'ResourceWithEtag' is considered an object but without a 'type:object', please add the missing 'type:object'.
    Location: resource-manager/common/1.0/types.json#L38
    ❌ R4037 - MissingTypeObject The schema 'DataConnectorList' is considered an object but without a 'type:object', please add the missing 'type:object'.
    Location: Microsoft.SecurityInsights/preview/2021-03-01-preview/dataConnectors.json#L698
    ❌ R4037 - MissingTypeObject The schema 'Resource' is considered an object but without a 'type:object', please add the missing 'type:object'.
    Location: resource-manager/common/1.0/types.json#L52
    ❌ R4037 - MissingTypeObject The schema 'ResourceWithEtag' is considered an object but without a 'type:object', please add the missing 'type:object'.
    Location: resource-manager/common/1.0/types.json#L38
    ❌ R4037 - MissingTypeObject The schema 'WatchlistList' is considered an object but without a 'type:object', please add the missing 'type:object'.
    Location: Microsoft.SecurityInsights/preview/2021-03-01-preview/Watchlists.json#L448
    ❌ R4037 - MissingTypeObject The schema 'WatchlistItemList' is considered an object but without a 'type:object', please add the missing 'type:object'.
    Location: Microsoft.SecurityInsights/preview/2021-03-01-preview/Watchlists.json#L598
    ❌ R4037 - MissingTypeObject The schema 'Resource' is considered an object but without a 'type:object', please add the missing 'type:object'.
    Location: resource-manager/common/1.0/types.json#L52
    ❌ R4037 - MissingTypeObject The schema 'ResourceWithEtag' is considered an object but without a 'type:object', please add the missing 'type:object'.
    Location: resource-manager/common/1.0/types.json#L38
    ❌ R4037 - MissingTypeObject The schema 'Resource' is considered an object but without a 'type:object', please add the missing 'type:object'.
    Location: resource-manager/common/1.0/types.json#L52
    ❌ R4037 - MissingTypeObject The schema 'ResourceWithEtag' is considered an object but without a 'type:object', please add the missing 'type:object'.
    Location: resource-manager/common/1.0/types.json#L38
    ❌ R4037 - MissingTypeObject The schema 'EntityQueryList' is considered an object but without a 'type:object', please add the missing 'type:object'.
    Location: Microsoft.SecurityInsights/preview/2021-03-01-preview/EntityQueries.json#L258
    ❌ R4037 - MissingTypeObject The schema 'Resource' is considered an object but without a 'type:object', please add the missing 'type:object'.
    Location: resource-manager/common/1.0/types.json#L52
    ❌ R4037 - MissingTypeObject The schema 'ResourceWithEtag' is considered an object but without a 'type:object', please add the missing 'type:object'.
    Location: resource-manager/common/1.0/types.json#L38
    ❌ R4037 - MissingTypeObject The schema 'ResourceWithEtag' is considered an object but without a 'type:object', please add the missing 'type:object'.
    Location: resource-manager/common/1.0/types.json#L38
    ❌ R4037 - MissingTypeObject The schema 'Resource' is considered an object but without a 'type:object', please add the missing 'type:object'.
    Location: resource-manager/common/1.0/types.json#L52
    ️️✔️Avocado succeeded [Detail] [Expand] 
    Validation passes for Avocado.
    ️️✔️ModelValidation succeeded [Detail] [Expand] 
    Validation passes for ModelValidation.
    ️️✔️SemanticValidation succeeded [Detail] [Expand] 
    Validation passes for SemanticValidation.
    ️️✔️Cross-Version Breaking Changes succeeded [Detail] [Expand] 
    There are no breaking changes.
    ️️✔️CredScan succeeded [Detail] [Expand] 
    There is no credential detected.
    ️️✔️[Staging] SDK Track2 Validation succeeded [Detail] [Expand] 
    Validation passes for SDKTrack2Validation

    The following errors/warnings are introduced by current PR:

    |:speech_balloon: AutorestCore/Exception|"readme":"securityinsights/resource-manager/readme.md",
    "tag":"package-2021-03-preview-only",
    "details":"> Installing AutoRest extension '@microsoft.azure/openapi-validator' (1.8.0)"|
    |:speech_balloon: AutorestCore/Exception|"readme":"securityinsights/resource-manager/readme.md",
    "tag":"package-2021-03-preview-only",
    "details":"> Installed AutoRest extension '@microsoft.azure/openapi-validator' (1.8.0->1.8.0)"|


    The following errors/warnings exist before current PR submission:

    |:speech_balloon: AutorestCore/Exception|"readme":"securityinsights/resource-manager/readme.md",
    "tag":"package-2021-03-preview-only",
    "details":"> Loading AutoRest extension '@autorest/modelerfour' (4.15.456->4.15.456)"|

    ️️✔️[Staging] PrettierCheck succeeded [Detail] [Expand] 
    Validation passes for PrettierCheck.
    ️️✔️[Staging] SpellCheck succeeded [Detail] [Expand] 
    Validation passes for SpellCheck.
    ️️✔️[Staging] Lint(RPaaS) succeeded [Detail] [Expand] 
    Validation passes for Lint(RPaaS).
    Posted by Swagger Pipeline | How to fix these errors?

    @openapi-pipeline-app
    Copy link

    openapi-pipeline-app bot commented Jul 22, 2021 •
    edited
    Loading

    Swagger Generation Artifacts

    ️️✔️[Staging] ApiDocPreview succeeded [Detail] [Expand]

    Only 0 items are rendered, please refer to log for more details.

    ️❌[Staging] SDK Breaking Change Tracking failed [Detail]

    Only 0 items are rendered, please refer to log for more details.

    ️️✔️ azure-sdk-for-net succeeded [Detail] [Expand]

    Only 0 items are rendered, please refer to log for more details.

    ️️✔️ azure-sdk-for-go succeeded [Detail] [Expand]

    Only 0 items are rendered, please refer to log for more details.

    ️⚠️ azure-sdk-for-python-track2 warning [Detail]

    Only 0 items are rendered, please refer to log for more details.

    ️️✔️ azure-sdk-for-js succeeded [Detail] [Expand]

    Only 0 items are rendered, please refer to log for more details.

    ️❌ azure-cli-extensions failed [Detail]

    Only 0 items are rendered, please refer to log for more details.

    ️⚠️ azure-resource-manager-schemas warning [Detail]

    Only 0 items are rendered, please refer to log for more details.

    Posted by Swagger Pipeline | How to fix these errors?

    @openapi-workflow-bot openapi-workflow-bot bot added the WaitForARMFeedback <valid label in PR review process> add this label when ARM review is required label Jul 22, 2021
    @openapi-workflow-bot
    Copy link

    Hi, @ori-licht your PR are labelled with WaitForARMFeedback. A notification email will be sent out shortly afterwards to notify ARM review board([email protected]).

    @openapi-workflow-bot
    Copy link

    Hi @ori-licht, Your PR has some issues. Please fix the CI sequentially by following the order of Avocado, semantic validation, model validation, breaking change, lintDiff.

    TaskHow to fixPrioritySupport (Microsoft alias)
    AvocadoFix-AvocadoHighruowan
    Semantic validationFix-SemanticValidation-ErrorHighraychen, jianyxi
    Model validationFix-ModelValidation-ErrorHighraychen,jianyxi
    LintDiffFix-LintDiffhighjianyxi, ruoxuan
     If you need further help, please feedback via swagger feedback."

    @openapi-workflow-bot
    Copy link

    NewApiVersionRequired reason:

    A service’s API is a contract with customers and is represented by using the api-version query parameter. Changes such as adding an optional property to a request/response or introducing a new operation is a change to the service’s contract and therefore requires a new api-version value. This is critically important for documentation, client libraries, and customer support.

    EXAMPLE: if a customer calls a service in the public cloud using api-version=2020-07-27, the new property or operation may exist but if they call the service in a government cloud, air-gapped cloud, or Azure Stack Hub cloud using the same api-version, the property or operation may not exist. Because there is no clear relationship between the service api-version and the new property/operation, customers can’t trust the documentation and Azure customer have difficulty helping customers diagnose issues. In addition, each client library version documents the service version it supports. When an optional property or new operation is added to a service and its Swagger, new client libraries must be produced to expose this functionality to customers. Without updating the api-version, it is unclear to customers which version of a client library supports these new features.

    This was referenced Jul 22, 2021
    Closed
    Closed
    Draft
    Closed
    Closed
    @ori-licht ori-licht marked this pull request as ready for review July 22, 2021 14:40
    @PhoenixHe-NV
    Copy link
    Contributor

    @ori-licht Please fix lint

    @j5lim
    Copy link
    Contributor

    j5lim commented Jul 27, 2021

    @ori-licht Please follow the breaking change review checklist in the pr description to get an approval from the breaking change review board.

    @ori-licht
    Copy link
    Contributor Author

    @ori-licht Please fix lint

    fixed, see that the error is regarding "top-level resource"
    which is OK in this case because all SecurityInsights resources are nested under LogAnalytics workspace

    @leniatgh leniatgh assigned jianyexi and unassigned leniatgh Aug 2, 2021
    @JeffreyRichter JeffreyRichter added the Approved-BreakingChange DO NOT USE! OBSOLETE label. See https://github.com/Azure/azure-sdk-tools/issues/6374 label Aug 2, 2021
    @ori-licht
    Copy link
    Contributor Author

    @jianyexi - what else is needed to close this one?
    thank you.

    @jianyexi
    Copy link
    Contributor

    jianyexi commented Aug 3, 2021

    @jianyexi Jianye Xi FTE - what else is needed to close this one?
    thank you.

    No, ready for merge ?

    @ori-licht
    Copy link
    Contributor Author

    @jianyexi Jianye Xi FTE - what else is needed to close this one?
    thank you.

    No, ready for merge ?

    yes,
    I attended the breaking changes weekly yesterday, as you can see Jeffery approved this,
    I have generated the ArmClient and validate it against our BE service - all set from my end (-:

    @ori-licht
    Copy link
    Contributor Author

    @jianyexi Jianye Xi FTE - what else is needed to close this one?
    thank you.

    No, ready for merge ?

    yes,
    I attended the breaking changes weekly yesterday, as you can see Jeffery approved this,
    I have generated the ArmClient and validate it against our BE service - all set from my end (-:

    @jianyexi - any updates regarding this?

    @jianyexi
    Copy link
    Contributor

    jianyexi commented Aug 4, 2021

    @ori-licht needs ARM team sign off

    @ori-licht
    Copy link
    Contributor Author

    @ori-licht needs ARM team sign off

    how can we get this? I don't see anyone from Arm on this PR

    raosuhas
    raosuhas reviewed Aug 6, 2021
    View reviewed changes
    ...iew/2021-03-01-preview/examples/entityQueryTemplates/GetActivityEntityQueryTemplateById.json
    "id": "/subscriptions/d0cfe6b2-9ac0-4464-9919-dccaee2e48c0/resourceGroups/myRg/providers/Microsoft.OperationalInsights/workspaces/myWorkspace/providers/Microsoft.SecurityInsights/entityQueryTemplates/07da3cc8-c8ad-4710-a44e-334cdcb7882b",
    "name": "07da3cc8-c8ad-4710-a44e-334cdcb7882b",
    "type": "Microsoft.SecurityInsights/entityQueryTemplate",
    "kind": "Activity",
    Copy link

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    kind

    kind is a reserved top level property. More details here :

    https://github.com/Azure/azure-resource-manager-rpc/blob/master/v1.0/resource-api-reference.md

    Copy link
    Contributor Author

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    what is the problem with this example?
    when performing "get by id" request, the return resource is of kind "Activity".

    Copy link
    Contributor Author

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    @raosuhas - any updates regarding this?

    Copy link
    Contributor Author

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    @raosuhas - any updates?
    Thank you

    Copy link
    Contributor

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    I think it looks ok, we are using kind in the similar way (as a discriminator):

    azure-rest-api-specs/specification/resources/resource-manager/Microsoft.Resources/stable/2020-10-01/examples/DeploymentScripts_Get.json

    Line 11 in 44017a2

    "kind": "AzurePowerShell",

    @jianyexi
    Copy link
    Contributor

    @ori-licht you can check the 'ARM API Review Checklist' in the first comment , #15306 (comment)

    @ori-licht
    Copy link
    Contributor Author

    @ori-licht you can check the 'ARM API Review Checklist' in the first comment , #15306 (comment)

    Hi,
    I have reviewed the check list, not sure what else is needed.
    I got comments from @raosuhas and reply to them.

    what else is needed here? this PR is open for a month now

    @jorgecotillo jorgecotillo added ARMSignedOff <valid label in PR review process>add this label when ARM approve updates after review and removed WaitForARMFeedback <valid label in PR review process> add this label when ARM review is required labels Aug 25, 2021
    @ori-licht
    Copy link
    Contributor Author

    @ori-licht you can check the 'ARM API Review Checklist' in the first comment , #15306 (comment)

    Hi @jianyexi, I see Arm singed off this PR.
    can we close it now or something else is needed?
    thank you

    @jianyexi
    Copy link
    Contributor

    Approved, tell me if you want to merge

    @ori-licht
    Copy link
    Contributor Author

    Approved, tell me if you want to merge
    yes, I do want to merge. thank you

    @jianyexi jianyexi merged commit 9206928 into Azure:main Aug 31, 2021
    This was referenced Aug 31, 2021
    Closed
    Closed
    Closed
    Closed
    Closed
    @anat-gilenson anat-gilenson mentioned this pull request Oct 3, 2021
    Merged
    26 tasks
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Reviewers

    @jorgecotillo jorgecotillo jorgecotillo left review comments

    @raosuhas raosuhas raosuhas left review comments

    @jianyexi jianyexi jianyexi approved these changes

    Assignees

    @jianyexi jianyexi

    Labels
    Approved-BreakingChange DO NOT USE! OBSOLETE label. See https://github.com/Azure/azure-sdk-tools/issues/6374 Approved-LintDiff ARMSignedOff <valid label in PR review process>add this label when ARM approve updates after review CI-BreakingChange-Go CI-FixRequiredOnFailure
    Projects
    None yet
    Milestone
    No milestone
    Development

    Successfully merging this pull request may close these issues.
    9 participants
    @ori-licht @PhoenixHe-NV @j5lim @jianyexi @jorgecotillo @raosuhas @zhenglaizhang @JeffreyRichter @leniatgh