[Security Solution] Unskip flaky Cypress tests for Related Integrations #154663
Closed
Assignees
Labels
8.10 candidate
Feature:Related Integrations
Security Solution Detection Rules Related Integrations feature
impact:high
Addressing this issue will have a high level of impact on the quality/strength of our product.
skipped-test
Team:Detection Rule Management
Security Detection Rule Management Team
Team:Detections and Resp
Security Detection Response Team
Team: SecuritySolution
Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc.
test_ui_functional
test
Comments
spong
added
failed-test
|
Pinging @elastic/security-detections-response (Team:Detections and Resp) |
|
Pinging @elastic/security-solution (Team: SecuritySolution) |
spong
added
v8.8.0
impact:high
This was referenced Apr 10, 2023
banderror
pushed a commit
that referenced
this issue
Apr 11, 2023
## Summary As detailed in #154694, need to address some follow-up flake in addition to #154663. This includes the Rules Table auto-refresh and rule selection suites: [rules_table_auto_refresh.cy.ts](https://github.com/elastic/kibana/blob/ca696ac50c0591acf6723e130d2f9278c2d6ef65/x-pack/plugins/security_solution/cypress/e2e/detection_rules/rules_table_auto_refresh.cy.ts#L46) Failed builds: * https://buildkite.com/elastic/kibana-pull-request/builds/118556#018762fd-eb9b-4210-a1a7-f28a36e304e7 * https://buildkite.com/elastic/kibana-pull-request/builds/118310#018756a3-4f98-4404-81e5-b55b9644c651 [rules_selection.cy.ts](https://github.com/elastic/kibana/blob/3d146298a43e1ba24d83e0ede2758b87e826d0b6/x-pack/plugins/security_solution/cypress/e2e/detection_rules/rules_selection.cy.ts#L34) Failed builds: * https://buildkite.com/elastic/kibana-pull-request/builds/118310#01875708-6710-47a9-bd3f-892e878bbeb1 * https://buildkite.com/elastic/kibana-pull-request/builds/118700#01876bfe-e766-42b7-a99d-bcdd3c02823a
banderror
changed the title
banderror
added
test
test_ui_functional
skipped-test
8.8 candidate
and removed
failed-test
maximpn
added a commit
that referenced
this issue
Aug 31, 2023
**Fixes: #154663 **Fixes: #153684 ## Summary This PR unskips Cypress rule related integration tests ([related_integrations.cy.ts](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/related_integrations/related_integrations.cy.ts)). ## Details Testing approach has changed. Instead of importing a rule and installing agent and package policies via Fleet UI the rule is created by mocking a prebuilt rule asset and Fleet API is used to install required integrations. Along the way it required to add and update some testing selectors as UI had changed while the tests were skipped. ## Flaky test runner [related_integrations.cy.ts (150 runs)](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2995) 🟢
maximpn
added a commit
to maximpn/kibana
that referenced
this issue
Aug 31, 2023
**Fixes: elastic#154663 **Fixes: elastic#153684 ## Summary This PR unskips Cypress rule related integration tests ([related_integrations.cy.ts](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/related_integrations/related_integrations.cy.ts)). ## Details Testing approach has changed. Instead of importing a rule and installing agent and package policies via Fleet UI the rule is created by mocking a prebuilt rule asset and Fleet API is used to install required integrations. Along the way it required to add and update some testing selectors as UI had changed while the tests were skipped. ## Flaky test runner [related_integrations.cy.ts (150 runs)](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2995) 🟢 (cherry picked from commit 2c3464c) # Conflicts: # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/related_integrations/related_integrations.cy.ts # x-pack/test/security_solution_cypress/cypress/tasks/api_calls/prebuilt_rules.ts
maximpn
referenced
this issue
Aug 31, 2023
#165310) # Backport This will backport the following commits from `main` to `8.10`: - [[Security Solution] Unskip related integrations tests (#164933)](#164933) <!--- Backport version: 8.9.8 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Maxim Palenov","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-08-31T10:28:16Z","message":"[Security Solution] Unskip related integrations tests (#164933)\n\n**Fixes: https://github.com/elastic/kibana/issues/154663**\r\n**Fixes: https://github.com/elastic/kibana/issues/153684**\r\n\r\n## Summary\r\n\r\nThis PR unskips Cypress rule related integration tests ([related_integrations.cy.ts](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/related_integrations/related_integrations.cy.ts)).\r\n\r\n## Details\r\n\r\nTesting approach has changed. Instead of importing a rule and installing agent and package policies via Fleet UI the rule is created by mocking a prebuilt rule asset and Fleet API is used to install required integrations. Along the way it required to add and update some testing selectors as UI had changed while the tests were skipped.\r\n\r\n## Flaky test runner\r\n\r\n[related_integrations.cy.ts (150 runs)](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2995) 🟢","sha":"2c3464ca8457e09f51852c6c0d60ab7f6e21afc0","branchLabelMapping":{"^v8.11.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["test","release_note:skip","Team:Detections and Resp","Team: SecuritySolution","Feature:Rule Management","Team:Detection Rule Management","v8.10.0","v8.11.0","flake-docs"],"number":164933,"url":"https://github.com/elastic/kibana/pull/164933","mergeCommit":{"message":"[Security Solution] Unskip related integrations tests (#164933)\n\n**Fixes: https://github.com/elastic/kibana/issues/154663**\r\n**Fixes: https://github.com/elastic/kibana/issues/153684**\r\n\r\n## Summary\r\n\r\nThis PR unskips Cypress rule related integration tests ([related_integrations.cy.ts](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/related_integrations/related_integrations.cy.ts)).\r\n\r\n## Details\r\n\r\nTesting approach has changed. Instead of importing a rule and installing agent and package policies via Fleet UI the rule is created by mocking a prebuilt rule asset and Fleet API is used to install required integrations. Along the way it required to add and update some testing selectors as UI had changed while the tests were skipped.\r\n\r\n## Flaky test runner\r\n\r\n[related_integrations.cy.ts (150 runs)](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2995) 🟢","sha":"2c3464ca8457e09f51852c6c0d60ab7f6e21afc0"}},"sourceBranch":"main","suggestedTargetBranches":["8.10"],"targetPullRequestStates":[{"branch":"8.10","label":"v8.10.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.11.0","labelRegex":"^v8.11.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/164933","number":164933,"mergeCommit":{"message":"[Security Solution] Unskip related integrations tests (#164933)\n\n**Fixes: https://github.com/elastic/kibana/issues/154663**\r\n**Fixes: https://github.com/elastic/kibana/issues/153684**\r\n\r\n## Summary\r\n\r\nThis PR unskips Cypress rule related integration tests ([related_integrations.cy.ts](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/related_integrations/related_integrations.cy.ts)).\r\n\r\n## Details\r\n\r\nTesting approach has changed. Instead of importing a rule and installing agent and package policies via Fleet UI the rule is created by mocking a prebuilt rule asset and Fleet API is used to install required integrations. Along the way it required to add and update some testing selectors as UI had changed while the tests were skipped.\r\n\r\n## Flaky test runner\r\n\r\n[related_integrations.cy.ts (150 runs)](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2995) 🟢","sha":"2c3464ca8457e09f51852c6c0d60ab7f6e21afc0"}}]}] BACKPORT-->
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Caused by: #154662
Summary
As detailed in #154662 we've got some flake with our Related Integrations Cypress tests. This issue is for un-skipping this test and resolving the flake.
The text was updated successfully, but these errors were encountered: