Skip to content

Commit

Permalink
[8.11] [Security Solution] Enable Serverless for exceptions/entry a…
Browse files Browse the repository at this point in the history
…nd `exceptions/rule_details_flow` Cypress tests (#169211) (#169679)

# Backport

This will backport the following commits from `main` to `8.11`:
- [[Security Solution] Enable Serverless for `exceptions/entry` and
`exceptions/rule_details_flow` Cypress tests
(#169211)](#169211)

<!--- Backport version: 8.9.7 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Juan Pablo
Djeredjian","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-10-24T16:04:12Z","message":"[Security
Solution] Enable Serverless for `exceptions/entry` and
`exceptions/rule_details_flow` Cypress tests (#169211)\n\n##
Summary\r\n\r\nFlaky test runner for Cypress tests in:\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/exceptions/entry`
\r\n-\r\n`x-pack/test/security_solution_cypress/cypress/e2e/exceptions/rule_details_flow`\r\n\r\n##
Changes\r\n\r\n-\r\n`x-pack/test/security_solution_cypress/cypress/e2e/exceptions/entry/use_value_list.cy.ts`\r\n**enabled
on
Serverless**\r\n-\r\n`x-pack/test/security_solution_cypress/cypress/e2e/exceptions/entry/multiple_conditions.cy.ts`\r\n**enabled
on
Serverless**\r\n-\r\n`x-pack/test/security_solution_cypress/cypress/e2e/exceptions/rule_details_flow/read_only_view.cy.ts`\r\n**removed
from Serverless testing**\r\n\r\n## Related failing-test
issues\r\n\r\n### `exceptions/entry`\r\n1.
https://github.com/elastic/kibana/issues/165734\r\n2.
https://github.com/elastic/kibana/issues/165652\r\n3.
https://github.com/elastic/kibana/issues/165651\r\n\r\n###
`exceptions/rule_details_flow`\r\n4.
https://github.com/elastic/kibana/issues/165773\r\n5.
https://github.com/elastic/kibana/issues/165742\r\n6.
https://github.com/elastic/kibana/issues/165736\r\n7.
https://github.com/elastic/kibana/issues/163923\r\n8.
https://github.com/elastic/kibana/issues/164007\r\n9.
https://github.com/elastic/kibana/issues/169382\r\n**Unreported flaky
test that was discovered when investigating these\r\nissues. Tagged as
`legit-flake` and assigned
to\r\n@elastic/security-detection-engine**\r\n10.
https://github.com/elastic/kibana/issues/169274\r\n**Flaky test reported
by the Kibana machine while this investigation was\r\ntaking
place.\r\nSince the flaky test runner for the related tests detected
flakiness for\r\nthe same test file (though not same test), I'm marking
this as\r\nlegit-flake for it to be investigated.**\r\n\r\n### Flaky
test runner
link\r\n\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3577\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3621\r\n[SERVERLESS
TAGS
ADJUSTED]","sha":"9a73009c2b2effab436cae61735cccc79af1a21d","branchLabelMapping":{"^v8.12.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Detections
and Resp","Team:Detection
Engine","v8.11.0","v8.12.0"],"number":169211,"url":"https://github.com/elastic/kibana/pull/169211","mergeCommit":{"message":"[Security
Solution] Enable Serverless for `exceptions/entry` and
`exceptions/rule_details_flow` Cypress tests (#169211)\n\n##
Summary\r\n\r\nFlaky test runner for Cypress tests in:\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/exceptions/entry`
\r\n-\r\n`x-pack/test/security_solution_cypress/cypress/e2e/exceptions/rule_details_flow`\r\n\r\n##
Changes\r\n\r\n-\r\n`x-pack/test/security_solution_cypress/cypress/e2e/exceptions/entry/use_value_list.cy.ts`\r\n**enabled
on
Serverless**\r\n-\r\n`x-pack/test/security_solution_cypress/cypress/e2e/exceptions/entry/multiple_conditions.cy.ts`\r\n**enabled
on
Serverless**\r\n-\r\n`x-pack/test/security_solution_cypress/cypress/e2e/exceptions/rule_details_flow/read_only_view.cy.ts`\r\n**removed
from Serverless testing**\r\n\r\n## Related failing-test
issues\r\n\r\n### `exceptions/entry`\r\n1.
https://github.com/elastic/kibana/issues/165734\r\n2.
https://github.com/elastic/kibana/issues/165652\r\n3.
https://github.com/elastic/kibana/issues/165651\r\n\r\n###
`exceptions/rule_details_flow`\r\n4.
https://github.com/elastic/kibana/issues/165773\r\n5.
https://github.com/elastic/kibana/issues/165742\r\n6.
https://github.com/elastic/kibana/issues/165736\r\n7.
https://github.com/elastic/kibana/issues/163923\r\n8.
https://github.com/elastic/kibana/issues/164007\r\n9.
https://github.com/elastic/kibana/issues/169382\r\n**Unreported flaky
test that was discovered when investigating these\r\nissues. Tagged as
`legit-flake` and assigned
to\r\n@elastic/security-detection-engine**\r\n10.
https://github.com/elastic/kibana/issues/169274\r\n**Flaky test reported
by the Kibana machine while this investigation was\r\ntaking
place.\r\nSince the flaky test runner for the related tests detected
flakiness for\r\nthe same test file (though not same test), I'm marking
this as\r\nlegit-flake for it to be investigated.**\r\n\r\n### Flaky
test runner
link\r\n\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3577\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3621\r\n[SERVERLESS
TAGS
ADJUSTED]","sha":"9a73009c2b2effab436cae61735cccc79af1a21d"}},"sourceBranch":"main","suggestedTargetBranches":["8.11"],"targetPullRequestStates":[{"branch":"8.11","label":"v8.11.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.12.0","labelRegex":"^v8.12.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/169211","number":169211,"mergeCommit":{"message":"[Security
Solution] Enable Serverless for `exceptions/entry` and
`exceptions/rule_details_flow` Cypress tests (#169211)\n\n##
Summary\r\n\r\nFlaky test runner for Cypress tests in:\r\n-
`x-pack/test/security_solution_cypress/cypress/e2e/exceptions/entry`
\r\n-\r\n`x-pack/test/security_solution_cypress/cypress/e2e/exceptions/rule_details_flow`\r\n\r\n##
Changes\r\n\r\n-\r\n`x-pack/test/security_solution_cypress/cypress/e2e/exceptions/entry/use_value_list.cy.ts`\r\n**enabled
on
Serverless**\r\n-\r\n`x-pack/test/security_solution_cypress/cypress/e2e/exceptions/entry/multiple_conditions.cy.ts`\r\n**enabled
on
Serverless**\r\n-\r\n`x-pack/test/security_solution_cypress/cypress/e2e/exceptions/rule_details_flow/read_only_view.cy.ts`\r\n**removed
from Serverless testing**\r\n\r\n## Related failing-test
issues\r\n\r\n### `exceptions/entry`\r\n1.
https://github.com/elastic/kibana/issues/165734\r\n2.
https://github.com/elastic/kibana/issues/165652\r\n3.
https://github.com/elastic/kibana/issues/165651\r\n\r\n###
`exceptions/rule_details_flow`\r\n4.
https://github.com/elastic/kibana/issues/165773\r\n5.
https://github.com/elastic/kibana/issues/165742\r\n6.
https://github.com/elastic/kibana/issues/165736\r\n7.
https://github.com/elastic/kibana/issues/163923\r\n8.
https://github.com/elastic/kibana/issues/164007\r\n9.
https://github.com/elastic/kibana/issues/169382\r\n**Unreported flaky
test that was discovered when investigating these\r\nissues. Tagged as
`legit-flake` and assigned
to\r\n@elastic/security-detection-engine**\r\n10.
https://github.com/elastic/kibana/issues/169274\r\n**Flaky test reported
by the Kibana machine while this investigation was\r\ntaking
place.\r\nSince the flaky test runner for the related tests detected
flakiness for\r\nthe same test file (though not same test), I'm marking
this as\r\nlegit-flake for it to be investigated.**\r\n\r\n### Flaky
test runner
link\r\n\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3577\r\n-\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3621\r\n[SERVERLESS
TAGS ADJUSTED]","sha":"9a73009c2b2effab436cae61735cccc79af1a21d"}}]}]
BACKPORT-->

Co-authored-by: Juan Pablo Djeredjian <[email protected]>
  • Loading branch information
kibanamachine and jpdjere authored Oct 26, 2023
1 parent 9f1ce69 commit a0c37df
Show file tree
Hide file tree
Showing 4 changed files with 65 additions and 77 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -27,13 +27,9 @@ import {

import { deleteAlertsAndRules } from '../../../tasks/common';

// TODO: https://github.com/elastic/kibana/issues/161539
// FLAKY: https://github.com/elastic/kibana/issues/165651
// FLAKY: https://github.com/elastic/kibana/issues/165734
// FLAKY: https://github.com/elastic/kibana/issues/165652
describe(
'Add multiple conditions and validate the generated exceptions',
{ tags: ['@ess', '@serverless', '@skipInServerless'] },
{ tags: ['@ess', '@serverless'] },
() => {
beforeEach(() => {
cy.task('esArchiverResetKibana');
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -47,72 +47,66 @@ const goToRulesAndOpenValueListModal = () => {
openValueListsModal();
};

// TODO: https://github.com/elastic/kibana/issues/161539
// Flaky on serverless
describe(
'Use Value list in exception entry',
{ tags: ['@ess', '@serverless', '@skipInServerless'] },
() => {
beforeEach(() => {
cleanKibana();
login();
createListsIndex();
cy.task('esArchiverLoad', { archiveName: 'exceptions' });
importValueList(KNOWN_VALUE_LIST_FILES.TEXT, 'keyword');

createRule(
getNewRule({
query: 'user.name:*',
index: ['exceptions*'],
exceptions_list: [],
rule_id: '2',
enabled: false,
})
).then((rule) => visitRuleDetailsPage(rule.body.id, { tab: 'rule_exceptions' }));
});

afterEach(() => {
cy.task('esArchiverUnload', 'exceptions');
});

it('Should use value list in exception entry, and validate deleting value list prompt', () => {
const ITEM_NAME = 'Exception item with value list';
const ITEM_FIELD = 'agent.name';

// open add exception modal
openExceptionFlyoutFromEmptyViewerPrompt();

// add exception item name
addExceptionFlyoutItemName(ITEM_NAME);

addExceptionEntryFieldValue(ITEM_FIELD, 0);
addExceptionEntryOperatorValue('is in list', 0);

addExceptionEntryFieldMatchIncludedValue(KNOWN_VALUE_LIST_FILES.TEXT, 0);

// The Close all alerts that match attributes in this exception option is disabled
cy.get(CLOSE_ALERTS_CHECKBOX).should('exist');
cy.get(CLOSE_ALERTS_CHECKBOX).should('have.attr', 'disabled');

// Create exception
submitNewExceptionItem();

// displays existing exception items
cy.get(EXCEPTION_ITEM_VIEWER_CONTAINER).should('have.length', 1);
cy.get(NO_EXCEPTIONS_EXIST_PROMPT).should('not.exist');
cy.get(EXCEPTION_CARD_ITEM_NAME).should('have.text', ITEM_NAME);
cy.get(EXCEPTION_CARD_ITEM_CONDITIONS).should(
'have.text',
` ${ITEM_FIELD}included in value_list.txt`
);

// Go back to value list to delete the existing one
goToRulesAndOpenValueListModal();

deleteValueListsFile(KNOWN_VALUE_LIST_FILES.TEXT);

// Toast should be shown because of exception reference
cy.get(EXCEPTIONS_TABLE_MODAL).should('exist');
});
}
);
describe('Use Value list in exception entry', { tags: ['@ess', '@serverless'] }, () => {
beforeEach(() => {
cleanKibana();
login();
createListsIndex();
cy.task('esArchiverLoad', { archiveName: 'exceptions' });
importValueList(KNOWN_VALUE_LIST_FILES.TEXT, 'keyword');

createRule(
getNewRule({
query: 'user.name:*',
index: ['exceptions*'],
exceptions_list: [],
rule_id: '2',
enabled: false,
})
).then((rule) => visitRuleDetailsPage(rule.body.id, { tab: 'rule_exceptions' }));
});

afterEach(() => {
cy.task('esArchiverUnload', 'exceptions');
});

it('Should use value list in exception entry, and validate deleting value list prompt', () => {
const ITEM_NAME = 'Exception item with value list';
const ITEM_FIELD = 'agent.name';

// open add exception modal
openExceptionFlyoutFromEmptyViewerPrompt();

// add exception item name
addExceptionFlyoutItemName(ITEM_NAME);

addExceptionEntryFieldValue(ITEM_FIELD, 0);
addExceptionEntryOperatorValue('is in list', 0);

addExceptionEntryFieldMatchIncludedValue(KNOWN_VALUE_LIST_FILES.TEXT, 0);

// The Close all alerts that match attributes in this exception option is disabled
cy.get(CLOSE_ALERTS_CHECKBOX).should('exist');
cy.get(CLOSE_ALERTS_CHECKBOX).should('have.attr', 'disabled');

// Create exception
submitNewExceptionItem();

// displays existing exception items
cy.get(EXCEPTION_ITEM_VIEWER_CONTAINER).should('have.length', 1);
cy.get(NO_EXCEPTIONS_EXIST_PROMPT).should('not.exist');
cy.get(EXCEPTION_CARD_ITEM_NAME).should('have.text', ITEM_NAME);
cy.get(EXCEPTION_CARD_ITEM_CONDITIONS).should(
'have.text',
` ${ITEM_FIELD}included in value_list.txt`
);

// Go back to value list to delete the existing one
goToRulesAndOpenValueListModal();

deleteValueListsFile(KNOWN_VALUE_LIST_FILES.TEXT);

// Toast should be shown because of exception reference
cy.get(EXCEPTIONS_TABLE_MODAL).should('exist');
});
});
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,6 @@ import {
} from '../../../tasks/api_calls/exceptions';

// TODO: https://github.com/elastic/kibana/issues/161539
// FLAKY: https://github.com/elastic/kibana/issues/165736
describe(
'Add endpoint exception from rule details',
{ tags: ['@ess', '@serverless', '@brokenInServerless'] },
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,7 @@ import {
deleteExceptionList,
} from '../../../tasks/api_calls/exceptions';

// TODO: https://github.com/elastic/kibana/issues/161539 Do we need this to run in Serverless?
describe('Exceptions viewer read only', { tags: ['@ess', '@skipInServerless'] }, () => {
describe('Exceptions viewer read only', { tags: ['@ess'] }, () => {
const exceptionList = getExceptionList();

beforeEach(() => {
Expand Down

0 comments on commit a0c37df

Please sign in to comment.