Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failing test: X-Pack Detection Engine API Integration Tests.x-pack/test/detection_engine_api_integration/security_and_spaces/group10/risk_engine/risk_scoring_task_execution·ts - detection engine api security and spaces enabled - Group 10 Risk Engine - Risk Scoring Task with auditbeat data with some alerts containing hosts initializing the risk engine "before each" hook for "calculates and persists risk scores for alert documents" #168415

Closed
kibanamachine opened this issue Oct 9, 2023 · 14 comments
Closed

Failing test: X-Pack Detection Engine API Integration Tests.x-pack/test/detection_engine_api_integration/security_and_spaces/group10/risk_engine/risk_scoring_task_execution·ts - detection engine api security and spaces enabled - Group 10 Risk Engine - Risk Scoring Task with auditbeat data with some alerts containing hosts initializing the risk engine "before each" hook for "calculates and persists risk scores for alert documents" #168415

kibanamachine opened this issue Oct 9, 2023 · 14 comments
Assignees
@yctercero
Labels
failed-test A test failure on a tracked branch, potentially flaky-test Team:Detection Engine Security Solution Detection Engine Area

Comments

@kibanamachine
Copy link
Contributor

kibanamachine commented Oct 9, 2023 •
edited
Loading

A test failed on a tracked branch

Error: expected 200 "OK", got 400 "Bad Request"
    at Object.init (utils.ts:454:8)
    at Context.apply (risk_scoring_task_execution.ts:98:36)
    at Object.apply (wrap_function.js:73:30)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
----
    at Test._assertStatus (node_modules/supertest/lib/test.js:252:14)
    at /var/lib/buildkite-agent/builds/kb-n2-4-spot-97517926fd011b59/elastic/kibana-on-merge/kibana/node_modules/supertest/lib/test.js:308:13
    at Test._assertFunction (node_modules/supertest/lib/test.js:285:13)
    at Test.assert (node_modules/supertest/lib/test.js:164:23)
    at localAssert (node_modules/supertest/lib/test.js:120:14)
    at fn (node_modules/supertest/lib/test.js:125:7)
    at Test.callback (node_modules/superagent/src/node/index.js:925:3)
    at fn (node_modules/superagent/src/node/index.js:1166:18)
    at IncomingMessage.<anonymous> (node_modules/superagent/src/node/parsers/json.js:19:7)
    at IncomingMessage.emit (node:events:526:35)
    at endReadableNT (node:internal/streams/readable:1359:12)
    at processTicksAndRejections (node:internal/process/task_queues:82:21)

First failure: CI Build - main
@kibanamachine kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label Oct 9, 2023
@botelastic botelastic bot added the needs-team Issues missing a team label label Oct 9, 2023
@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@mistic mistic added the Team:Detection Engine Security Solution Detection Engine Area label Oct 10, 2023
@botelastic botelastic bot removed the needs-team Issues missing a team label label Oct 10, 2023
mistic added a commit that referenced this issue Oct 10, 2023
@mistic
skip flaky suite (#168415)
ed54eaf
@mistic
Copy link
Member

mistic commented Oct 10, 2023 •
edited
Loading

Skipped.

main: ed54eaf
8.11: 787b8ab

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - 8.11

mistic added a commit that referenced this issue Oct 11, 2023
@mistic
skip flaky suite (#168415)
787b8ab
@mistic mistic added the v8.11.1 label Oct 11, 2023
dej611 pushed a commit to dej611/kibana that referenced this issue Oct 17, 2023
@mistic @dej611
skip flaky suite (elastic#168415)
869be91
@jpdjere jpdjere mentioned this issue Oct 18, 2023
Merged
@jpdjere
Copy link
Contributor

jpdjere commented Oct 19, 2023 •
edited
Loading

Flakiness investigated as part of: #169262

Flaky test runner shows no flakiness for this test on 125 runs. (See GROUP 10)

Closing issue but tagging @yctercero and @elastic/security-detection-engine team to keep an eye open on this issue in case it's reopened, especially since the issue has been reported a quite large number of times by the Kibana machine.

@jpdjere jpdjere closed this as completed Oct 19, 2023
jpdjere added a commit that referenced this issue Oct 31, 2023
@jpdjere
[Security Solution] Unskip tests for Detection Engine and Security So…
9eff64a 
…lution API Integration tests (#169262)

## Summary

### Changes

-
`x-pack/test/detection_engine_api_integration/security_and_spaces/group4/telemetry/task_based/detection_rules.ts`
**unskipped**
-
`x-pack/test/detection_engine_api_integration/security_and_spaces/rule_execution_logic/machine_learning.ts`
**unskipped**

### Related flaky test issues

**Tests already moved to
`x-pack/test/security_solution_api_integration`**

1. #169055
2. #168775
This issue is reported for the legacy test, but is now moved to
`x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/exceptions/operators_data_types/ips_text_array/text_array.ts`
3. #163511
This issue is reported for the legacy test, but is now moved to
`x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/exceptions/operators_data_types/keyword_text_long/keyword.ts`

- Flaky test runner for above tests:
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3631

**Test still in `x-pack/test/detection_engine_api_integration`**
1. #168415
2. #164318
3. #164313
4. #145776

**Other**
1. #151636

## Flaky test runner link

Part 1 - `x-pack/test/detection_engine_api_integration` - tests not yet
migrated
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3597
Part 2 - `x-pack/test/security_solution_api_integration` - tests already
moved
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3631
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Oct 31, 2023
@jpdjere
[Security Solution] Unskip tests for Detection Engine and Security So…
8c5cb65 
…lution API Integration tests (elastic#169262)

## Summary

### Changes

-
`x-pack/test/detection_engine_api_integration/security_and_spaces/group4/telemetry/task_based/detection_rules.ts`
**unskipped**
-
`x-pack/test/detection_engine_api_integration/security_and_spaces/rule_execution_logic/machine_learning.ts`
**unskipped**

### Related flaky test issues

**Tests already moved to
`x-pack/test/security_solution_api_integration`**

1. elastic#169055
2. elastic#168775
This issue is reported for the legacy test, but is now moved to
`x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/exceptions/operators_data_types/ips_text_array/text_array.ts`
3. elastic#163511
This issue is reported for the legacy test, but is now moved to
`x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/exceptions/operators_data_types/keyword_text_long/keyword.ts`

- Flaky test runner for above tests:
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3631

**Test still in `x-pack/test/detection_engine_api_integration`**
1. elastic#168415
2. elastic#164318
3. elastic#164313
4. elastic#145776

**Other**
1. elastic#151636

## Flaky test runner link

Part 1 - `x-pack/test/detection_engine_api_integration` - tests not yet
migrated
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3597
Part 2 - `x-pack/test/security_solution_api_integration` - tests already
moved
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3631

(cherry picked from commit 9eff64a)
jpdjere referenced this issue Oct 31, 2023
@kibanamachine @jpdjere
[8.11] [Security Solution] Unskip tests for Detection Engine and Secu…
8a3c213 
…rity Solution API Integration tests (#169262) (#170218)

# Backport

This will backport the following commits from `main` to `8.11`:
- [[Security Solution] Unskip tests for Detection Engine and Security
Solution API Integration tests
(#169262)](#169262)

<!--- Backport version: 8.9.7 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Juan Pablo
Djeredjian","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-10-31T10:36:41Z","message":"[Security
Solution] Unskip tests for Detection Engine and Security Solution API
Integration tests (#169262)\n\n## Summary\r\n\r\n###
Changes\r\n\r\n-\r\n`x-pack/test/detection_engine_api_integration/security_and_spaces/group4/telemetry/task_based/detection_rules.ts`\r\n**unskipped**\r\n-\r\n`x-pack/test/detection_engine_api_integration/security_and_spaces/rule_execution_logic/machine_learning.ts`\r\n**unskipped**\r\n\r\n###
Related flaky test issues\r\n\r\n**Tests already moved
to\r\n`x-pack/test/security_solution_api_integration`**\r\n\r\n1.
https://github.com/elastic/kibana/issues/169055\r\n2.
https://github.com/elastic/kibana/issues/168775\r\nThis issue is
reported for the legacy test, but is now moved
to\r\n`x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/exceptions/operators_data_types/ips_text_array/text_array.ts`\r\n3.
https://github.com/elastic/kibana/issues/163511\r\nThis issue is
reported for the legacy test, but is now moved
to\r\n`x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/exceptions/operators_data_types/keyword_text_long/keyword.ts`\r\n\r\n-
Flaky test runner for above
tests:\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3631\r\n\r\n**Test
still in `x-pack/test/detection_engine_api_integration`**\r\n1.
https://github.com/elastic/kibana/issues/168415\r\n2.
https://github.com/elastic/kibana/issues/164318\r\n3.
https://github.com/elastic/kibana/issues/164313\r\n4.
https://github.com/elastic/kibana/issues/145776\r\n\r\n**Other**\r\n1.
https://github.com/elastic/kibana/issues/151636\r\n\r\n## Flaky test
runner link\r\n\r\nPart 1 -
`x-pack/test/detection_engine_api_integration` - tests not
yet\r\nmigrated\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3597\r\nPart
2 - `x-pack/test/security_solution_api_integration` - tests
already\r\nmoved\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3631","sha":"9eff64a39120e364a4f9a6b71d9501dbb1be0999","branchLabelMapping":{"^v8.12.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:fix","Team:Detections
and Resp","Team:Detection
Engine","v8.11.0","v8.12.0"],"number":169262,"url":"https://github.com/elastic/kibana/pull/169262","mergeCommit":{"message":"[Security
Solution] Unskip tests for Detection Engine and Security Solution API
Integration tests (#169262)\n\n## Summary\r\n\r\n###
Changes\r\n\r\n-\r\n`x-pack/test/detection_engine_api_integration/security_and_spaces/group4/telemetry/task_based/detection_rules.ts`\r\n**unskipped**\r\n-\r\n`x-pack/test/detection_engine_api_integration/security_and_spaces/rule_execution_logic/machine_learning.ts`\r\n**unskipped**\r\n\r\n###
Related flaky test issues\r\n\r\n**Tests already moved
to\r\n`x-pack/test/security_solution_api_integration`**\r\n\r\n1.
https://github.com/elastic/kibana/issues/169055\r\n2.
https://github.com/elastic/kibana/issues/168775\r\nThis issue is
reported for the legacy test, but is now moved
to\r\n`x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/exceptions/operators_data_types/ips_text_array/text_array.ts`\r\n3.
https://github.com/elastic/kibana/issues/163511\r\nThis issue is
reported for the legacy test, but is now moved
to\r\n`x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/exceptions/operators_data_types/keyword_text_long/keyword.ts`\r\n\r\n-
Flaky test runner for above
tests:\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3631\r\n\r\n**Test
still in `x-pack/test/detection_engine_api_integration`**\r\n1.
https://github.com/elastic/kibana/issues/168415\r\n2.
https://github.com/elastic/kibana/issues/164318\r\n3.
https://github.com/elastic/kibana/issues/164313\r\n4.
https://github.com/elastic/kibana/issues/145776\r\n\r\n**Other**\r\n1.
https://github.com/elastic/kibana/issues/151636\r\n\r\n## Flaky test
runner link\r\n\r\nPart 1 -
`x-pack/test/detection_engine_api_integration` - tests not
yet\r\nmigrated\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3597\r\nPart
2 - `x-pack/test/security_solution_api_integration` - tests
already\r\nmoved\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3631","sha":"9eff64a39120e364a4f9a6b71d9501dbb1be0999"}},"sourceBranch":"main","suggestedTargetBranches":["8.11"],"targetPullRequestStates":[{"branch":"8.11","label":"v8.11.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.12.0","labelRegex":"^v8.12.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/169262","number":169262,"mergeCommit":{"message":"[Security
Solution] Unskip tests for Detection Engine and Security Solution API
Integration tests (#169262)\n\n## Summary\r\n\r\n###
Changes\r\n\r\n-\r\n`x-pack/test/detection_engine_api_integration/security_and_spaces/group4/telemetry/task_based/detection_rules.ts`\r\n**unskipped**\r\n-\r\n`x-pack/test/detection_engine_api_integration/security_and_spaces/rule_execution_logic/machine_learning.ts`\r\n**unskipped**\r\n\r\n###
Related flaky test issues\r\n\r\n**Tests already moved
to\r\n`x-pack/test/security_solution_api_integration`**\r\n\r\n1.
https://github.com/elastic/kibana/issues/169055\r\n2.
https://github.com/elastic/kibana/issues/168775\r\nThis issue is
reported for the legacy test, but is now moved
to\r\n`x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/exceptions/operators_data_types/ips_text_array/text_array.ts`\r\n3.
https://github.com/elastic/kibana/issues/163511\r\nThis issue is
reported for the legacy test, but is now moved
to\r\n`x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/exceptions/operators_data_types/keyword_text_long/keyword.ts`\r\n\r\n-
Flaky test runner for above
tests:\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3631\r\n\r\n**Test
still in `x-pack/test/detection_engine_api_integration`**\r\n1.
https://github.com/elastic/kibana/issues/168415\r\n2.
https://github.com/elastic/kibana/issues/164318\r\n3.
https://github.com/elastic/kibana/issues/164313\r\n4.
https://github.com/elastic/kibana/issues/145776\r\n\r\n**Other**\r\n1.
https://github.com/elastic/kibana/issues/151636\r\n\r\n## Flaky test
runner link\r\n\r\nPart 1 -
`x-pack/test/detection_engine_api_integration` - tests not
yet\r\nmigrated\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3597\r\nPart
2 - `x-pack/test/security_solution_api_integration` - tests
already\r\nmoved\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3631","sha":"9eff64a39120e364a4f9a6b71d9501dbb1be0999"}}]}]
BACKPORT-->

Co-authored-by: Juan Pablo Djeredjian <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@yctercero yctercero

Labels
failed-test A test failure on a tracked branch, potentially flaky-test Team:Detection Engine Security Solution Detection Engine Area
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@mistic @jpdjere @yctercero @kibanamachine