[Investigate App] add log pattern context to assistant hypothesis #195247

dominiqueclarke · 2024-10-07T12:51:26Z

Summary

Adds a route to perform log pattern analysis on all entity sources. Optionally performs log pattern analysis on the entities dependencies as well.

This data is then formatted and passed to the Investigation Contextual Insight. The LLM interprets the patterns and determines which ones may indicate a critical failure.

Example response

Testing

Create some APM data. I'm using the otel demo and triggering a failure via the flagd service. Since this is in flux, you can reach out to me about this workflow. However, you can also create APM data via synth-trace.
Create an custom threshold rule that you expect to trigger an alert. I created mine to using event.outcome: "failure" / event.outcome : * and set a low threshold base on the amount of failures in my current test data. Be sure to also group the alert by service.name
Wait for the alert to fire. Find the alert for the frontend service. This service will have dependencies. Click through to the alert and start an investigation.
Notice the contextual insight. Expand it to see more information

reakaleek · 2024-10-07T12:51:43Z

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

/oblt-deploy : Deploy a Kibana instance using the Observability test environments.
run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

elasticmachine · 2024-10-09T15:45:07Z

Pinging @elastic/obs-ux-management-team (Team:obs-ux-management)

…fix'

jloleysens

Kibana.jsonc LGTM

…miniqueclarke/kibana into fix/investigation-app-log-pattern-llm

…estigation-app-log-pattern-llm

…miniqueclarke/kibana into fix/investigation-app-log-pattern-llm

weltenwort · 2024-10-16T09:53:55Z

x-pack/plugins/observability_solution/investigate_app/server/lib/get_document_categories.ts

@@ -388,6 +424,9 @@ export const createCategorizationRequestParams = ({
  return {
    index,
    size: 0,
+    /* We occassionally end up with a  search_phase_execution_exception Caused by: illegal_argument_exception: 0 > -1


This is a known error I reported here: elastic/elasticsearch#112805

weltenwort · 2024-10-16T10:04:12Z

x-pack/plugins/observability_solution/investigate_app/server/services/get_log_patterns.ts

+    timeField: '@timestamp',
+    messageField: 'message',
+    ignoredCategoryTerms: primaryCategories.categories.map((category) => category.terms),
+    samplingProbability: 0.1,


The idea of the original implementation was to not sample in the second pass as to not miss any rare documents.

…-fix'

elasticmachine · 2024-10-22T15:05:45Z

💔 Build Failed

Buildkite Build
Commit: 53e90c9
Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-195247-53e90c9e3373

Failed CI Steps

Test Failures

[job] [logs] Jest Tests #9 / getPaddedAlertTimeRange active alert with end time than 10 minutes before now
[job] [logs] Jest Tests #9 / getPaddedAlertTimeRange active alert with end time than 10 minutes before now
[job] [logs] Jest Tests #9 / getPaddedAlertTimeRange active alert without end time
[job] [logs] Jest Tests #9 / getPaddedAlertTimeRange active alert without end time
[job] [logs] Jest Tests #9 / getPaddedAlertTimeRange Duration 4 hour, time range will be extended it with 30 minutes from each side
[job] [logs] Jest Tests #9 / getPaddedAlertTimeRange Duration 4 hour, time range will be extended it with 30 minutes from each side
[job] [logs] Jest Tests #9 / getPaddedAlertTimeRange Duration 5 minutes, time range will be extended it with 20 minutes from each side
[job] [logs] Jest Tests #9 / getPaddedAlertTimeRange Duration 5 minutes, time range will be extended it with 20 minutes from each side
[job] [logs] Jest Tests #8 / getViewInAppUrl should call getRedirectUrl with data view, time range and filters
[job] [logs] Jest Tests #8 / getViewInAppUrl should call getRedirectUrl with data view, time range and filters
[job] [logs] Jest Tests #8 / getViewInAppUrl should call getRedirectUrl with empty if there are multiple metrics
[job] [logs] Jest Tests #8 / getViewInAppUrl should call getRedirectUrl with empty if there are multiple metrics
[job] [logs] Jest Tests #8 / getViewInAppUrl should call getRedirectUrl with empty query if metrics and filter are not not provided
[job] [logs] Jest Tests #8 / getViewInAppUrl should call getRedirectUrl with empty query if metrics and filter are not not provided
[job] [logs] Jest Tests #8 / getViewInAppUrl should call getRedirectUrl with filters if group and searchConfiguration filter are provided
[job] [logs] Jest Tests #8 / getViewInAppUrl should call getRedirectUrl with filters if group and searchConfiguration filter are provided
[job] [logs] Jest Tests #8 / getViewInAppUrl should call getRedirectUrl with only count filter
[job] [logs] Jest Tests #8 / getViewInAppUrl should call getRedirectUrl with only count filter
[job] [logs] Jest Tests #8 / getViewInAppUrl should call getRedirectUrl with only filter
[job] [logs] Jest Tests #8 / getViewInAppUrl should call getRedirectUrl with only filter

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`investigateApp`	579	583	+4

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`@kbn/investigation-shared`	82	96	+14

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`investigateApp`	483.5KB	488.7KB	+5.2KB

Unknown metric groups

API count

id	before	after	diff
`@kbn/investigation-shared`	82	96	+14

History

investigate app - add log pattern context to assistant hypothesis

edc48c8

dominiqueclarke added v9.0.0 backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) v8.16.0 Team:obs-ux-management Observability Management User Experience Team labels Oct 9, 2024

add apm dependencies to log pattern analysis

96b3fe9

dominiqueclarke force-pushed the fix/investigation-app-log-pattern-llm branch from 8029f8b to 96b3fe9 Compare October 9, 2024 15:27

merge main

555f173

dominiqueclarke changed the title ~~investigate app - add log pattern context to assistant hypothesis~~ [Investigate App] add log pattern context to assistant hypothesis Oct 9, 2024

dominiqueclarke marked this pull request as ready for review October 9, 2024 15:45

dominiqueclarke requested review from a team as code owners October 9, 2024 15:45

dominiqueclarke added the release_note:skip Skip the PR/issue when compiling release notes label Oct 9, 2024

[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…

aa23902

…fix'

botelastic bot added the ci:project-deploy-observability Create an Observability project label Oct 9, 2024

jloleysens approved these changes Oct 10, 2024

View reviewed changes

dominiqueclarke and others added 7 commits October 11, 2024 10:24

add sample document to LLM prompt

b0f15df

Merge branch 'fix/investigation-app-log-pattern-llm' of github.com:do…

88f3aca

…miniqueclarke/kibana into fix/investigation-app-log-pattern-llm

Merge branch 'main' of https://github.com/elastic/kibana into fix/inv…

3c9121c

…estigation-app-log-pattern-llm

update assistant prompt to include dependencies for service

6476145

Use longer time range from alert

798c7bc

increase the similarity threshold slightly

8b0c7f2

Merge branch 'fix/investigation-app-log-pattern-llm' of github.com:do…

17d75ae

…miniqueclarke/kibana into fix/investigation-app-log-pattern-llm

mgiota self-requested a review October 11, 2024 21:06

weltenwort reviewed Oct 16, 2024

View reviewed changes

kdelemme and others added 2 commits October 16, 2024 08:38

update screen context hook to use unregister fn

2ab63bf

filter out uuids and reduce amount of log patterns

b10acd7

benakansara and others added 3 commits October 21, 2024 22:17

add version release events to contextual insights

299e4b0

[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…

aff3879

…-fix'

Merge branch 'main' into fix/investigation-app-log-pattern-llm

53e90c9

mgiota mentioned this pull request Oct 24, 2024

AI-assisted root cause analysis R&D #197591

Open

dominiqueclarke closed this Nov 1, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Investigate App] add log pattern context to assistant hypothesis #195247

[Investigate App] add log pattern context to assistant hypothesis #195247

dominiqueclarke commented Oct 7, 2024 •

edited

Loading

reakaleek commented Oct 7, 2024

elasticmachine commented Oct 9, 2024

jloleysens left a comment

weltenwort Oct 16, 2024

weltenwort Oct 16, 2024

elasticmachine commented Oct 22, 2024 •

edited

Loading

API count

[Investigate App] add log pattern context to assistant hypothesis #195247

[Investigate App] add log pattern context to assistant hypothesis #195247

Conversation

dominiqueclarke commented Oct 7, 2024 • edited Loading

Summary

Testing

reakaleek commented Oct 7, 2024

🤖 GitHub comments

elasticmachine commented Oct 9, 2024

jloleysens left a comment

Choose a reason for hiding this comment

weltenwort Oct 16, 2024

Choose a reason for hiding this comment

weltenwort Oct 16, 2024

Choose a reason for hiding this comment

elasticmachine commented Oct 22, 2024 • edited Loading

💔 Build Failed

Failed CI Steps

Test Failures

Metrics [docs]

Module Count

Public APIs missing comments

Async chunks

API count

History

dominiqueclarke commented Oct 7, 2024 •

edited

Loading

elasticmachine commented Oct 22, 2024 •

edited

Loading