[Logs UI] Add ML job status callouts to results page #47642

weltenwort · 2019-10-08T22:18:41Z

Summary

This adds callouts to the log rate analysis results screen that indicate when the underlying job is out of date, doesn't match the source configuration or has been stopped. It also links to the setup screen in which the user can trigger the redeployment of the jobs.

relates to #44414

Previews

Job definition outdated

Job configuration outdated

Job stopped

Data loss warning

Implementation notes

Since the container that these call-outs are supposed to be located in doesn't exist on master yet, they are located beneath the chart for now. This will change once [Logs UI] Adapt log entry rate data visualisations #47558 has been merged.
The setup state now contains the additional members skippedButUpdatable, skippedButReconfigurable to indicate the new setup states. It also contains requiredForUpdate and requiredForReconfiguration to track the new states when the setup screen is shown.
The setup and job state reducer now stores the job summaries and job definitions and gained several new actions to...
- track the job definitions to check for consistency.
- track the source configuration to check for consistency.
- trigger the setup screen for update/reconfiguration.

Testing

job definition outdated: That check is performed by comparing the job_revision custom setting of the job with the value from x-pack/legacy/plugins/ml/server/models/data_recognizer/modules/logs_ui_analysis/ml/log_entry_rate.json. Incrementing the value in that file and reloading the page should cause this call-out to appear. Following the process redeployment via the included button should deploy a job with the new revision in the custom setting, which should cause the call-out to disappear.
job configuration outdated: Changing the index pattern (e.g. to filebeat-**) or the timestamp field on the source configuration screen should cause this call-out to appear. Following the process redeployment via the included button should deploy a job with the new index pattern, which should cause the call-out to disappear.
job stopped: Stopping the job via the ML UI should cause this call-out to appear. Starting it again via the ML UI should cause this call-out to disappear.

Checklist

This was checked for cross-browser compatibility, including a check against IE11 (testing this on IE11 was not possible yet because Kibana doesn't load there at all right now)
Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
~~Documentation was added for features that require explanation or tutorials~~
~~Unit or functional tests were updated or added to match the most common scenarios~~
This was checked for keyboard-only and screenreader accessibility

…ntry-rate-data-vis

…y350/kibana into 47201-adapt-log-entry-rate-data-vis

elasticmachine · 2019-10-08T22:18:43Z

Pinging @elastic/infra-logs-ui (Team:infra-logs-ui)

elasticmachine · 2019-10-08T23:42:14Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request
Commit: 5e00b3d358b6dca45706e534dbbd26bc9cbcce03

…/log_entry_rate.ts Co-Authored-By: Felix Stürmer <[email protected]>

…y350/kibana into 47201-adapt-log-entry-rate-data-vis

elasticmachine · 2019-10-15T11:30:51Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 6e7f872

…ntry-rate-data-vis

…sis-add-job-status-callouts

elasticmachine · 2019-10-15T16:43:36Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request
Commit: 949ffa1

elasticmachine · 2019-10-15T20:43:29Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request
Commit: b4267a3

elasticmachine · 2019-10-15T21:49:27Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 8828a9d

* Amend outer page / panel structure * Remove unused imports * Add anomalies section and overall chart * Add legend * Rename dataSet to partition * Add max bucket anomaly score * Move job and setup status types to common * Move initial job status fetching into page content * Poll for job status on results page * Add table / expanded row charts * Add bucket span text * Add stat section to expanded rows * Amend annotations on overall graph * Add rule to account for EuiFlexItem edge case * Move functions that handle derivations of data to a new file * Tweak data points fetched * Style bars in grey for anomalies charts * Add severity scoring to annotations * Fix default * Remove decimal places from anomaly score representations * Show all partitions and overall anomaly score in annotation tooltip for overall chart * Handle 'unknown' to workaround lack of '' suuport in tables * Add stats section to overall anomalies section * Base x-domain off the series so that certain buckets aren't omitted * Tweak colours and DRY up annotation rendering * Add sorting to table * Add "number of logs" to API results and render in UI stats * Track and render out-of-sync job configurations * Adjust translation labels * Add stopped state callout * Add more callout icons * Fix api integration tests * Use "pretty" numbers for "Number of logs" stats * Improve status message wording * Change recreate job button color back to default * Add toolbar text * Format all y axis values to 3 digits * Remove "Overall anomaly score" and change all wording / calculations to "Max anomaly scores" * Sort anomaly maximum scores for the overall chart tooltip * Remove unused translations * Use white text with badge in toolbar * Factor out a job recreation callout * Replace `filter()[0]` with `find()` call * Amend key * Use Math.round and introduce a formatAnomalyScore helper function * Update x-pack/legacy/plugins/infra/public/pages/logs/analysis/sections/anomalies/index.tsx Co-Authored-By: Felix Stürmer <[email protected]> * Update x-pack/legacy/plugins/infra/public/pages/logs/analysis/sections/log_rate/index.tsx Co-Authored-By: Felix Stürmer <[email protected]> * Format y-axis of log entry rate chart the same as anomalies charts * Update x-pack/legacy/plugins/infra/public/pages/logs/analysis/sections/anomalies/expanded_row.tsx Co-Authored-By: Felix Stürmer <[email protected]> * Remove grow prop * Update x-pack/legacy/plugins/infra/common/http_api/log_analysis/results/log_entry_rate.ts Co-Authored-By: Felix Stürmer <[email protected]> * Update x-pack/legacy/plugins/infra/common/http_api/log_analysis/results/log_entry_rate.ts Co-Authored-By: Felix Stürmer <[email protected]> * Update x-pack/legacy/plugins/infra/server/routes/log_analysis/results/log_entry_rate.ts Co-Authored-By: Felix Stürmer <[email protected]> * Update x-pack/legacy/plugins/infra/public/pages/logs/analysis/page_results_content.tsx Co-Authored-By: Felix Stürmer <[email protected]> * Update x-pack/legacy/plugins/infra/public/pages/logs/analysis/sections/anomalies/expanded_row.tsx Co-Authored-By: Felix Stürmer <[email protected]> * Update x-pack/legacy/plugins/infra/public/pages/logs/analysis/sections/anomalies/expanded_row.tsx Co-Authored-By: Felix Stürmer <[email protected]> * Update x-pack/legacy/plugins/infra/server/routes/log_analysis/results/log_entry_rate.ts Co-Authored-By: Felix Stürmer <[email protected]> * Update x-pack/legacy/plugins/infra/public/pages/logs/analysis/sections/helpers/data_formatters.tsx Co-Authored-By: Felix Stürmer <[email protected]> * Update x-pack/legacy/plugins/infra/public/pages/logs/analysis/sections/anomalies/expanded_row.tsx Co-Authored-By: Felix Stürmer <[email protected]> * Update x-pack/legacy/plugins/infra/public/pages/logs/analysis/sections/anomalies/expanded_row.tsx Co-Authored-By: Felix Stürmer <[email protected]> * Update x-pack/legacy/plugins/infra/public/pages/logs/analysis/sections/anomalies/expanded_row.tsx Co-Authored-By: Felix Stürmer <[email protected]> * Update x-pack/legacy/plugins/infra/public/pages/logs/analysis/sections/anomalies/index.tsx Co-Authored-By: Felix Stürmer <[email protected]> * Update x-pack/legacy/plugins/infra/public/pages/logs/analysis/sections/anomalies/index.tsx Co-Authored-By: Felix Stürmer <[email protected]> * Update x-pack/legacy/plugins/infra/public/pages/logs/analysis/sections/helpers/data_formatters.tsx Co-Authored-By: Felix Stürmer <[email protected]> * Update x-pack/legacy/plugins/infra/public/pages/logs/analysis/sections/helpers/data_formatters.tsx Co-Authored-By: Felix Stürmer <[email protected]> * Update x-pack/legacy/plugins/infra/public/pages/logs/analysis/sections/helpers/data_formatters.tsx Co-Authored-By: Felix Stürmer <[email protected]> * Update x-pack/legacy/plugins/infra/public/pages/logs/analysis/sections/helpers/data_formatters.tsx Co-Authored-By: Felix Stürmer <[email protected]> * Update x-pack/legacy/plugins/infra/server/lib/log_analysis/log_analysis.ts Co-Authored-By: Felix Stürmer <[email protected]> * Update x-pack/legacy/plugins/infra/server/lib/log_analysis/log_analysis.ts Co-Authored-By: Felix Stürmer <[email protected]> * Update x-pack/legacy/plugins/infra/server/routes/log_analysis/results/log_entry_rate.ts Co-Authored-By: Felix Stürmer <[email protected]> * Update x-pack/legacy/plugins/infra/server/routes/log_analysis/results/log_entry_rate.ts Co-Authored-By: Felix Stürmer <[email protected]> * Change path * Amend property name * Use proper default value argument * Use Kibana dateFormat setting for toolbar formatting * Change logic for calculating severity score categories * Add missing translations * Add interface to top of file * Add no-wrap to tooltip * Use more idomatic code * Use static value for series styles * Move the callouts into the right location * Fix linter warning * Add non-functional ML link button * Fix merge mistake

Backports the following commits to 7.x: - [Logs UI] Add ML job status callouts to results page (#47642)

Kerry350 and others added 16 commits October 3, 2019 15:15

Amend outer page / panel structure

4adc91d

Remove unused imports

62c7a6b

Merge remote-tracking branch 'upstream/master' into 47201-adapt-log-e…

9793f4c

…ntry-rate-data-vis

Add anomalies section and overall chart

0cefd23

Merge remote-tracking branch 'upstream/master' into 47201-adapt-log-e…

b5f3bf8

…ntry-rate-data-vis

Add legend

7524569

Rename dataSet to partition

4119a05

Add max bucket anomaly score

8d887a8

Move job and setup status types to common

6ff0634

Move initial job status fetching into page content

5eeefce

Poll for job status on results page

43c2edf

Merge branch '47201-adapt-log-entry-rate-data-vis' of github.com:Kerr…

bb157fd

…y350/kibana into 47201-adapt-log-entry-rate-data-vis

Add table / expanded row charts

7811629

Add bucket span text

ae059f0

Add stat section to expanded rows

a3820b7

Amend annotations on overall graph

1e8678d

weltenwort added v8.0.0 Feature:Logs UI Logs UI feature Team:Infra Monitoring UI - DEPRECATED DEPRECATED - Label for the Infra Monitoring UI team. Use Team:obs-ux-infra_services v7.5.0 labels Oct 8, 2019

weltenwort self-assigned this Oct 8, 2019

Kerry350 added 7 commits October 9, 2019 13:05

Add rule to account for EuiFlexItem edge case

427105d

Move functions that handle derivations of data to a new file

123084b

Tweak data points fetched

737a25d

Style bars in grey for anomalies charts

ded5117

Add severity scoring to annotations

94fdb76

Fix default

25d6e54

Remove decimal places from anomaly score representations

7be35d2

Kerry350 and others added 3 commits October 15, 2019 12:15

Update x-pack/legacy/plugins/infra/server/routes/log_analysis/results…

721b7f8

…/log_entry_rate.ts Co-Authored-By: Felix Stürmer <[email protected]>

Merge branch '47201-adapt-log-entry-rate-data-vis' of github.com:Kerr…

914790e

…y350/kibana into 47201-adapt-log-entry-rate-data-vis

Change path

a491bff

Kerry350 and others added 14 commits October 15, 2019 12:36

Amend property name

f7787bd

Use proper default value argument

305cdf4

Use Kibana dateFormat setting for toolbar formatting

7f468b4

Change logic for calculating severity score categories

7fa26ed

Add missing translations

d01e5f1

Add interface to top of file

a1493ce

Add no-wrap to tooltip

8197183

Use more idomatic code

5bcad0d

Use static value for series styles

a5d6e45

Merge remote-tracking branch 'upstream/master' into 47201-adapt-log-e…

abae288

…ntry-rate-data-vis

Merge branch '47201-adapt-log-entry-rate-data-vis' into logs-ui-analy…

a688c3b

…sis-add-job-status-callouts

Move the callouts into the right location

2727c69

Fix linter warning

ae7de65

Add non-functional ML link button

949ffa1

Kerry350 mentioned this pull request Oct 15, 2019

[Logs UI] Add "Analyze in ML" buttons #48268

Merged

5 tasks

weltenwort added 2 commits October 15, 2019 21:39

Merge branch 'master' into logs-ui-analysis-add-job-status-callouts

b4267a3

Fix merge mistake

8828a9d

weltenwort merged commit 197a63b into elastic:master Oct 15, 2019

weltenwort deleted the logs-ui-analysis-add-job-status-callouts branch October 15, 2019 22:33

weltenwort mentioned this pull request Oct 15, 2019

[7.x] [Logs UI] Add ML job status callouts to results page (#4… #48309

Merged

weltenwort added a commit that referenced this pull request Oct 16, 2019

[7.x] [Logs UI] Add ML job status callouts to results page (#4… (#48309)

4cd02c2

Backports the following commits to 7.x: - [Logs UI] Add ML job status callouts to results page (#47642)

Kerry350 mentioned this pull request Oct 16, 2019

[7.5] [Logs UI] Add "Analyze in ML" buttons (#48268) #48418

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Logs UI] Add ML job status callouts to results page #47642

[Logs UI] Add ML job status callouts to results page #47642

weltenwort commented Oct 8, 2019 •

edited

Loading

elasticmachine commented Oct 8, 2019

elasticmachine commented Oct 8, 2019

elasticmachine commented Oct 15, 2019

elasticmachine commented Oct 15, 2019

elasticmachine commented Oct 15, 2019

elasticmachine commented Oct 15, 2019

[Logs UI] Add ML job status callouts to results page #47642

[Logs UI] Add ML job status callouts to results page #47642

Conversation

weltenwort commented Oct 8, 2019 • edited Loading

Summary

Previews

Implementation notes

Testing

Checklist

elasticmachine commented Oct 8, 2019

elasticmachine commented Oct 8, 2019

💔 Build Failed

elasticmachine commented Oct 15, 2019

💚 Build Succeeded

elasticmachine commented Oct 15, 2019

💔 Build Failed

elasticmachine commented Oct 15, 2019

💔 Build Failed

elasticmachine commented Oct 15, 2019

💚 Build Succeeded

weltenwort commented Oct 8, 2019 •

edited

Loading