[Logs UI] Display log analysis job status and offer deletion capability

[weltenwort]

Summary

The logs analysis page should offer the possibility for the user to delete the deployed ML jobs associated with the current space and source.

Rationale

It is important that the user can delete the ML jobs for several reasons:

• Since the ML jobs used for log analysis can use up valuable cluster resources, it might become necessary or economical for the user to delete the jobs.
• An updated version of Kibana might offer new or improved ML job configurations. Since most job parameters can not be changed, there should be a way to delete and recreate the jobs.
• The job and datafeed configurations might have become inconsistent due to various reasons such as data loss, manipulation or deletion of the jobs via the ML UI or source configuration changes. In all of these cases there should be a way to delete the potentially dysfunctional jobs and replace them with consistently configured jobs.

Acceptance Criteria

• The log analysis results page contains a UI element that indicates the overall status of the log analysis jobs.
• If the status is nominal, it has a neutral to positive appearance and offers at least the following pieces of information about the deployed jobs:
  • The index pattern used in the job config.
  • The timestamp field used in the job config.
• If the status is inconsistent, it has an appearance that indicates a warning and, in addition to the nominal information above, displays the reason for the warning:
  • The index pattern or timestamp field differ from the current source configuration.
  • The job revision is older than the revision shipped with the current Kibana version.
  • The job or datafeed are not open and started respectively.
  • The job or datafeed do not exist.
  • Or any other reason...
• To avoid cluttering the UI the information beyond the overall status can be put into a popover that is only displayed after a user action (e.g. click on the status indicator).
• If the status is inconsistent, the popover contains a call to action that navigates to a reset page, which in turn...
  • warns about the loss of result data when the jobs are deleted.
  • offers an unambiguous way for the user to confirm the deletion of the jobs (e.g. by typing a phrase).
  • perform the deletion after user confirmation.
  • display the progress and outcome of the deletion.
  • offers a way to navigate to the setup screen after performing the deletion.
  • offers a way to return to the results page without performing the deletion.

Mockups

Next step would be to include a warning when chaning the settings

...and a toast message indicating that a new ML job has been created

[elasticmachine]

Pinging @elastic/infra-logs-ui

[katrin-freihofner]

Copy for callout on Log rate view
I think we need to cover these three cases:

• The configuration has changed
  You are viewing the results of a new ML job
As your index configuration changed, we automatically created a new ML job. The old job stopped, you can _view the results here_.

• The analysis uses up too many resources
  Your ML job has been stopped
This can happen due to a lack of resources. You can _create a new job here_.

• Someone stopped or deleted jobs
  Your ML job has been stopped or deleted
You can _restart the job here_. If it has been deleted, you can _create a new job_.

[weltenwort]

@katrin-freihofner thanks for adding the mockups! I have a hard time mapping the messages to the technical states though 🤔

Job config is outdated

You are viewing the results of a new ML job
As your index configuration changed, we automatically created a new ML job. The old job stopped, you can view the results here.

We don't automatically create a new ML job. This would only happen in response to an affirmative user action. So instead we would need something like

The configuration of this ML job differs from the source configuration. Recreate the job to apply the current source configuration. This removes previously detected anomalies.

Button: Recreate ML job

Job is not running

Your ML job has been stopped
This can happen due to a lack of resources. You can create a new job here.

We don't distinguish the reason why the job has been stopped. We just notice it's not running. Starting the job doesn't require recreating it, so I imagine the message to be something like

The ML job has been stopped manually or due to lack of resources. New log entries will not be processed until it has been restarted.

Button: Restart ML job

Job is missing

This means that there would be no results, so it would immediate show the setup screen.

We also need some additional states:

Job definition is outdated

This happens when Kibana has been updated with a version in which we ship an updated ML module definition.

A newer version of the ML job is available. Recreate the job to deploy the newest version. This removes previously detected anomalies.

Button: Recreate ML job

Does that make sense? I would image the actions should probably be buttons that call to action as shown in this EUI example?

[katrin-freihofner]

Apparently, I misunderstood that. Yes, it makes sense to add these call to actions as buttons within the callouts. @Titch990 would you mind helping us with the final wording?

[Titch990]

@katrin-freihofner Of course.

Does the text in @weltenwort's comment above reflect the final set of error conditions you want to handle, and the related explanations, or are you still working that out? If the latter, ping me when you are ready.

[weltenwort]

@Titch990 Not sure about "final" 😉 I'll let you know when I have more confidence about the list of messages.

[Titch990]

@weltenwort By "final" I meant a list you were happy for me to review, not "final final" if you get what I mean.

[weltenwort]

@Titch990 I believe the set of messages would be the following. I've started adding them in #47642.

Job configuration is outdated

The job was created based on a different source configuration. Recreate the job to apply the current configuration. This removes previously detected anomalies.

Button: Recreate ML job

Job is not running

The ML job has been stopped manually or due to lack of resources. New log entries will not be processed until it has been restarted.

Job definition is outdated

This happens when Kibana has been updated with a version in which we ship an updated ML module definition.

A newer version of the ML job is available. Recreate the job to deploy the newest version. This removes previously detected anomalies.

Button: Recreate ML job

[Titch990]

Hi there,
Overall, looks good. A few (very minor) suggestions for clarity and consistency across the messages.

The job was created based on a different source configuration. Recreate the job to apply the current configuration. This removes previously detected anomalies.

Perhaps
The ML job was created using a different source configuration. Recreate the job to apply the current configuration. This removes previously detected anomalies.

The ML job has been stopped manually or due to lack of resources. New log entries will not be processed until it has been restarted.

Perhaps
The ML job has been stopped manually or due to a lack of resources. New log entries will not be processed until the job has been restarted.

A newer version of the ML job is available. Recreate the job to deploy the newest version. This removes previously detected anomalies.

Perhaps
A newer version of the ML job is available. Recreate the job to deploy the newer version. This removes previously detected anomalies.

[weltenwort]

closed via #47642