[UI] Improve App background statuses feedback

[evertonlperes]

Preflight Checklist

• I have searched the issue tracker for a feature request that matches the one I want to file, without success.

Problem Description

Relying on progress bar to make sure that all background services are up/running could be a danger zone.
If something went really bad and the process bar was detached (hidden) and you than you notice something isn't working...
you might need to dig into the logs, access the VM and do a bunch of manual process to double check if everything is working.

Proposed Solution

• Provide a health check endpoint/feature for each services loaded on background
  e.g: K8s cluster healthz check ( kubectl healthz curl -k https://localhost:6443/livez?verbose)
• Add a visual system status on the UI (could be a HTML element, but an element that receive a list with all statuses from the services)

Additional Information

Motivator:
Currently our E2E tests rely on .progress class element. We're assuming after the progress bar being detached from the page, it means that all services were properly loaded (k3s, k8s api, VM, and etc).
So far, it is working, but it will be much better to have feature that we can ensure all the services were loaded properly or not, without rely on a HTML element (visible/hidden states).

[jandubois]

I was wondering if it would make sense to write a JSON file with status information to disk (in the log directory) every time the status changes?

To avoid race conditions between writing and reading the file, the data should always be written to a temp file, which then gets renamed to the canonical name. If the rename fails, the error is ignored because the canonical file is probably locked by somebody reading it.

This file could both be used to wait until RD reaches a specific state, and also for inspecting the last state when the app hangs or crashes.

Not sure if this would be helpful for CI and/or QA.

PS: I just noticed that this issue has a [UI] tag in the subject, so this comment is probably off-topic. 😄

[evertonlperes]

I was wondering if it would make sense to write a JSON file with status information to disk (in the log directory) every time the status changes?

To avoid race conditions between writing and reading the file, the data should always be written to a temp file, which then gets renamed to the canonical name. If the rename fails, the error is ignored because the canonical file is probably locked by somebody reading it.

This file could both be used to wait until RD reaches a specific state, and also for inspecting the last state when the app hangs or crashes.

Not sure if this would be helpful for CI and/or QA.

PS: I just noticed that this issue has a [UI] tag in the subject, so this comment is probably off-topic. 😄

I like the idea to have a centralized feature that we can rely on in order to check if the app is health. IT will be helpful for everyone :)
It will be a plus if we can extend it to the UI either, showing a status label/flag straight to the General page or even in the column tabs.

In terms of CI/QA, your solution seems fine and there's no restrictions from the test framework.
Thanks for sharing your thoughts @jandubois

[rak-phillip]

I think it would make sense to fit in a red/yellow/green status indicator. I imagine that it could include a dropdown with a status check for each of our services, so that we could easily see where a warning or failure occurred. Making use of a status file could provide us with all the information we need to get a quick heads up about the state of our system, while clicking on any given service could open the respective log for further troubleshooting.

[evertonlperes]

I think it would make sense to fit in a red/yellow/green status indicator. I imagine that it could include a dropdown with a status check for each of our services, so that we could easily see where a warning or failure occurred. Making use of a status file could provide us with all the information we need to get a quick heads up about the state of our system, while clicking on any given service could open the respective log for further troubleshooting.

That's another really good solution! Thanks for sharing your thoughts @rak-phillip
What you think to have all those information/features in a App Settings tab? We can also include the Troubleshooting page into a single location... just brainstorming... :D

[evertonlperes]

Tagging another feature request on the same matter: #1354

[jandubois]

Since we are talking about a GUI status indicator, there is another situation that I would like to get a prominent visual alert for: when the current kubernetes context is not "rancher-desktop".

This can happen e.g. when the user works with their EKS cluster one day, boots up the workstation the next day, starts Rancher Desktop, and want to interact with the cluster using kubectl. The command will still go to the EKS cluster instead. This may be intentional, so we cannot change the context for the user automatically, but we should alert them to the mismatch!

I haven't thought this through yet, but we might eventually have a similar situation with Docker contexts, so any alerting mechanism we design should be extensible to that as well.

[jsoref]

I've been meaning to file a bug about automatically switching contexts. I haven't checked to see if docker desktop does this when one enables kubernetes, but in general we found rancher changing the default context from one we were using to be surprising.

If there isn't a selected context, it's fine to select a new one (=rancher-desktop), but otherwise, my default expectation would be to leave it alone.

[jandubois]

@jsoref This should have been fixed recently. See also #622.