-
Notifications
You must be signed in to change notification settings - Fork 30
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #515 from nebari-dev/healthcheck-docs
add intitial healthcheck docs
- Loading branch information
Showing
6 changed files
with
96 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,63 @@ | ||
--- | ||
id: setup-healthcheck | ||
title: Set up healthchecks with Kuberhealthy | ||
description: Set up healthchecks with Kuberhealthy | ||
--- | ||
|
||
:::warning | ||
|
||
This feature is in beta status. It should be used with caution. | ||
|
||
::: | ||
|
||
# Overview | ||
|
||
Nebari integrates [Kuberhealthy](https://kuberhealthy.github.io/kuberhealthy/) to perform internal healthchecks on Nebari. This is an extensible Kubernetes native framework for continuous synthetic testing. Kuberhealthy is set up to export metrics to Prometheus. This allows them to be seen in Grafana. | ||
|
||
## Enabling | ||
|
||
Healthchecks are currently considered a beta feature that we are testing. Due to this, they are disabled by default. To enable healthchecks, add the following configuration under the `monitoring` configuration in your `nebari-config.yaml`. | ||
|
||
```yaml | ||
monitoring: | ||
healthchecks: | ||
enabled: true | ||
``` | ||
## Checking status of Healthchecks | ||
All healthchecks are exported as metrics to Prometheus and can be viewed in Grafana. | ||
For example: To see the uptime for the conda-store service, you can run: | ||
``` | ||
1 - (sum(count_over_time(kuberhealthy_check{check="dev/conda-store-http-check", status="0"}[30d])) OR vector(0))/(sum(count_over_time(kuberhealthy_check{check="dev/conda-store-http-check", status="1"}[30d])) * 100) | ||
``` | ||
|
||
in Grafana, which will show you the following chart. | ||
|
||
![Grafana chart showing the uptime for conda store](/img/how-tos/nebari-healthchecks.png) | ||
|
||
To see what other healthchecks are available, you can use the metric explorer in Grafana. Select the metric type of `kuberhealthy_check` and the label filter of `check`. The values list will be a list of the checks that | ||
have metrics available. | ||
|
||
![Display of available kuberhealthy metrics in Grafana](/img/how-tos/nebari-healthchecks1.png) | ||
|
||
:::note | ||
|
||
If you have previously deployed Nebari without healthchecks, You may need to restart your Prometheus service to get it to pick up the kuberhealthy metrics. | ||
|
||
::: | ||
|
||
## Summary of available healthchecks | ||
|
||
Below is an explanation of the available healthchecks. This list may not be comprehensive as work on this feature is ongoing. | ||
|
||
| <div style={{width:180}}>Check Label</div> | Description | | ||
| ------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | | ||
| conda-store-http-check | verifies that conda-store is accessible via it's REST API | | ||
| jupyterhub-http-check | verifies JupyterHub is running | | ||
| dns-status-internal | verifies internal DNS is accessible | | ||
| daemonset | verifies that a daemonset can be created, fully provisioned, and torn down. This checks the full kubelet functionality of every node in your Kubernetes cluster | | ||
| deployment | verifies that a fresh deployment can run, deploy multiple pods, pass traffic, do a rolling update (without dropping connections), and clean up successfully | | ||
| keycloak-http-check | verifies Keycloak is accessible | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
{ | ||
"devDependencies": { | ||
"prettier": "3.3.3" | ||
} | ||
} |