Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent KubeletDown alerts on custom project monitoring setup #22

Merged
merged 1 commit into from
Oct 19, 2023

Conversation

groundhog2k
Copy link
Contributor

Default monitoring setup for CaaS projects shows "KuebeletDown" alerts in alertmanager. @eumel8 we spoke abot that in chat.
To prevent that, the KubeletDown alert should be disabled by default, which is prepared with this change.

@puffitos
Copy link
Member

LGTM. Interestingly enough, disabling the kubelet doesn't seem to remove all kubelet related alerts (I didn't do a deep-dive to search where they are created):

KubeletTooManyPods (0 active)
KubeletPlegDurationHigh (0 active)
KubeletPodStartUpLatencyHigh (0 active)
KubeletClientCertificateExpiration (0 active)
KubeletClientCertificateExpiration (0 active)
KubeletServerCertificateExpiration (0 active)
KubeletServerCertificateExpiration (0 active)
KubeletClientCertificateRenewalErrors (0 active)
KubeletServerCertificateRenewalErrors (0 active)

@puffitos puffitos requested a review from eumel8 October 18, 2023 06:43
@puffitos puffitos self-assigned this Oct 18, 2023
@groundhog2k
Copy link
Contributor Author

groundhog2k commented Oct 18, 2023

@puffitos I think this is ok. Because we only see "red alert" for KubeletDown, all the other metrics are "green". I didn't want to kill them all if not necessary. -- Info: All the kubelet alerts are defined in https://github.com/prometheus-community/helm-charts/blob/main/charts/kube-prometheus-stack/templates/prometheus/rules-1.14/kubernetes-system-kubelet.yaml

@eumel8 eumel8 merged commit 7270880 into caas-team:main Oct 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants