-
Notifications
You must be signed in to change notification settings - Fork 2.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Prometheus fix inf bucket count #15287
Prometheus fix inf bucket count #15287
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
Are you planning to fix the prometheus exporter separately?
This change fixed it: prometheus/client_golang#1148 Or is there something I missed? |
I just noticed a change in pkg/translator/prometheusremotewrite/helper.go, but not in the prometheus exporter. Is the dependency bump all that is required for that exporter? |
Yes, as far as I can tell, that is all that is required. |
+Inf bucket counts are handled incorrectly in both prometheusexporter and prometheusremotewriteexporter.
Description:
+Inf
bucket counts are handled incorrectly in bothprometheusexporter
andprometheusremotewriteexporter
.In prometheus this bug was taken care of by the following PR: prometheus/client_golang#1148
For
prometheusremotewriteexporter
we need to make the same fix and assign value of the total count to the+Inf
bucket as well.This issue caused New Relic to drop our histogram datapoints that had invalid values. It was a problem with 0.61.0 as well, but became much worse with 0.62.0. We built a custom image updating
github.com/prometheus/client_golang
to the SHA version (dcea97eee2b3257f34fd3203cb922eedeabb42a6
) that contained our fix and the issue disappeared:Link to tracking Issue: #4975