Skip to content

Commit

Permalink
Update how histogram&summary metrics are reported
Browse files Browse the repository at this point in the history
As New Relic is improving its support for Prometheus native format, we
could change the way how we report Summary and Histogram metric types
and do it in less disruptive way, meaning that we send them as they come
from Prometheus, without adding custom suffixes or calculating
percentiles from the integration side.

These are the details of the current implementation:

- xxx_bucket is ingested as a Counter (converted to a delta rather than
accumulative). Metrics containing dimension "le" == '+Inf' are also
sent, we don't omit them.
- xxx_count is passed through as a Counter (converted to a delta rather than accumulative).
- xxx_total passed through as a Counter (converted to a delta rather than accumulative).
- xxx_sum is passed through as a NR Summary converted to a delta rather
than accumulative, Min and Max here should be NaN. Its value can be negative.
The count field of the summary should be 1.
- For Prometheus summary metrics, we report the quantile as a dimension
and we don't add a "percentile" dimension.
  • Loading branch information
Toni Reina committed Jun 19, 2020
1 parent 276f7cf commit 0250ae2
Show file tree
Hide file tree
Showing 8 changed files with 103 additions and 422 deletions.
6 changes: 1 addition & 5 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,11 @@ module github.com/newrelic/nri-prometheus
go 1.13

require (
github.com/Bowery/prompt v0.0.0-20190916142128-fa8279994f75 // indirect
github.com/dchest/safefile v0.0.0-20151022103144-855e8d98f185 // indirect
github.com/fsnotify/fsnotify v1.4.8-0.20190312181446-1485a34d5d57 // indirect
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 // indirect
github.com/googleapis/gnostic v0.2.3-0.20181019180348-e2aafd60c944 // indirect
github.com/hashicorp/hcl v1.0.1-0.20190611123218-cf7d376da96d // indirect
github.com/imdario/mergo v0.3.8 // indirect
github.com/kardianos/govendor v1.0.9 // indirect
github.com/newrelic/newrelic-telemetry-sdk-go v0.2.1-0.20200116224429-790ff853d12b
github.com/newrelic/newrelic-telemetry-sdk-go v0.3.0
github.com/onsi/ginkgo v1.10.1 // indirect
github.com/onsi/gomega v1.7.0 // indirect
github.com/pelletier/go-toml v1.2.1-0.20181124002727-27c6b39a135b // indirect
Expand Down
12 changes: 2 additions & 10 deletions go.sum
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
github.com/Azure/go-autorest v11.1.2+incompatible/go.mod h1:r+4oMnoxhatjLLJ6zxSWATqVooLgysK6ZNox3g/xq24=
github.com/Bowery/prompt v0.0.0-20190916142128-fa8279994f75 h1:xGHheKK44eC6K0u5X+DZW/fRaR1LnDdqPHMZMWx5fv8=
github.com/Bowery/prompt v0.0.0-20190916142128-fa8279994f75/go.mod h1:4/6eNcqZ09BZ9wLK3tZOjBA1nDj+B0728nlX5YRlSmQ=
github.com/BurntSushi/toml v0.3.1 h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/NYTimes/gziphandler v0.0.0-20170623195520-56545f4a5d46/go.mod h1:3wb06e3pkSAbeQ52E9H9iFoQsEEwGN64994WTCIhntQ=
Expand All @@ -26,8 +24,6 @@ github.com/davecgh/go-spew v0.0.0-20151105211317-5215b55f46b2/go.mod h1:J7Y8YcW2
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dchest/safefile v0.0.0-20151022103144-855e8d98f185 h1:3T8ZyTDp5QxTx3NU48JVb2u+75xc040fofcBaN+6jPA=
github.com/dchest/safefile v0.0.0-20151022103144-855e8d98f185/go.mod h1:cFRxtTwTOJkz2x3rQUNCYKWC93yP1VKjR8NUhqFxZNU=
github.com/dgrijalva/jwt-go v0.0.0-20160705203006-01aeca54ebda/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ=
github.com/dgrijalva/jwt-go v3.2.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ=
github.com/dgryski/go-sip13 v0.0.0-20181026042036-e10d5fee7954/go.mod h1:vAd38F8PWV+bWy6jNmig1y/TA+kYO4g3RSRF0IAv0no=
Expand Down Expand Up @@ -73,8 +69,6 @@ github.com/google/gofuzz v0.0.0-20161122191042-44d81051d367/go.mod h1:HP5RmnzzSN
github.com/google/gofuzz v0.0.0-20170612174753-24818f796faf/go.mod h1:HP5RmnzzSNb993RKQDq4+1A4ia9nllfqcQFTQJedwGI=
github.com/google/gofuzz v1.0.0 h1:A8PeW59pxE9IoFRqBp37U+mSNaQoZ46F1f0f863XSXw=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 h1:El6M4kTTCOh6aBiKaUGG7oYTSPP8MxqL4YI3kZKwcP4=
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510/go.mod h1:pupxD2MaaD3pAXIBCelhxNneeOaAeabZDe5s4K6zSpQ=
github.com/google/uuid v1.0.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.1.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/googleapis/gnostic v0.0.0-20170729233727-0c5108395e2d/go.mod h1:sJBsCZ4ayReDTBIg8b9dl28c5xFWyhBTVRp3pOg5EKY=
Expand Down Expand Up @@ -102,8 +96,6 @@ github.com/json-iterator/go v0.0.0-20180701071628-ab8a2e0c74be/go.mod h1:+SdeFBv
github.com/json-iterator/go v1.1.7 h1:KfgG9LzI+pYjr4xvmz/5H4FXjokeP+rlHLhv3iH62Fo=
github.com/json-iterator/go v1.1.7/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w=
github.com/kardianos/govendor v1.0.9 h1:WOH3FcVI9eOgnIZYg96iwUwrL4eOVx+aQ66oyX2R8Yc=
github.com/kardianos/govendor v1.0.9/go.mod h1:yvmR6q9ZZ7nSF5Wvh40v0wfP+3TwwL8zYQp+itoZSVM=
github.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q=
github.com/kisielk/errcheck v1.2.0/go.mod h1:/BMXB+zMLi60iA8Vv6Ksmxu/1UDYcXs4uQLJ+jE2L00=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
Expand Down Expand Up @@ -132,8 +124,8 @@ github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3Rllmb
github.com/munnerz/goautoneg v0.0.0-20120707110453-a547fc61f48d/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f/go.mod h1:ZdcZmHo+o7JKHSa8/e818NopupXU1YMK5fe1lsApnBw=
github.com/newrelic/newrelic-telemetry-sdk-go v0.2.1-0.20200116224429-790ff853d12b h1:SIJT+jppa/rNo2Dl6gTjvYYeVxxiCZ7q5cpu2yNkINw=
github.com/newrelic/newrelic-telemetry-sdk-go v0.2.1-0.20200116224429-790ff853d12b/go.mod h1:G9MqE/cHGv3Hx3qpYhfuyFUsGx2DpVcGi1iJIqTg+JQ=
github.com/newrelic/newrelic-telemetry-sdk-go v0.3.0 h1:KKeKemCT+woxK9GOI1XxiPui/g59ldJSoU9QtfEjXRs=
github.com/newrelic/newrelic-telemetry-sdk-go v0.3.0/go.mod h1:G9MqE/cHGv3Hx3qpYhfuyFUsGx2DpVcGi1iJIqTg+JQ=
github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U=
github.com/onsi/ginkgo v0.0.0-20170829012221-11459a886d9c/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
Expand Down
10 changes: 0 additions & 10 deletions internal/cmd/scraper/scraper.go
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,6 @@ type Config struct {
BearerTokenFile string `mapstructure:"bearer_token_file"`
InsecureSkipVerify bool `mapstructure:"insecure_skip_verify" default:"false"`
ProcessingRules []integration.ProcessingRule `mapstructure:"transformations"`
Percentiles []float64 `mapstructure:"percentiles"`
DecorateFile bool
EmitterProxy string `mapstructure:"emitter_proxy"`
// Parsed version of `EmitterProxy`
Expand Down Expand Up @@ -80,14 +79,6 @@ func validateConfig(cfg *Config) error {
if cfg.LicenseKey == "" {
return fmt.Errorf(requiredMsg, "license_key")
}
for _, p := range cfg.Percentiles {
if p < 0.0 {
return fmt.Errorf("percentiles must be greater than or equal to 0.0, got %f", p)
}
if p > 100.0 {
return fmt.Errorf("percentiles must be less than or equal to 100.0, got %f", p)
}
}

if cfg.EmitterProxy != "" {
proxyURL, err := url.Parse(cfg.EmitterProxy)
Expand Down Expand Up @@ -244,7 +235,6 @@ func Run(cfg *Config) error {
}

c := integration.TelemetryEmitterConfig{
Percentiles: cfg.Percentiles,
HarvesterOpts: harvesterOpts,
DeltaExpirationAge: cfg.TelemetryEmitterDeltaExpirationAge,
DeltaExpirationCheckInternval: cfg.TelemetryEmitterDeltaExpirationCheckInterval,
Expand Down
27 changes: 0 additions & 27 deletions internal/histogram/bucket.go

This file was deleted.

129 changes: 0 additions & 129 deletions internal/histogram/percentile.go

This file was deleted.

143 changes: 0 additions & 143 deletions internal/histogram/percentile_test.go

This file was deleted.

Loading

0 comments on commit 0250ae2

Please sign in to comment.