Optionally disable tag parsing

[glightfoot]

This adds globals to the line package to allow disabling of individual tag parsing formats and unit tests for the line package. I don't love using package globals for this, but it does maintain backwards compatibility with the package, and is cleaner than adding four more parameters to the lineToEvents function. Open to other suggestions. Fixes #324

Before:

$ go test -bench=BenchmarkLine.
goos: darwin
goarch: amd64
pkg: github.com/prometheus/statsd_exporter
BenchmarkLineToEventsMixed1-12            142213              9420 ns/op            4778 B/op         91 allocs/op
BenchmarkLineToEventsMixed5-12             28290             40648 ns/op           23892 B/op        455 allocs/op
BenchmarkLineToEventsMixed50-12             2908            426202 ns/op          238924 B/op       4550 allocs/op
BenchmarkLineFormats/dogStatsd-12        1870549               623 ns/op             464 B/op          6 allocs/op
BenchmarkLineFormats/invalidDogStatsd-12                 1575892               710 ns/op             496 B/op          8 allocs/op
BenchmarkLineFormats/signalFx-12                         1847803               675 ns/op             496 B/op          8 allocs/op
BenchmarkLineFormats/invalidSignalFx-12                  1661490               708 ns/op             448 B/op         11 allocs/op
BenchmarkLineFormats/influxDb-12                         1805835               614 ns/op             464 B/op          7 allocs/op
BenchmarkLineFormats/invalidInfluxDb-12                  1348052               890 ns/op             800 B/op         13 allocs/op
BenchmarkLineFormats/statsd-12                           3096838               394 ns/op             176 B/op          6 allocs/op
BenchmarkLineFormats/invalidStatsd-12                    1619024               748 ns/op             432 B/op         10 allocs/op
PASS
ok      github.com/prometheus/statsd_exporter   20.487s

After:

$ go test -bench=BenchmarkLine.
goos: darwin
goarch: amd64
pkg: github.com/prometheus/statsd_exporter
BenchmarkLineToEventsMixed1-12            131325              8444 ns/op            4778 B/op         91 allocs/op
BenchmarkLineToEventsMixed5-12             27319             40279 ns/op           23892 B/op        455 allocs/op
BenchmarkLineToEventsMixed50-12             2986            401224 ns/op          238925 B/op       4550 allocs/op
BenchmarkLineFormats/invalidStatsd-12            1604922               745 ns/op             432 B/op         10 allocs/op
BenchmarkLineFormats/dogStatsd-12                1986829               595 ns/op             464 B/op          6 allocs/op
BenchmarkLineFormats/invalidDogStatsd-12         1713426               718 ns/op             496 B/op          8 allocs/op
BenchmarkLineFormats/signalFx-12                 1893074               636 ns/op             496 B/op          8 allocs/op
BenchmarkLineFormats/invalidSignalFx-12          1715712               712 ns/op             448 B/op         11 allocs/op
BenchmarkLineFormats/influxDb-12                 1927936               621 ns/op             464 B/op          7 allocs/op
BenchmarkLineFormats/invalidInfluxDb-12          1328978               888 ns/op             800 B/op         13 allocs/op
BenchmarkLineFormats/statsd-12                   3057933               431 ns/op             176 B/op          6 allocs/op
PASS
ok      github.com/prometheus/statsd_exporter   19.469s

Benchstat:

$ benchstat old.out new.out
name                             old time/op    new time/op    delta
LineToEventsMixed1-12              8.67µs ±11%    8.26µs ± 3%     ~     (p=0.421 n=5+5)
LineToEventsMixed5-12              41.5µs ± 2%    40.5µs ± 3%     ~     (p=0.056 n=5+5)
LineToEventsMixed50-12              432µs ± 5%     414µs ± 4%     ~     (p=0.056 n=5+5)
LineFormats/invalidInfluxDb-12      966ns ± 8%     885ns ± 1%   -8.43%  (p=0.016 n=5+5)
LineFormats/statsd-12               429ns ± 7%     393ns ± 1%   -8.48%  (p=0.008 n=5+5)
LineFormats/invalidStatsd-12        756ns ± 3%     777ns ± 4%     ~     (p=0.310 n=5+5)
LineFormats/dogStatsd-12            608ns ± 3%     598ns ± 2%     ~     (p=0.246 n=5+5)
LineFormats/invalidDogStatsd-12     720ns ± 3%     745ns ± 4%     ~     (p=0.095 n=5+5)
LineFormats/signalFx-12             690ns ± 1%     689ns ± 5%     ~     (p=0.500 n=5+5)
LineFormats/invalidSignalFx-12      765ns ± 1%     794ns ±20%     ~     (p=0.690 n=5+5)
LineFormats/influxDb-12             723ns ± 5%     633ns ± 4%  -12.43%  (p=0.008 n=5+5)

name                             old alloc/op   new alloc/op   delta
LineToEventsMixed1-12              4.78kB ± 0%    4.78kB ± 0%     ~     (all equal)
LineToEventsMixed5-12              23.9kB ± 0%    23.9kB ± 0%     ~     (all equal)
LineToEventsMixed50-12              239kB ± 0%     239kB ± 0%     ~     (p=0.071 n=5+5)
LineFormats/invalidInfluxDb-12       800B ± 0%      800B ± 0%     ~     (all equal)
LineFormats/statsd-12                176B ± 0%      176B ± 0%     ~     (all equal)
LineFormats/invalidStatsd-12         432B ± 0%      432B ± 0%     ~     (all equal)
LineFormats/dogStatsd-12             464B ± 0%      464B ± 0%     ~     (all equal)
LineFormats/invalidDogStatsd-12      496B ± 0%      496B ± 0%     ~     (all equal)
LineFormats/signalFx-12              496B ± 0%      496B ± 0%     ~     (all equal)
LineFormats/invalidSignalFx-12       448B ± 0%      448B ± 0%     ~     (all equal)
LineFormats/influxDb-12              464B ± 0%      464B ± 0%     ~     (all equal)

name                             old allocs/op  new allocs/op  delta
LineToEventsMixed1-12                91.0 ± 0%      91.0 ± 0%     ~     (all equal)
LineToEventsMixed5-12                 455 ± 0%       455 ± 0%     ~     (all equal)
LineToEventsMixed50-12              4.55k ± 0%     4.55k ± 0%     ~     (all equal)
LineFormats/invalidInfluxDb-12       13.0 ± 0%      13.0 ± 0%     ~     (all equal)
LineFormats/statsd-12                6.00 ± 0%      6.00 ± 0%     ~     (all equal)
LineFormats/invalidStatsd-12         10.0 ± 0%      10.0 ± 0%     ~     (all equal)
LineFormats/dogStatsd-12             6.00 ± 0%      6.00 ± 0%     ~     (all equal)
LineFormats/invalidDogStatsd-12      8.00 ± 0%      8.00 ± 0%     ~     (all equal)
LineFormats/signalFx-12              8.00 ± 0%      8.00 ± 0%     ~     (all equal)
LineFormats/invalidSignalFx-12       11.0 ± 0%      11.0 ± 0%     ~     (all equal)
LineFormats/influxDb-12              7.00 ± 0%      7.00 ± 0%     ~     (all equal)

[glightfoot]

@matthiasr

[matthiasr]

Thank you (and sorry for the delay in review, work has been busy)! I am uneasy about globals that change behavior. What do you think about a "parser" struct with these that we can then use? Something like

import "github.com/prometheus/statsd_exporter/pkg/line"
p := line.NewParser(*dogstatsdTagsEnabled,*influxdbTagsEnabled,*libratoTagsEnabled,*signalFXTagsEnabled)
…
event := p.Parse(line)

Or, to avoid the "hmm which order do the booleans go" ambiguity, a builder pattern?

p := line.NewParser()
if !*dogstatsdTagsEnabled {
  p.DisableDogstatsdTags()
}

or

p := line.NewParser()
p.EnableDogstatsdTags(*dogstatsdTagsEnabled)

I don't know for sure what the best "Go style" is.

[glightfoot]

Yeah I had a feeling, but was lazy :-) I've changed it to use a struct for line parsing with option functions to enable tag parsing

[glightfoot]

Benchstat:

$ benchstat old.out new.out
name                             old time/op    new time/op    delta
LineToEventsMixed1-12              8.67µs ±11%    8.50µs ± 3%     ~     (p=0.841 n=5+5)
LineToEventsMixed5-12              41.5µs ± 2%    41.6µs ± 3%     ~     (p=0.841 n=5+5)
LineToEventsMixed50-12              432µs ± 5%     422µs ± 2%     ~     (p=0.222 n=5+5)
LineFormats/invalidInfluxDb-12      966ns ± 8%    1046ns ± 3%   +8.24%  (p=0.032 n=5+5)
LineFormats/statsd-12               429ns ± 7%     427ns ± 4%     ~     (p=0.952 n=5+5)
LineFormats/invalidStatsd-12        756ns ± 3%     871ns ± 8%  +15.19%  (p=0.008 n=5+5)
LineFormats/dogStatsd-12            608ns ± 3%     763ns ± 4%  +25.39%  (p=0.008 n=5+5)
LineFormats/invalidDogStatsd-12     720ns ± 3%     745ns ± 6%     ~     (p=0.103 n=5+5)
LineFormats/signalFx-12             690ns ± 1%     682ns ± 3%     ~     (p=0.183 n=5+5)
LineFormats/invalidSignalFx-12      765ns ± 1%     810ns ±17%     ~     (p=0.151 n=5+5)
LineFormats/influxDb-12             723ns ± 5%     698ns ± 7%     ~     (p=0.310 n=5+5)

name                             old alloc/op   new alloc/op   delta
LineToEventsMixed1-12              4.78kB ± 0%    4.78kB ± 0%     ~     (all equal)
LineToEventsMixed5-12              23.9kB ± 0%    23.9kB ± 0%     ~     (all equal)
LineToEventsMixed50-12              239kB ± 0%     239kB ± 0%   -0.00%  (p=0.016 n=5+5)
LineFormats/invalidInfluxDb-12       800B ± 0%      800B ± 0%     ~     (all equal)
LineFormats/statsd-12                176B ± 0%      176B ± 0%     ~     (all equal)
LineFormats/invalidStatsd-12         432B ± 0%      432B ± 0%     ~     (all equal)
LineFormats/dogStatsd-12             464B ± 0%      464B ± 0%     ~     (all equal)
LineFormats/invalidDogStatsd-12      496B ± 0%      496B ± 0%     ~     (all equal)
LineFormats/signalFx-12              496B ± 0%      496B ± 0%     ~     (all equal)
LineFormats/invalidSignalFx-12       448B ± 0%      448B ± 0%     ~     (all equal)
LineFormats/influxDb-12              464B ± 0%      464B ± 0%     ~     (all equal)

name                             old allocs/op  new allocs/op  delta
LineToEventsMixed1-12                91.0 ± 0%      91.0 ± 0%     ~     (all equal)
LineToEventsMixed5-12                 455 ± 0%       455 ± 0%     ~     (all equal)
LineToEventsMixed50-12              4.55k ± 0%     4.55k ± 0%     ~     (all equal)
LineFormats/invalidInfluxDb-12       13.0 ± 0%      13.0 ± 0%     ~     (all equal)
LineFormats/statsd-12                6.00 ± 0%      6.00 ± 0%     ~     (all equal)
LineFormats/invalidStatsd-12         10.0 ± 0%      10.0 ± 0%     ~     (all equal)
LineFormats/dogStatsd-12             6.00 ± 0%      6.00 ± 0%     ~     (all equal)
LineFormats/invalidDogStatsd-12      8.00 ± 0%      8.00 ± 0%     ~     (all equal)
LineFormats/signalFx-12              8.00 ± 0%      8.00 ± 0%     ~     (all equal)
LineFormats/invalidSignalFx-12       11.0 ± 0%      11.0 ± 0%     ~     (all equal)
LineFormats/influxDb-12              7.00 ± 0%      7.00 ± 0%     ~     (all equal)

[matthiasr]

This looks great, thank you very much and sorry for the delay in reviewing!

[glightfoot]

No worries! thanks for the help again