Calculating derivative for telegraf net statistics

[jdowning]

Hi! I'm having trouble using kapacitor with telegraf's network statistics, specifically the bytes_recv and bytes_sent fields. I couldn't really see why until LogNode was added recently:

E! failed to marshal alert data json json: error calling MarshalJSON for type influxql.Result: json: unsupported value: +Inf`

I'm trying to take these two fields, calculate the Mbps rate, and alert on a threshold. My TICK script:

var netdata = stream
    .from().measurement('net')
    .groupBy('host', 'env')

// if-in
netdata
    .window()
      .period(1m)
      .every(1m)
    .derivative('bytes_recv')
        .unit(1s).nonNegative()
        .as('bytes_recv_der')
    .eval(lambda: "bytes_recv_der" / 1024.0, lambda: "in-kbps" / 1024.0 )
        .as('in-kbps', 'net-in-mbps')
        .keep('in-kbps', 'net-in-mbps')
    .log()
    .alert()
        .id('{{ .Name }} // {{ index .Tags "host" }}.{{ index .Tags "env"}}')
        .message('{{ .ID }} is {{ .Level }} net-in: {{ index .Fields "net-in-mbps" }} Mbps')
        .warn(lambda: "net-in-mbps" > 40.0)
        .crit(lambda: "net-in-mbps" > 80.0)

    // post to logfile
        .log('/tmp/alerts.log')

// if-out
netdata
    .window()
      .period(1m)
      .every(1m)
    .derivative('bytes_sent')
        .unit(1s).nonNegative()
        .as('bytes_sent_der')
    .eval(lambda: "bytes_sent_der" / 1024.0, lambda: "out-kbps" / 1024.0 )
        .as('out-kbps', 'net-out-mbps')
        .keep('out-kbps', 'net-out-mbps')
    .log()
    .alert()
        .id('{{ .Name }} // {{ index .Tags "host" }}.{{ index .Tags "env"}}')
        .message('{{ .ID }} is {{ .Level }} net-out: {{ index .Fields "net-out-mbps" }} Mbps')
        .warn(lambda: "net-out-mbps" > 40.0)
        .crit(lambda: "net-out-mbps" > 80.0)

    // post to logfile
        .log('/tmp/alerts.log')

[nathanielc]

@justindowning I think I found the issue. If multiple points arrive at the same time for a group then the derivative divides the points by 0, thus creating the +Inf value. Since Telegraf is not likely sending values at the same time its is probably a grouping issue.

I will fix the +Inf bug, but you can likely solve your issue by changing the groupBy dims:

var netdata = stream
    .from().measurement('net')
    // Try grouping by everything to start with to see if that fixes the issue
    .groupBy(*)

[jdowning]

Wow, I thought I was doing something wrong all this time! I can confirm the groupBy(*) fixes the +Inf output. Thanks for the fast bug fix @nathanielc!