[Filebeat] Netflow input - add input metrics

[andrewkroh]

Describe the enhancement:

Expose metrics from each instance of the netflow input. Users should be able to query the HTTP endpoint (e.g. curl 'http://localhost:5066/inputs/?type=netflow&pretty') and get back metrics from each instance of the netflow input.

This input exposes some global metrics now, but these are summed across all instances of the input.

beats/x-pack/filebeat/input/netflow/input.go

Lines 33 to 35 in d4d2ddc

	numPackets = monitoring.NewUint(nil, "filebeat.input.netflow.packets.received")
	numDropped = monitoring.NewUint(nil, "filebeat.input.netflow.packets.dropped")
	numFlows = monitoring.NewUint(nil, "filebeat.input.netflow.flows")

In my opinion the same metrics from the UDP input should exist for the Netflow input plus some netflow specific metrics like

• discarded_events_total - The number of events dropped by this input.
• decode_errors_total - The total number of decode errors observed while decoding data in an input.
• open_connections - Active number of netflow sessions (gauge). UDP doesn't have "connections" but as a metric I feel the meaning could be applied consistently in other inputs.

Describe a specific use case for the enhancement or feature:

The feature would be used for monitoring and troubleshooting purposes.

[pkoutsovasilis]

Hi @andrewkroh thank you for the initial investigation. From a quick look having metrics per netflow instance makes a lot of sense and I believe it will indeed be extra helpful in monitoring but especially in troubleshooting purposes.

Now some quick questions:

In my opinion the same metrics from the UDP input should exist for the Netflow input

@efd6 has already done the heavy-lifting for us in udp input (thank you 🙂) but should we just duplicate his code for the netflow input or try to make both udp and netflow use the same code? Although it may sound weird, I lean towards the first option (duplication) as these two are different inputs and are susceptible to different metricsets even though currently they significantly overlap

open_connections - Active number of netflow sessions (gauge). UDP doesn't have "connections" but as a metric I feel the meaning could be applied consistently in other inputs.

Now if I don't miss anything netflow sessions are present in v9 and ipfx which are created by SessionMap and they are deemed as expired (inactive?!) by an interval-based loop. So to capture the open_connections in the context of netflow I assume we would want a gauge metric that increases when a session is created and decreased when a session is cleaned from the SessionMap?

[andrewkroh]

I assume we would want a gauge metric that increases when a session is created and decreased when a session is cleaned from the SessionMap?

I think that would be be good for the value of open_connections.

should we just duplicate his code for the netflow input or try to make both udp and netflow use the same code?

I only part that I would caution against duplicating is the procNetUDP function. IIRC it has taken several bug fixes to cover all the edge cases we have encountered. I think it would be worthwhile to share this code so that it remains in sync if there are more fixes.

[pkoutsovasilis]

@andrewkroh @efd6 (since you are the original author of tcp and udp metrics 🙂) I have pushed this commit e97e121 that essentially moves the UDP and the TCP metrics (this last one was necessary to maintain the procnet package as internal) to the inputsource package. Thus each input can instantiate and utilise the same UDP and TCP metrics according to it's need. When you get the chance, have the look and tell me if this seems okay with you two as I want to make sure that adding metrics to netflow won't result in breaking other inputs or making the code less flexible

[efd6]

Suggest rather putting them both in internal as either two packages, so that you have e.g. .../internal/tcpmetrics.New, or one package that can subsume procnet, e.g. .../internal/netmetrics.NewTCP. The only reason procnet exists is to be shared between the two, so if there can be only one, this need goes away and the code locality improves.