filtering out windows services

[iatalay]

Relevant telegraf.conf:

[[inputs.win_services]]

System info:

v1.6.2
windows server 2008 r2

Steps to reproduce:

there are some windows services that telegraf does not have read access.
I want to filter out these services. I used tagdrop option with no success.
Currently it thows error log every interval and it causes log file getting bigger.

Expected behavior:

should filter out services don't want to monitor

Actual behavior:

error log;

2018-08-06T09:17:00Z E! Error in plugin [inputs.win_services]: Could not open service 'DcomLaunch': Access is denied.
2018-08-06T09:17:00Z E! Error in plugin [inputs.win_services]: Could not open service 'MSDTC': Access is denied.

Additional info:

[Include gist of relevant config, logs, etc.]

[danielnelson]

I think we should lower the log level to debug for Access is denied error, as a workaround you could try using the service_names option to whitelist the services you are interested in.

[iatalay]

Setting log level to debug did not affect. It caused to receive extra logs like;
2018-08-07T07:16:01Z D! Output [kafka] wrote batch of 1000 metrics in 1.8321047s

Also, there are hundreds of windows services we have to monitor. Setting service_names on each server is not effective. There should be an exclude list imo.

[danielnelson]

Setting log level to debug did not affect.

Don't set this on your end, I will need to lower the level of this log message so it will not appear in the normal output (and hidden unless --debug is set). This will ensure you no longer see this error.

Also, there are hundreds of windows services we have to monitor. Setting service_names on each server is not effective. There should be an exclude list imo.

I think we should also do this if it is helpful for performance, otherwise the logging ought to be sufficient and avoids the requirement to add new options. Could you enable the internal input plugin and check the amount of time the input is spends gathering metrics? When the internal plugin is enabled it will emit a metric similar to:

internal_gather,input=win_services gather_time_ns=1443110i,metrics_gathered=4i 1533675762000000000

[iatalay]

Here is the output of internal plugin;

date	gather_time_ns	host	input	metrics_gathered
8/8/2018 11:02	302017300	XXX01	win_services	147
8/8/2018 11:03	2028116000	XXX01	win_services	294
8/8/2018 11:04	2058117700	XXX01	win_services	441
8/8/2018 11:05	103005900	XXX01	win_services	588
8/8/2018 11:06	2028116000	XXX01	win_services	735
8/8/2018 11:07	96005500	XXX01	win_services	882
8/8/2018 11:08	94005300	XXX01	win_services	1029
8/8/2018 11:09	91005200	XXX01	win_services	1176
8/8/2018 11:10	94005300	XXX01	win_services	1323
8/8/2018 11:11	1924110000	XXX01	win_services	1470
8/8/2018 11:12	1892108200	XXX01	win_services	1617
8/8/2018 11:13	1904108900	XXX01	win_services	1764
8/8/2018 11:14	2011115000	XXX01	win_services	1911

[danielnelson]

1s-3s to gather the services is pretty slow, it would make sense to try to reduce this and it does seem that we could reduce the number of calls we make with an include/exclude list.

[danielnelson]

I have reduced the log level of the error message for 1.8. We can still add the exclude option but we will need to be mindful not to list services unless required.

[iatalay]

this solved my issue.
thanks