Improve visibility of windows_exporter configuration errors

[thampiotr]

Request

Currently it is possible to configure windows_exporter incorrectly and the Agent will not log any errors or report unhealthy otherwise. For example in static mode:

integrations:
  windows_exporter:
    enabled: true
    service:
      where_clause: "Name LIKE ´%something%´" // <- this is an invalid where clause

the agent won't report any issues to the user. However, the where_clause is not valid query syntax due to use of backticks instead of single quotes.

The underlying windows_exporter has an option to configure logger and the Agent is passing a logger to it.

A failed service query should return an error which is logged here on error level.

However, somehow the log lines do not appear as reported by some users.

Use case

As a user, when I provide invalid service query to windows_exporter, I'd like to receive feedback and error message to help me correct the query.

[ptodev]

I just tried this config file with Agent's Static mode, version 0.37.4 and 0.38.0:

server:
  log_level: debug
metrics:
  wal_directory: C:\Users\paulin_todev\Desktop\grafana-agent-wal
  global:
    scrape_interval: 1m
  configs:
    - name: integrations
integrations:
  windows_exporter:
    enabled: true
    service:
      where_clause: "Name LIKE `%elobase01%`"
  labels:
    gruppe: /test/test_1/versuch/
    testkey: testwert

Both versions produced "invalid query" error-level log lines.

Then I changed this line...

where_clause: "Name LIKE `%elobase01%`"

... to this...

where_clause: "Name LIKE '%elobase01%'"

... and now I now longer see these errors.

The interesting thing is that in 0.37.4, the "invalid query" errors are not formatted in the same was as other Agent log lines - e.g. they have a time label instead of ts:

time="2023-11-27T07:25:04-06:00" level=error msg="failed collecting WMI service metrics:Exception occurred. (Invalid query )" source="service.go:106"
time="2023-11-27T07:25:04-06:00" level=error msg="collector service failed after 0.018560s: Exception occurred. (Invalid query )" source="prometheus.go:185"
ts=2023-11-27T13:25:04.2446155Z caller=logging.go:122 level=debug msg="GET /integrations/windows_exporter/metrics (200) 24.8812ms"
ts=2023-11-27T13:25:06.6753046Z caller=config_watcher.go:106 level=debug agent=prometheus component=cluster msg="reshard timer ticked, scheduling refresh"

In version 0.38.0, they are formatted just like other Agent log lines:

ts=2023-11-27T12:08:04.3650794Z caller=prometheus.go:189 level=debug integration=windows_exporter msg="collector cpu succeeded after 0.000000s."
ts=2023-11-27T12:08:04.3670227Z caller=prometheus.go:189 level=debug integration=windows_exporter msg="collector os succeeded after 0.002932s."
ts=2023-11-27T12:08:04.3845977Z caller=service.go:137 level=error integration=windows_exporter collector=service msg="failed collecting WMI service metrics:" err="Exception occurred. (Invalid query )"
ts=2023-11-27T12:08:04.3856031Z caller=prometheus.go:186 level=error integration=windows_exporter msg="collector service failed after 0.021512s" err="Exception occurred. (Invalid query )"

I think this is due to #5784, which supplies the Agent's logger to the exporter in pkg/integrations/windows_exporter/windows_exporter_windows.go.

I think we can close this issue as resolved. #5784 and #3603 were major improvements to the windows_exporter integration, and I'd encourage any customers which were affected by this or other windows_exporter issues to pugrade to at least Agent 0.38.0.

[thampiotr]

Nice, thanks @ptodev for investigating this. I agree, we can close it.