[core][wmi] new WMI metric collection core

[yannmh]

Original discussion thread: #1952
Credits to @TheCloudlessSky (https://github.com/TheCloudlessSky) . Thanks a lot !!!

WMI module + CounterType calculators

A lightweight Python WMI module wrapper built on top of pywin32 and
win32com extensions.

Specifications

• Based on top of the pywin32 and win32com third party extensions only
• Compatible with Raw* and Formatted Performance Data classes
  • Dynamically resolve properties' counter types
  • Hold the previous/current Raw samples to compute/format new values*
• Fast and lightweight
  • Avoid queries overhead
  • Cache connections and qualifiers
  • Use wbemFlagForwardOnly flag to improve enumeration/memory performance

__ Raw data formatting relies on the avaibility of the corresponding calculator.
Please refer to checks.lib.wmi.counter_type for more information*

Windows system check speedup

Speedup the Windows system check:

• Drop third party wmi Python package in favor of the local checks.lib.wmi module
• Use PerfRawData WMI Performance classes rather than PerfFormattedData

WMI check speedup

Speedup the generic WMI check:

• Drop third party wmi Python package in favor of the local checks.lib.wmi module

[remh]

Haven't reviewed it yet but really excited about this one!

[TheCloudlessSky]

Awesome job @yannmh ⭐!

1. Were you able to see similar performance improvements?
2. Given that implementors of custom checks will be using this (I already have several WMI classes I'd want to use it with), should it be possible to pass unimplemented counter type calculators to the WMISampler? Ideally, they'd be contributed back to the dd-agent repository. But, I don't think manually maintaining the previous/current sample outside of the WMISampler is a good idea (the value from get_raw). Perhaps we could expose it by inheritance or a dictionary parameter?

Thanks again for moving forward with this!

[yannmh]

@TheCloudlessSky,

Were you able to see similar performance improvements?

I built a custom Datadog Agent to benchmark those changes. It run both (old and new) system and WMI checks and add metrics to measure those checks' run times.

Results 🎺
I made a Datadog screen board to visualize the results.

To sum up:

1. System check takes ~10 times less to run.
2. WMI check takes between ~5 and ~10 time less to run depending on the class data type (RawData vs FormattedData). On my machine that's ~10 seconds less to collect ~2k WMI metrics.
3. Collecting RawData vs FormattedData shows ~2 'gain' factor.

Overall the performance gain is as significant as it was advertised before !:tada: :confetti_ball: :balloon:

Given that implementors of custom checks will be using this (I already have several WMI classes I'd want to use it with), should it be possible to pass unimplemented counter type calculators to the WMISampler? Ideally, they'd be contributed back to the dd-agent repository. But, I don't think manually maintaining the previous/current sample outside of the WMISampler is a good idea (the value from get_raw). Perhaps we could expose it by inheritance or a dictionary parameter?

Thanks for the suggestion. I agree, I think it would be a nice feature to have.
We'll work on it on a separate PR and keep you informed about it !

[olivielpeau]

Looking pretty awesome, let's !

[TheCloudlessSky]

Awesome @yannmh! I'm super excited about these changes landing in 5.6.0! 🎉 !

[irabinovitch]

@TheCloudlessSky Thank you for your excellent work on this refactor. Can you shoot me a note with your contact info via email? (ilan at datadoghq.com). Thanks!