vSphere Plugin: VM whitelist

[khord]

Feature Request

Proposal:

Wondering if there could be a VM whitelist in the vsphere plugin config so that I can just include my vCenter VM for the purposes of a VCSA utilization graph, as well as a smaller group of infrastructure type VMs. In an environment with thousands of VMs, I do not want to be collecting for all of them.

Current behavior:

Can only exclude or include metrics.

Desired behavior:

Option to include only specified VMs

Use case:

Streamline data collection in a large 15,000+ VM environment. Useful for getting stats on a handful of infrastructure VMs...

• without getting SNMP going on them
• that are on completely different isolated subnets
• without flooding the DB with thousands of unwanted VMs.

[voiprodrigo]

What about using output metric filter tagpass?Collect all but only output some, based on vm name tag value.

_{Sent with GitHawk}

[khord]

Hmm didn't know about that and it sounds useful, but I'm also concerned about the strain this could put on the VMware API if this plugin is still gathering data on all 15000 VMs on a continuous interval, with the drop off only happening at Telegraf before it gets shipped to InfluxDB. If you are saying that it would only query the name/tagpass against the API, then I will definitely try that.

[voiprodrigo]

Correct, this only reduces what you send to InfluxDB, not what you collect. Just a mitigation for not filling the DB.

Only the input plugin can provide the feature of only querying data for some VM’s (if the remote API allows it)

[prydin]

Filtering on resource names is something we're considering for a later release. For now, the best way to reduce the amount of data emitted is to filter the metrics as @voiprodrigo suggested.

[prydin]

Also, the metric filters DO affect what gets collected. It's essentially what builds the query against the SDK.

As for performance, vm and host queries are relatively cheap and shouldn't bog down your vCenter too much, since they're just scraped from ESXi memory. It's the other stuff, like clusters and datastores that hits the vCenter database and can cause heavy load.

Another tip is to turn off instance collection (e.g. vm_instances = false). This avoids collecting data for every CPU, disk, etc. This is especially true for datastores.

[danielnelson]

If you use tagpass or any of the other metric filtering options, it is best to apply them to the input plugin if possible. This gives a slight performance improvement as they are removed from the Telegraf pipeline earlier.

[prydin]

Starting work on this.

[prydin]

This has been implemented over a year ago. Can we close this? @danielnelson