vSphere: Use inventory paths to select resources to monitor

[prydin]

Feature Request

Use inventory paths to select resources to monitor

Proposal:

Some users of the vSphere plugin has suggested there should be a mechanism for selecting the resources to monitor in addition to the existing mechanism for selecting metrics. For example, see issue #4790.

We are proposing a mechanism based on inventory paths. In vCenter, resources are arranged in a tree structure as follows:

<root>
+-DC0 # Virtual datacenter
   +-datastore # Datastore folder (created by system)
   | +-Datastore1
   +-host # Host folder (created by system)
   | +-Cluster1
   | | +-Host1
   | | | +-VM1
   | | | +-VM2
   | | | +-hadoop1
   | +-Host2 # Dummy cluster created for non-clustered host
   | | +-Host2
   | | | +-VM3
   | | | +-VM4
   +-vm # VM folder (created by system)
   | +-VM1
   | +-VM2
   | +-Folder1
   | | +-hadoop1
   | | +-NestedFolder1
   | | | +-VM3
   | | | +-VM4

Using familiar UNIX-style paths, one could select e.g. VM2 with the path /DC0/vm/VM2.

Often, we want to select a group of resource, such as all the VMs in a folder. We could use the path /DC0/vm/Folder1/* for that.

Another possibility is to select objects using a partial name, such as /DC0/vm/Folder1/hadoop* yielding all vms in Folder1 with a name starting with "hadoop".

Finally, due to the arbitrary nesting of the folder structure, we need a "recursive wildcard" for traversing multiple folders. We use the "**" symbol for that. If we want to look for a VM with a name starting with "hadoop" in any folder, we could use the following path: /DC0/vm/**/hadoop*

Multiple paths to VMs

As we can see from the example tree above, VMs appear both in its on folder under the datacenter, as well as under the hosts. This is useful when you like to select VMs on a specific host. For example, /DC0/host/Cluster1/Host1/hadoop* selects all VMs with a name starting with "hadoop" that are running on Host1.

We can extend this to looking at a cluster level: /DC0/host/Cluster1/*/hadoop*. This selects any VM matching "hadoop*" on any host in Cluster1.

Current behavior:

All resources under a vCenter are collected and there's no way of selecting them.

Desired behavior:

Introduce the config variables host_include, vm_include, cluster_include, datacenter_include and datastore_include. Each variable takes an array of strings that represent paths as described above. The plugin will select the union of the matches of all paths in the array per resource type.

Use case:

1. Certain parts of the resource tree may be confidential and we don't want to publish metrics for them.
2. We want to reduce the load on the system by reducing the dataset.
3. We want to reduce the storage utilization and cost at the target database.

A working prototype is already implemented here: https://github.com/prydin/telegraf/tree/prydin-issue-4790

[danielnelson]

Sounds good but glancing over the prototype I highly recommend trying to find a way to avoid reflect, possibly using github.com/gobwas/glob to simplify the matching and avoiding package level variables. Maybe some of it would make sense to add to govmomi itself?

[prydin]

I wish I had a way of avoiding reflect here, but I can't think of one. This is a classic type coercion pattern. The underlying inventory can hold many different type of objects and Go's lack of object polymorphism makes it impossible to treat them as a common superclass. So what I do (and a lot of developers of database adapters etc) is to send in a typed array and let the Find() function coerce whatever it found into the typed array you supplied.

The idea is to use it like this:

vms := []mo.VirtualMachine
finder.Find(ctx, "VirtualMachine", "/DC0/vm/*", &vms)

This is the only way I know to avoid nasty type casts on the caller side and the only way I know how to implement it is using reflect.

As for making this part of govmomi: Yes, that might be an option in the future. Something similar exists, but that implementation would have required me to first ask for a bunch of IDs, then look up the actual object for every one of them, increasing the number of REST calls made by orders of magnitude. So I completely rewrote it in a way that optimized it into only a handful of REST calls. This may be too specialized to roll into govmomi. I'd have to discuss that with @dougm who owns Govmomi.

[prydin]

Merged and resolved.