metricbeat vsphere add field

[adelbot]

For confirmed bugs, please report:

• Version: 7.0.0-alpha1
• Operating System: RedHat 7.4
• Steps to Reproduce:

I'm looking for add network Name in event
like vm.Guest.net[].Network

[amandahla]

@adelbot Would you like this field at "virtualmachine" metric? Collecting network names for every virtual machine.

[adelbot]

On all when it’s is posible De : Amanda H. L. de Andrade [mailto:[email protected]] Envoyé : mardi 21 novembre 2017 17:42 À : elastic/beats <[email protected]> Cc : DELBOT Alain <[email protected]>; Mention <[email protected]> Objet : Re: [elastic/beats] metricbeat vsphere add field (#5646) @adelbot<https://github.com/adelbot> Would you like this field at "virtualmachine" metric? Collecting network names for every virtual machine. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#5646 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AF5FTCPZOGjwz_N_Qd7yMfyhnFOkgj8Vks5s4v1MgaJpZM4Qj0Rq>. Post-scriptum La Poste Ce message est confidentiel. Sous reserve de tout accord conclu par ecrit entre vous et La Poste, son contenu ne represente en aucun cas un engagement de la part de La Poste. Toute publication, utilisation ou diffusion, meme partielle, doit etre autorisee prealablement. Si vous n'etes pas destinataire de ce message, merci d'en avertir immediatement l'expediteur.

[ruflin]

Hi @adelbot

The library that is used to connect to the vSphere API is https://github.com/vmware/govmomi. So if the metrics are available through this library it should be possible to add them. I had a quick look at the code and there are quite a few Network fields. Could you link to the govmomi code which metrics you are referring to? Or feel free to directly open a PR with it.

[amandahla]

Just to share: I gave it a try but found a error then I opened a issue to govmomi. :-(
I believe that must be another ways to get this property, lets see.
vmware/govmomi#933

[amandahla]

@adelbot @ruflin Do you think it would be better to be optional? Like custom fields.

Event example:

Publish event: {
  "@timestamp": "2017-11-23T12:19:49.139Z",
  "@metadata": {
    "beat": "metricbeat",
    "type": "doc",
    "version": "7.0.0-alpha1"
  },
  "vsphere": {
    "virtualmachine": {
      "name": "machine",
      "cpu": {
        "used": {
          "mhz": 0
        }
      },
      "memory": {
        "total": {
          "guest": {
            "bytes": 2147483648
          }
        },
        "free": {
          "guest": {
            "bytes": 2083520512
          }
        },
        "used": {
          "host": {
            "bytes": 851443712
          },
          "guest": {
            "bytes": 63963136
          }
        }
      },
      "custom_fields": {
        "dtCriacaoVM": "2016-11-09 16:23:08",
        "Ambiente": "pro"
      },
      "network_names": {
        "name": "Gerencia Lab"
      },
      "host": "host-19"
    }
  },
  "metricset": {
    "host": "https://vcenter/sdk",
    "rtt": 3050826,
    "module": "vsphere",
    "name": "virtualmachine"
  },
  "beat": {
    "name": "metricbeat",
    "hostname": "host123",
    "version": "7.0.0-alpha1"
  }
}

[ruflin]

As far as I understand, the fields added are:

      "custom_fields": {
        "dtCriacaoVM": "2016-11-09 16:23:08",
        "Ambiente": "pro"
      },
      "network_names": {
        "name": "Gerencia Lab"
      },

2 questions here:

• It says network names, so could this be a list? Or is name a unique key?
• What is in this custom fields? Are these values configured by the user? Or where are these fields coming from?

Sorry for potentially basic question but not really familiar with vsphere.

[adelbot]

Hello,

1. Yes, it could be a list
2. The custom field can be configued by admin or by flow

[amandahla]

@ruflin custom_fields already exists, it's not a new field :-) I use custom_fields as example because he is collected only if the user says so. It's optional, not default. So, I thought that network_names could be the same thing. In config, we could have a "get_network_names" to be set true/false.

[ruflin]

@amandahla Ah, great. Depending on how big the list is normally, I'm ok also to collect it by default (assuming it is like 1-3 entries normally). People that don't like it can filter it out. Like this we don't need to add a config option for it.

As network_names is a list, should it be:

    "network_names": [
        "Gerencia Lab",
        "Bla Lab",
      },

Otherwise we would have multiple name entries overwriting each other?

[amandahla]

@adelbot For metric "host", at least in the environment that I'm testing, we have a lot of "Distributed port group" instead of "Network". For this metric, is ok to collect only network names?

[amandahla]

@ruflin I believe that a list will work fine 👍

[amandahla]

This is a first version. I'll wait for adelbot to answer if is important/interesting to collect the Distributed port group names for host metric.

[ruflin]

@adelbot Have a look at #5646
@amandahla Thanks for taking this over and push it so quickly.

[PauliniMrtns]

And can this custom_fields be generated by metricbeat it self or are they being pulled from VMWare directly?

Thanks

[amandahla]

@PauliniMrtns they are pulled from Vmware
https://pubs.vmware.com/vi3/sdk/ReferenceGuide/vim.CustomFieldsManager.Value.html

[adelbot]

so sorry !!!!
I don't read old comment ;-(
Yes, have a list of Distributed port group names of virtual is perfect