Convert processor supports validation of IPv4/IPv6 addresses

[danhermann]

Adds a new type to the convert processor that validates IPv4/IPv6 addresses. Useful for guaranteeing that a particular field value can be mapped to an Elasticsearch ip field type. This is similar to the Beats processor functionality (described here) in which "the convert processor supports a psuedo-type for IPs. It doesn't really change the data type, but it will fail the "conversion" if the value is not an IP."

E.g.:

PUT _ingest/pipeline/ip_convert
{
  "processors": [
    {
      "convert": {
        "field": "source",
        "target_field": "source_ip",
        "type": "ip",
        "ignore_failure": true
      }
    }
  ]
}

# does not populate source_ip because invalid IP
POST /my-ip-test/_doc?pipeline=ip_convert
{
    "source" : "not_a_valid_ip"
}

# results in the following document (does not fail ingest pipeline because ignore_failure is set to true):
{
    "source" : "not_a_valid_ip"
}

# does populate source_ip because valid IP
POST /my-ip-test/_doc?pipeline=ip_convert
{
    "source" : "192.168.1.2"
}

# results in the following document (note that source_ip is still a string value):
{
    "source" : "192.168.1.2",
    "source_ip" : "192.168.1.2"
}

Resolves #36145

[elasticmachine]

Pinging @elastic/es-core-features (Team:Core/Features)

[danhermann]

@andrewkroh and @geekpete, I would appreciate it if you and/or other interested parties could verify that this works for the use cases you described in #36145.

[danhermann]

@elasticmachine update branch

[andrewkroh]

Thanks @danhermann, this looks like what I expected. One question, since this writes an InetAddress as opposed to a string into the event, will any of the downstream processors like geoip have issues accepting that value type?

[danhermann]

Thanks @danhermann, this looks like what I expected. One question, since this writes an InetAddress as opposed to a string into the event, will any of the downstream processors like geoip have issues accepting that value type?

Actually, it parses the string into an InetAddress just to validate it. If successful, it sets the original string as the value and discards the InetAddress. If unsuccessful, it fails although the failure can be suppressed with the ignore_failure option.

[andrewkroh]

LGTM

[jbaiera]

LGTM!

[danhermann]

Thanks, @jbaiera!

[danhermann]

cc: @elastic/es-ui in case auto-complete needs to be updated to accommodate this new type value.

[alisonelizabeth]

Thanks @danhermann! Opened elastic/kibana#97461 to add support in Kibana.