Cardinality problem on source of type 'fluent'

[zoneadm]

We are running vector 0.22.2 with several sources enabled (fluent, http, syslog, tcp) and are experiencing metrics cardinality problems on our fluent source. We have instances delivering logs via fluent protocol to vector via fluentbit/td-agent-bit. When retrying connections after an idle_timeout or when restarting an instance metrics are produced for the source_host:port pair. That way we up to thousands of metrics for a single host as the source port changed. That leads to a tremendous number and volume of metrics scraped by prometheus.

[...]
vector_processed_bytes_total{component_id="fwdproto",component_kind="source",component_name="fwdproto",component_type="fluent",peer_addr="[10.178.65.49:57252](http://10.178.65.49:57252/)",protocol="tcp"} 1383 1657000317351
vector_processed_bytes_total{component_id="fwdproto",component_kind="source",component_name="fwdproto",component_type="fluent",peer_addr="[10.178.65.49:57270](http://10.178.65.49:57270/)",protocol="tcp"} 1383 1657000317313
vector_processed_bytes_total{component_id="fwdproto",component_kind="source",component_name="fwdproto",component_type="fluent",peer_addr="[10.178.65.49:57344](http://10.178.65.49:57344/)",protocol="tcp"} 1383 1657000317306
vector_processed_bytes_total{component_id="fwdproto",component_kind="source",component_name="fwdproto",component_type="fluent",peer_addr="[10.178.65.49:57368](http://10.178.65.49:57368/)",protocol="tcp"} 1413 1657000317301
vector_processed_bytes_total{component_id="fwdproto",component_kind="source",component_name="fwdproto",component_type="fluent",peer_addr="[10.178.65.49:57418](http://10.178.65.49:57418/)",protocol="tcp"} 1413 1657000317289
[...]

We would like to see metrics for fluentd configurable in that way, that including the source port is either an optional feature or that we can suppress getting metrics on a source_host:source_port basis by a configuration option giving us metrics only for the source_host.

[jszwedko]

Thanks @zoneadm . We plan to do a sweep soon to remove these high cardinality labels for all components. I expect this to be done in time for the release after next, 0.24.0.

[jszwedko]

Rolling this into #15426