Expose agg usage in Feature Usage API

[polyfractal]

After the ValuesSource refactor lands, all aggregation usage will flow through the new registry. This gives us an opportunity to easily count usage of the aggs and expose on the Nodes feature usage API.

The API is helpful for single-purpose endpoints, but endpoints like Search have a huge number of disparate actions which are all lumped together. Searches, aggregations, highlighting, suggestions, scrolling, composite, etc etc all show up under the Search endpoint.

There are many times an administrator would like to see more granular data about what's flowing through their Search endpoint, without enabling slow logs and picking through the firehose. Since queries and aggregations are among the more expensive operations, we should provide more granular feedback on these activities. With a relatively small change to the VS registry we can record how often agg builders are parsed and give an administrator a high-level overview of usage.

It would also make opt-in telemetry simpler and less invasive to collect for XPack features.

[elasticmachine]

Pinging @elastic/es-analytics-geo (:Analytics/Aggregations)

[imotov]

We have two ways of implementing this feature: incrementing the usage counter on the parser invocation and by incrementing the counter on ValuesSourceRegistry access.

In case of parser, we can count high level invocations. Basically for each search that contains one aggregation of a certain type we will count it as one use. Unfortunately, on this level we will not have information about value type the aggregation is executed one. So for example if a percentiles was executed we will not know if it was executed on histogram data type or not.

In case of ValuesSourceRegistry access counters, we will now the type, but we will count each shard access as a separate invocation. So, if we execute a single search with a single aggregation on 2 indices with 5 shards each, we will count his as 10 invocation. We can do some tricks like counting it only on shard 0. In this case it will be counted as 2 invocations.

[polyfractal]

I'm tentatively leaning towards the second option. I think it'd be more useful to know the complete list of aggs being used, as well as which field types they are being used against... even if it means they are "over-counted" by doing it per-shard. My feeling is that the absolute numbers don't matter as much as relative numbers?

E.g. as an administrator I think I'd care that an expensive scripted-metric might be running two orders of magnitude more than all the other aggs, but not necessarily care about the exact numbers. So it's all relative and doesn't really rely on precise once-per-query counting.

[giladgal]

I agree between these two the second option seems better.

[imotov]

Thanks! I am going with the second option then.

[imotov]

At the end of the day @not-napoleon found a good place to make a single call per parsing while having values source type in hand.