Expose more metrics about segments, search type, ...

[gehel]

Thanks for the great exporter! We are migrating from graphite to prometheus and using this exporter to expose elasticsearch metrics. There are a few metrics we used to collect that don't seem to be exposed:

• elasticsearch.indices.segments.terms_memory_in_bytes
• elasticsearch.indices.segments.index_writer_memory_in_bytes
• elasticsearch.indices.completion.size_in_bytes
• elasticsearch.indices.segments.norms_memory_in_bytes
• elasticsearch.indices.segments.stored_fields_memory_in_bytes
• elasticsearch.indices.segments.doc_values_memory_in_bytes
• elasticsearch.indices.segments.fixed_bit_set_memory_in_bytes
• elasticsearch.indices.search.groups.prefix.query_total
• elasticsearch.indices.search.groups.prefix.query_time_in_millis
• elasticsearch.indices.suggest.total
• elasticsearch.indices.suggest.time_in_millis
• elasticsearch.indices.search.groups.full_text.query_total
• elasticsearch.indices.search.groups.full_text.query_time_in_millis
• elasticsearch.indices.search.groups.more_like.query_total
• elasticsearch.indices.search.groups.more_like.query_time_in_millis

All those metrics are available through the node stat API. Our current implementation based on diamond and is available here.

Any chance you could add those as well?

Thanks!

[gehel]

For reference, this is tracked on our side at https://phabricator.wikimedia.org/T181627

[dominikschulz]

At first sight it looks reasonable to add those metrics but I want to check back with @zwopir

[zwopir]

Yes, looks reasonable. If all supported ES versions expose the metrics, it's just a matter of adding a few struct fields - no fancy logic involved. I'll check it asap. If not we need to find a solution how to handle different json input from the stats endpoint.

[gehel]

Looking at the history of our diamond collector, we changed almost nothing since 2 years ago. I would need to check what version of elasticsearch we were running at that time, but it should be compatible back to at least 2.x, possibly 1.8. We are running this collector on 5.5 at the moment. I have not checked compatibility with ES 6.x yet.

Thanks for looking into this!

[gehel]

Thanks! I'll test this as soon as I find some time...

[dominikschulz]

@gehel Thanks! I know that problem wrt. time ;) I'm sorry this took so long, somehow my PR got lost over Christmas.

[gehel]

After far too long, I finally deployed that new version. All looks good! Thanks for the help!

[dominikschulz]

@gehel Amazing. Thanks for reporting back!