feat(prom): expose cluster id in identity

[johanneswuerbach]

Proposed Changes

Expose the persistent cluster id as label under rabbitmq_identity_info to reliably detect how many nodes are part of a cluster. Unlike the cluster_name, which can be the same for across multiple clusters, the persistent cluster id is unique per cluster and allows to reliably detect split-brain scenarios.

Types of Changes

What types of changes does your code introduce to this project?
Put an x in the boxes that apply

• Bug fix (non-breaking change which fixes issue #NNNN)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause an observable behavior change in existing systems)
• Documentation improvements (corrections, new content, etc)
• Cosmetic change (whitespace, formatting, etc)
• Build system and/or CI

Checklist

Put an x in the boxes that apply.
You can also fill these out after creating the PR.
If you're unsure about any of them, don't hesitate to ask on the mailing list.
We're here to help!
This is simply a reminder of what we are going to look for before merging your code.

• I have read the CONTRIBUTING.md document
• I have signed the CA (see https://cla.pivotal.io/sign/rabbitmq)
• I have added tests that prove my fix is effective or that my feature works
• All tests pass locally with my changes
• If relevant, I have added necessary documentation to https://github.com/rabbitmq/rabbitmq-website
• If relevant, I have added this change to the first version(s) in release-notes that I expect to introduce it

Further Comments

If this is a relatively large or complex change, kick off the discussion by explaining why you chose the solution you did and what alternatives you considered, etc.

[michaelklishin]

That ID was meant to only be used internally. I don't mind it being exposed too much as it is the only non-volatile value of this kind.

[johanneswuerbach]

We recently suffered a case where a 3 node cluster split into 3 single node clusters and there seems to be currently no easy way to alert on this using prometheus.

I'm aware off #2508, but the having the cluster id exposed would have been a really good indicator for us that we are in a broken state. In our case each node had a different one despite having the same configured cluster name.

[michaelklishin]

Many team members see no issue in adopting this but I'd like to improve field naming before we do.

[gerhard]

This is a good one, thank you for contributing 🚀