Ability to perform runtime dedup using multiple labels for --query.replica-label flag

[sanamana]

Context/Use Case

The querier component accepts --query.replica-label flag. Right now only one label name can be specified and it will be used to perform dedup. We have distributed our data across multiple Prometheus instances and have labeled Prometheus instances with about 3 labels.

When the data is retrieved by thanos we would like to dedup on all 3 labels.

One way to do that is to collapse the 3 labels into a single label and use it for --query.replica-label flag.

Another way is to enhance thanos, to perform dedup on multiple replica labels.

Would like to get feedback on the second approach. The second approach allows us to perform rollups on our custom labels when dedup=false is set in the query.

[brancz]

@squat I remember us having discussed this as well (I thought we opened an issue but I can't find it). Additionally I'd like to see if we could even choose the labels to dedup on at runtime (the use case is thanos receive users that have varying external labels).

[bwplotka]

AC:

• Allow deduplication with multiple labels instead of just one (e.g use repeated replica-label flag)
• Allow specifying deduplication label on run time via extra parameter to query and query_range
• Document changes.

[bwplotka]

cc @krasi-georgiev

[krasi-georgiev]

thanks, looking into it.

[bwplotka]

BTW closing this as it's duplicate of #1174