query: metric type and scrape interval aware deduplication

[yeya24]

Is your proposal related to a problem?

The current deduplication algorithm is not perfect and there are still room to improve.

For example, we have issues like #981 and other deduplication issues related to counter metrics.
Also, the initial penalty value is set to 5000 by default https://github.com/thanos-io/thanos/blob/main/pkg/dedup/iter.go#L278. This value is reasonable, but we might be able to do better.

This is just a rough idea:
Prometheus has targets API and metric metadata API, which should be good enough for us to get the metrics type and the scrape interval of each scrape job.

Querier can maintain a cache and query Prometheus periodically to get this info. Ideally, this info can be set provided using some files as well (assuming metric type and scrape interval are hardly changed so users can provide a list of metrics). Then it can perform better deduplication based on the metric scrape interval and metrics type.

Describe the solution you'd like

(Describe your proposed solution here.)

Describe alternatives you've considered

(Write your answer here.)

[stale]

Hello 👋 Looks like there was no activity on this issue for the last two months.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity in the next two weeks, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command if you wish to be reminded at some point in future.

[stale]

Closing for now as promised, let us know if you need this to be reopened! 🤗

[matej-g]

Looks like this idea is still relevant, re-opening