Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

mimir-tool: analyse prometheus: ux: refactor / resiliency / concurrency #3062

Closed
Dentrax opened this issue Sep 27, 2022 · 1 comment · Fixed by #3349
Closed

mimir-tool: analyse prometheus: ux: refactor / resiliency / concurrency #3062

Dentrax opened this issue Sep 27, 2022 · 1 comment · Fixed by #3349
Labels
hacktoberfest

Comments

@Dentrax
Copy link
Contributor

Dentrax commented Sep 27, 2022

analyse_prometheus.go is not cleanly written and contains lots of duplicate logics. Also it does not have any resiliency and concurrency mechanism if we run against very large Prometheus instance. (it takes a few hours or longer). Here are my two cents:

  • introduce semgroup library for rate-limited concurrency (while traversing metrics x 2 times)
  • introduce backoff library for retrying v1api.Query()
  • use sync.Map and sync.RWMutex (for concurrency safety)
  • move duplicate logics to basic functions (clean code practices)

cc @anilmisirlioglu
@dimitarvdimitrov
Copy link
Contributor

FWIW I think it makes sense. Some notes on your suggestions:

  • introduce semgroup library for rate-limited concurrency (while traversing metrics x 2 times)

we already use dskit's concurrency package to do that, see the ruler

  • introduce backoff library for retrying v1api.Query()

we already use dskit's backoff - example

@anilmisirlioglu anilmisirlioglu mentioned this issue Nov 1, 2022
Merged
3 tasks
anilmisirlioglu added a commit to anilmisirlioglu/mimir that referenced this issue Nov 8, 2022
@anilmisirlioglu
refactor: add resiliency & concurrency and optimize code
b4ce498 
Fixes grafana#3062

Signed-off-by: Anıl Mısırlıoğlu <[email protected]>
Co-authered-by: Furkan Türkal <[email protected]>
Co-authered-by: Ayşe Sert <[email protected]>
Logiraptor pushed a commit that referenced this issue Nov 9, 2022
@anilmisirlioglu
refactor: add resiliency & concurrency and optimize code (#3349)
cbcc350 
Fixes #3062

Signed-off-by: Anıl Mısırlıoğlu <[email protected]>
Co-authered-by: Furkan Türkal <[email protected]>
Co-authered-by: Ayşe Sert <[email protected]>

Signed-off-by: Anıl Mısırlıoğlu <[email protected]>
masonmei pushed a commit to udmire/mimir that referenced this issue Dec 16, 2022
@anilmisirlioglu
refactor: add resiliency & concurrency and optimize code (grafana#3349)
bcf746f 
Fixes grafana#3062

Signed-off-by: Anıl Mısırlıoğlu <[email protected]>
Co-authered-by: Furkan Türkal <[email protected]>
Co-authered-by: Ayşe Sert <[email protected]>

Signed-off-by: Anıl Mısırlıoğlu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
hacktoberfest
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

refactor: mimirtool analyze prometheus: add concurrency and resiliency anilmisirlioglu/mimir
3 participants
@pracucci @Dentrax @dimitarvdimitrov