-
Notifications
You must be signed in to change notification settings - Fork 530
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Include distributor queue length in tempo-mixin #2623
Include distributor queue length in tempo-mixin #2623
Conversation
"uid": "cortex-ops-01" | ||
}, | ||
"editorMode": "code", | ||
"expr": "sum(tempo_distributor_queue_length{cluster=~\"$cluster\", namespace=~\"$namespace\", job=~\".*/distributor\"}) by (pod)\n", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should aggregate by name
as well. There can be multiple queues in distributors.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking again, I think you're right, but what about aggregate by name
only? Then we get a sum of the length across all instances, which might clarify a little.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's about preference. Either sound good to me. I'll leave to you to decide.
5cc3463
to
3303831
Compare
There are some errors in the mixin files, I think
This is a great feature to get, thank you! |
* Update `make docs` procedure (#3026) * Update `make docs` procedure * Trigger CI --------- Co-authored-by: grafanabot <[email protected]> Co-authored-by: Jack Baldry <[email protected]> * Fix: pass-through to runtime overrides for FilterPolicies and TargetInfoExcludedDimensions (#3012) * fix: handle case when userconfigurableoverrides are enabled but empty * Add CHANGELOG.md * fix panic in test * fix GetTargetInfoExcludedDimens & GetFilterPolicies * update changelog * Include distributor queue length in tempo-mixin (#2623) * Include distributor queue length in tempo-mixin * Rebuild * Compile jsonnet * Compile jsonnet * Use C jsonnet to format mixin --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: grafanabot <[email protected]> Co-authored-by: Jack Baldry <[email protected]> Co-authored-by: Suraj Nath <[email protected]> Co-authored-by: Zach Leslie <[email protected]>
What this PR does:
Include the distributor queue length in the write path row of the tempo operational dashboard.
Checklist
CHANGELOG.md
updated - the order of entries should be[CHANGE]
,[FEATURE]
,[ENHANCEMENT]
,[BUGFIX]