Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revert upgrade to google.golang.org/grpc v1.66.2 #9811

Merged
merged 3 commits into from
Nov 4, 2024

Conversation

duricanikolic
Copy link
Contributor

@duricanikolic duricanikolic commented Nov 4, 2024

What this PR does

We have discovered that upgrading Mimir to google.golang.org/grpc v1.66.2 caused significant memory issues in Mimir's ruler-querier. The following screenshot shows ruler-querier's memory usage before and after a rollout to a Mimir version containing google.golang.org/grpc v1.66.2.

Screenshot 2024-10-31 at 15-04-08 Mimir _ Remote ruler reads resources - Mimir - Dashboards - Grafana (1)

Therefore, we decided to revert the following PRs:

Which issue(s) this PR fixes or relates to

Fixes #

Checklist

  • Tests updated.
  • Documentation added.
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX].
  • about-versioning.md updated with experimental features.

@duricanikolic duricanikolic requested review from stevesg, grafanabot and a team as code owners November 4, 2024 06:34
@duricanikolic duricanikolic self-assigned this Nov 4, 2024
Copy link
Contributor

@aknuds1 aknuds1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks

@duricanikolic duricanikolic merged commit 8e90ec1 into main Nov 4, 2024
29 checks passed
@duricanikolic duricanikolic deleted the yuri/revert-grpc-upgrade branch November 4, 2024 08:58
@aknuds1 aknuds1 mentioned this pull request Nov 4, 2024
4 tasks
aknuds1 added a commit that referenced this pull request Nov 4, 2024
* [release-2.14] fix(deps): update github.com/thanos-io/objstore digest to f90c89a (main) (#9625)

* Update github.com/thanos-io/objstore digest to f90c89a (#9534)

Signed-off-by: Arve Knudsen <[email protected]>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
(cherry picked from commit 3c97a61)

* update changelog

Signed-off-by: Vladimir Varankin <[email protected]>

---------

Signed-off-by: Vladimir Varankin <[email protected]>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Vladimir Varankin <[email protected]>

* 2.14.1 Prepare patch release (#9796)

* bump patch version

Signed-off-by: Vladimir Varankin <[email protected]>

* rebuild assets

Signed-off-by: Vladimir Varankin <[email protected]>

---------

Signed-off-by: Vladimir Varankin <[email protected]>

* MQE: Fix handling of string results (#9803)

This would previously panic after the query was closed

* chore(deps): update grafana/mimirtool docker tag to v2.14.1 (#9806)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Revert upgrade to google.golang.org/grpc v1.66.2 (#9811)

* Revert "Don't free buffers after reading query stream (#9721)"

This reverts commit f7b6017.

* Revert: Upgrade to google.golang.org/grpc v1.66.2 / modify certain protobuf messages to retain their unmarshaling buffer (#9401)

Signed-off-by: Yuri Nikolic <[email protected]>

* Revert "Distributor.queryIngesterStream: Free gRPC buffers upon error (#9756)"

This reverts commit eda1a4b.

---------

Signed-off-by: Yuri Nikolic <[email protected]>

* Improve lock contention affecting read and write latencies during TSDB head compaction (cherry-pick Prometheus PR 15242) (#9822)

* Cherry-pick Prometheus PR 15242

Signed-off-by: Marco Pracucci <[email protected]>

* Added CHANGELOG entry

Signed-off-by: Marco Pracucci <[email protected]>

* Updated CHANGELOG

Signed-off-by: Marco Pracucci <[email protected]>

---------

Signed-off-by: Marco Pracucci <[email protected]>

---------

Signed-off-by: Vladimir Varankin <[email protected]>
Signed-off-by: Yuri Nikolic <[email protected]>
Signed-off-by: Marco Pracucci <[email protected]>
Co-authored-by: Grot (@grafanabot) <[email protected]>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Vladimir Varankin <[email protected]>
Co-authored-by: Joshua Hesketh <[email protected]>
Co-authored-by: Đurica Yuri Nikolić <[email protected]>
Co-authored-by: Marco Pracucci <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants