Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MQE: fix panic in rate or increase over native histograms with changing schema #9518

Merged
merged 3 commits into from
Oct 4, 2024

Conversation

charleskorn
Copy link
Contributor

@charleskorn charleskorn commented Oct 4, 2024

What this PR does

This PR fixes a panic in MQE when rate or increase is used over a range of native histograms that change schema multiple times.

Which issue(s) this PR fixes or relates to

(none)

Checklist

  • Tests updated.
  • [n/a] Documentation added.
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX].
  • [n/a] about-versioning.md updated with experimental features.

@charleskorn charleskorn marked this pull request as ready for review October 4, 2024 05:03
@charleskorn charleskorn requested a review from a team as a code owner October 4, 2024 05:03
Copy link
Contributor

@jhesketh jhesketh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice find. In checking why Prometheus' engine didn't suffer this I think we can optimise where we are copying the schema. Granted it's probably not a frequent operation.

@@ -114,7 +114,7 @@ func histogramRate(isRate bool, step types.RangeVectorStepData, hHead []promql.H
return histogram.ErrHistogramsIncompatibleSchema
}

if p.H.Schema < currentSchema {
if p.H.Schema < delta.Schema {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can avoid doing this so often. If we track the min schema, we only need to copy delta to the min schema once it has finished accumulating points (and /maybe/ on line 108 before the add (I'm not as sure about there, would need to check).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very possibly, but I'd like to keep the scope of this PR as small as possible, so I'll do that in a follow-up PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@charleskorn charleskorn merged commit caa2fb9 into main Oct 4, 2024
35 checks passed
@charleskorn charleskorn deleted the charleskorn/mqe-rate-increase-panic branch October 4, 2024 05:25
@grafanabot
Copy link
Contributor

The backport to r308 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new branch
git switch --create backport-9518-to-r308 origin/r308
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x caa2fb9b60ae460a84323c638f9c4875a2a12797
# Push it to GitHub
git push --set-upstream origin backport-9518-to-r308
git switch main
# Remove the local backport branch
git branch -D backport-9518-to-r308

Then, create a pull request where the base branch is r308 and the compare/head branch is backport-9518-to-r308.

@grafanabot
Copy link
Contributor

The backport to r309 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new branch
git switch --create backport-9518-to-r309 origin/r309
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x caa2fb9b60ae460a84323c638f9c4875a2a12797
# Push it to GitHub
git push --set-upstream origin backport-9518-to-r309
git switch main
# Remove the local backport branch
git branch -D backport-9518-to-r309

Then, create a pull request where the base branch is r309 and the compare/head branch is backport-9518-to-r309.

@grafanabot
Copy link
Contributor

The backport to r310 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new branch
git switch --create backport-9518-to-r310 origin/r310
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x caa2fb9b60ae460a84323c638f9c4875a2a12797
# Push it to GitHub
git push --set-upstream origin backport-9518-to-r310
git switch main
# Remove the local backport branch
git branch -D backport-9518-to-r310

Then, create a pull request where the base branch is r310 and the compare/head branch is backport-9518-to-r310.

charleskorn added a commit that referenced this pull request Oct 4, 2024
…anging schema (#9518)

* MQE: fix panic in `rate` or `increase` over native histograms with changing schema

* Add changelog entry

* Add test for `increase` too

(cherry picked from commit caa2fb9)

# Conflicts:
#	CHANGELOG.md
charleskorn added a commit that referenced this pull request Oct 4, 2024
…anging schema (#9518)

* MQE: fix panic in `rate` or `increase` over native histograms with changing schema

* Add changelog entry

* Add test for `increase` too

(cherry picked from commit caa2fb9)

# Conflicts:
#	CHANGELOG.md
charleskorn added a commit that referenced this pull request Oct 4, 2024
…anging schema (#9518)

* MQE: fix panic in `rate` or `increase` over native histograms with changing schema

* Add changelog entry

* Add test for `increase` too

(cherry picked from commit caa2fb9)

# Conflicts:
#	CHANGELOG.md
charleskorn added a commit that referenced this pull request Oct 4, 2024
…anging schema (#9518) (#9521)

* MQE: fix panic in `rate` or `increase` over native histograms with changing schema

* Add changelog entry

* Add test for `increase` too

(cherry picked from commit caa2fb9)

# Conflicts:
#	CHANGELOG.md
charleskorn added a commit that referenced this pull request Oct 4, 2024
…anging schema (#9518) (#9520)

* MQE: fix panic in `rate` or `increase` over native histograms with changing schema

* Add changelog entry

* Add test for `increase` too

(cherry picked from commit caa2fb9)

# Conflicts:
#	CHANGELOG.md
charleskorn added a commit that referenced this pull request Oct 4, 2024
…anging schema (#9518) (#9522)

* MQE: fix panic in `rate` or `increase` over native histograms with changing schema

* Add changelog entry

* Add test for `increase` too

(cherry picked from commit caa2fb9)

# Conflicts:
#	CHANGELOG.md
@jhesketh jhesketh mentioned this pull request Dec 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants