Expose more series size stats and use in lazy posting #7957
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
yeya24
changed the title
yeya24
force-pushed
the
expose-series-stats1
branch
from
06c8d30 to
447cad7
Compare
GiedriusS
reviewed
Dec 5, 2024
|
Can you please take a look at this PR? @GiedriusS @MichaHoffmann |
yeya24
force-pushed
the
expose-series-stats1
branch
from
4f11043 to
95b8c17
Compare
yeya24
force-pushed
the
expose-series-stats1
branch
from
bb3c830 to
9627fd6
Compare
Signed-off-by: Ben Ye <[email protected]>
Signed-off-by: Ben Ye <[email protected]> resolve conflict Signed-off-by: Ben Ye <[email protected]>
Signed-off-by: Ben Ye <[email protected]>
yeya24
force-pushed
the
expose-series-stats1
branch
from
27c675e to
e3c397b
Compare
Signed-off-by: Ben Ye <[email protected]>
yeya24
force-pushed
the
expose-series-stats1
branch
from
e3c397b to
27d4e07
Compare
Signed-off-by: Ben Ye <[email protected]>
Signed-off-by: Ben Ye <[email protected]>
GiedriusS
reviewed
Jan 9, 2025
Signed-off-by: Ben Ye <[email protected]>
yeya24
force-pushed
the
expose-series-stats1
branch
from
e2c72cf to
df34e85
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #7955
Changes
meta.jsonindex_statssection, includingseries_p90_size,series_p99_size,series_p999_sizeandseries_p9999_size.estimated-series-size-statto configure which stats to use for lazy posting. By default it is max but users can configure their own to p90 to p9999. I didn't call this flaglazy-expanded-posting-estimated-series-size-statbecause this estimated series size might be used by other features in the future for estimation.Note:
For quantile series size calculation, I have tried both
go-tdigest(already used by hedged requests feature) andDataDog/sketches-go. ddsketches-go seems 10s faster thango-tdigestwhen processing a block with 20M series. So I go with ddsketches-go in the implementation for performance.Verification
Existing tests should still pass