Better max_querier_bytes_read limit error messages #8916
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
There was a problem hiding this comment.
Looks great! Once we start using errata then we can link to the specific settings to be adjusted, and add explanations about what sharded queries are, etc. It's a bit difficult to fit that into our current error handling.
For now though this LGTM, i've left a couple small comments
pkg/querier/queryrange/limits.go
Outdated
Comment on lines
42
to
43
| limErrQuerierTooManyBytesShardableTmpl = "not shardable query too large to execute on a single querier: (query: %s, limit: %s). Consider adding more specific stream selectors or reduce the time range of the query" | ||
| limErrQuerierTooManyBytesUnshardableTmpl = "shard query is too big to execute on a single querier: (query: %s, limit: %s). Consider adding more specific stream selectors or reduce the time range of the query" |
There was a problem hiding this comment.
I think these two might be mixed up, based on the name of the const
There was a problem hiding this comment.
I agree but I think it's the most representative name. What about limErrQuerierTooManyBytesUNshardableTmpl, note the capital N
There was a problem hiding this comment.
What i mean is:
UnshardableTmpl => shard query is too big
ShardableTmpl => not shardable query too large
That seems backwards to me?
There was a problem hiding this comment.
Gotcha. As Sandeep pointed out. Those were the other way around. Fixed now.
pkg/querier/queryrange/limits.go
Outdated
Comment on lines
42
to
43
| limErrQuerierTooManyBytesShardableTmpl = "not shardable query too large to execute on a single querier: (query: %s, limit: %s). Consider adding more specific stream selectors or reduce the time range of the query" | ||
| limErrQuerierTooManyBytesUnshardableTmpl = "shard query is too big to execute on a single querier: (query: %s, limit: %s). Consider adding more specific stream selectors or reduce the time range of the query" |
There was a problem hiding this comment.
I feel error messages are not aligning with the names of the variables
There was a problem hiding this comment.
Yes! Good catch.
pkg/querier/queryrange/limits.go
Outdated
| requiredLabelsErrTmpl = "stream selector is missing required matchers [%s], labels present in the query were [%s]" | ||
| limErrQueryTooManyBytesTmpl = "the query would read too many bytes (query: %s, limit: %s). Consider adding more specific stream selectors or reduce the time range of the query" | ||
| limErrQuerierTooManyBytesTmpl = "query too large to execute on a single querier: (query: %s, limit: %s). Consider adding more specific stream selectors, reduce the time range of the query, or adjust parallelization settings" | ||
| limErrQuerierTooManyBytesShardableTmpl = "not shardable query too large to execute on a single querier: (query: %s, limit: %s). Consider adding more specific stream selectors or reduce the time range of the query" |
There was a problem hiding this comment.
Suggested change
| limErrQuerierTooManyBytesShardableTmpl = "not shardable query too large to execute on a single querier: (query: %s, limit: %s). Consider adding more specific stream selectors or reduce the time range of the query" | |
| limErrQuerierTooManyBytesUnshardableTmpl = "un-shardable query too large to execute on a single querier: (query: %s, limit: %s). Consider adding more specific stream selectors or reducing the time range of the query" |
pkg/querier/queryrange/limits.go
Outdated
| limErrQueryTooManyBytesTmpl = "the query would read too many bytes (query: %s, limit: %s). Consider adding more specific stream selectors or reduce the time range of the query" | ||
| limErrQuerierTooManyBytesTmpl = "query too large to execute on a single querier: (query: %s, limit: %s). Consider adding more specific stream selectors, reduce the time range of the query, or adjust parallelization settings" | ||
| limErrQuerierTooManyBytesShardableTmpl = "not shardable query too large to execute on a single querier: (query: %s, limit: %s). Consider adding more specific stream selectors or reduce the time range of the query" | ||
| limErrQuerierTooManyBytesUnshardableTmpl = "shard query is too big to execute on a single querier: (query: %s, limit: %s). Consider adding more specific stream selectors or reduce the time range of the query" |
There was a problem hiding this comment.
Suggested change
| limErrQuerierTooManyBytesUnshardableTmpl = "shard query is too big to execute on a single querier: (query: %s, limit: %s). Consider adding more specific stream selectors or reduce the time range of the query" | |
| limErrQuerierTooManyBytesShardableTmpl = "sharded query too large to execute on a single querier: (query: %s, limit: %s). Consider adding more specific stream selectors or reducing the time range of the query" |
dannykopping
approved these changes
Mar 28, 2023
Co-authored-by: Danny Kopping <[email protected]>
salvacorts
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
In #8670 we introduced a new limit
max_querier_bytes_read. When the limit was surpassed the following error message is printed:As pointed out in this comment, a user would have a hard time figuring out whether the cause was
parallelization is not enabled,the query is unshardableora shard query is too big to execute.This PR improves the error messaging for the
max_querier_bytes_readlimit to raise a different error for each of the causes above.Which issue(s) this PR fixes:
Followup for #8670
Special notes for your reviewer:
Checklist
CONTRIBUTING.mdguide (required)CHANGELOG.mdupdateddocs/sources/upgrading/_index.md