Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sec. index: copy scan filter conditions only arrays #3306

Open
Tracked by #201
islamaliev opened this issue Dec 9, 2024 · 0 comments
Open
Tracked by #201

Sec. index: copy scan filter conditions only arrays #3306

islamaliev opened this issue Dec 9, 2024 · 0 comments
Labels
area/query Related to the query component perf Performance issue or suggestion

Comments

@islamaliev
Copy link
Contributor

islamaliev commented Dec 9, 2024

in the initFetcher method of scanNode we check if we should split the given filter into 2: index-relevant and regular filters. For some special cases we do not split (i.e. don't move from out of the original filter), but rather copy index-relevant filter. This is a case for example, with array filter like _any, _all and _none. The index has no way of properly asserting these conditions, it can only give hint on what documents to check. That's why we keep the original filter intact so that it can do the final check.

With array filter within json documents the situation is the same. But it seems to be relevant only to array fields (need to double-check to be 100% sure).

For now in the method we check if fd.Kind.IsArray() || fd.Kind == client.FieldKind_NILLABLE_JSON which makes all json filters be treated this way. This is not very optimal as we can limit it only to arrays. This would mean for many json filters not on array fields, we won't need to run another filter check.

@islamaliev islamaliev added area/query Related to the query component perf Performance issue or suggestion labels Dec 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/query Related to the query component perf Performance issue or suggestion
Projects
None yet
Development

No branches or pull requests

1 participant