Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEAT] convert blocking rules to DNF #1493

Open
NickCrews opened this issue Aug 1, 2023 · 3 comments
Open

[FEAT] convert blocking rules to DNF #1493

NickCrews opened this issue Aug 1, 2023 · 3 comments
Labels
blocking enhancement New feature or request

Comments

@NickCrews
Copy link
Contributor

In the docs it suggests to avoid blocking rules with OR, because that prevents the engine from using a hash join.

But I think we can take care of this for users. can we use https://sqlglot.com/sqlglot/optimizer/normalize.html#normalize to convert whatever rules they give us into DNF, and then split this up on the ORs to get our disjoint blocking rules, all which contain only ANDs?

@NickCrews NickCrews added the enhancement New feature or request label Aug 1, 2023
@RossKen
Copy link
Contributor

RossKen commented Aug 11, 2023

Hey @NickCrews - this is really cool! I wasn't aware of that function in sqlglot - this is definitely something that would be great to implement under the hood then we can change the docs 😊

@RossKen
Copy link
Contributor

RossKen commented Oct 3, 2023

Possibility to use this normalize function to flag more performant blocking rules to users, rather than fixing it under the hood. Slightly hesitant to just fix things in the background.

@NickCrews
Copy link
Contributor Author

yeah that is a good thing to think about. But, I'm trying to think of a situation where just optimizing it for users wouldn't be the correct thing to do, and I'm drawing a blank...can you think of one?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blocking enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants