-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix(snowflake): convert parameters to qmark style #350
Conversation
the parameters value passed from the backend can be incompatible with pyformat mapping (dict or array values). I use this method that is already used on other conenctors Another way would be to flatten parameters, but it could still leave some incompatible values (arrays of dict).
7c6704f
to
fac0e98
Compare
Thanks for this fix 👍 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice 👍
A very small suggestion regarding delimiters used in examples:
@@ -36,7 +36,7 @@ | |||
domain='test_domain', | |||
database='test_database', | |||
warehouse='test_warehouse', | |||
query='test_query with %(foo)s and {{ pokemon }}', | |||
query='test_query with %(foo)s and %(pokemon)s', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
%(...)
is an old python syntax, I think we'd prefer to push Jinja' one in our connectors configs
Today, we're already replacing these automatically:
# Replace param templating with jinja templating: |
So my suggestion would be to use jinja {{ }} in your tests and documentation as well
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
les syntaxes compatibles avec execute de snowflake sont
- pyformat (
%()s
) => bug car on passe un dictionnaire parameters qui peut comporter des valeurs invalides. - qmark => cest dans ce format qu'on convertit notre query pour bypasser le probleme du dictionnaire.
Par contre on ne peut pas utiliser jinja car ce n'est pas safe sur des requetes SQL
cf le code que tu pointes, qui est extrait de la methode no_sql_apply_parameters_to_query :
def nosql_apply_parameters_to_query(query, parameters, handle_errors=False):
"""
WARNING : DO NOT USE THIS WITH VARIANTS OF SQL
Instead use your client library parameter substitution method.
https://www.owasp.org/index.php/Query_Parameterization_Cheat_Sheet
"""
on veut spécifiquement éviter le templating de cette manière.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cannot use jinja for sql query parameters
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We had a slack discussion about which style of param to use, let's open an issue/think more about it, but this PR is not the place for that.
Change Summary
The parameters value passed from the backend can be incompatible
with pyformat mapping (dict or array values).
I use convert to qmark method that is already used on other connectors
Another way would be to flatten parameters, but it could still leave
some incompatible values (arrays of dict).
Checklist