[omnibus] Fix snowflake-connector-python dependencies #6666
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
derekwbrown
approved these changes
Oct 30, 2020
…-connector-python-v2
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Adds software definitions to build a patched verison of
snowflake-connector-python==2.1.3that is compatible with the other python packages we ship.Also fixes the
pip checkstep ofdatadog-agent-integrations-py*, which was being executed before integrations were installed (making it pretty much useless).Motivation
DataDog/integrations-core#7869 bumped the
cryptographydependency to3.2.1. However, the version ofsnowflake-connector-pythonwe use (the last one that supports python 2) isn't compatible withcryptography3.x (setup.pyhascryptography < 3.0in its requirements).Additional Notes
The
connector_python2andconnector_python3folders in thesnowflake-connector-pythonrepo are symlinks to thetestfolder (see https://github.com/snowflakedb/snowflake-connector-python/tree/v2.1.3). These symlinks caused weirdEnvironmentErrorson Windows. That's why we're replacing them will real copies of thetestfolder.Small optimisation for later: we could have a
snowflake-connector-pythondefinition that does the core work (download + patch), andsnowflake-connector-python-py2andsnowflake-connector-python-py3would use this to install the package.Note: the
pip checkneeds to be moved as part of this PR, as running it before installing integrations now fails (because of the pre-builtsnowflake-connector-python).The snowflake integration should be tested to check that this build change doesn't break it.