Suspend Yandex provider due to protobuf limitation #30667
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
potiuk
force-pushed
the
suspend-yandex-provider
branch
from
face338 to
ecd1d3c
Compare
eladkal
approved these changes
Apr 16, 2023
eladkal
reviewed
Apr 16, 2023
potiuk
force-pushed
the
suspend-yandex-provider
branch
2 times, most recently
from
31c1682 to
bbebd70
Compare
hussein-awala
approved these changes
Apr 16, 2023
There was a problem hiding this comment.
wow, it shows how much #30422 has simplified the suspension process! LGTM
Yep. All with a single flag flipped. That's the beauty of having everything under control of our CI/Breeze/pre-commits framework. |
Yandex provider brings protobuf dependency down to <4 and we are gearing up to updating it everywhere else. Protobuf3 support ends in Q2 2023 for Python https://protobuf.dev/support/version-support/#python Yandex is the last provider that we do not closely collaborate with on fixing * Gogle provider dependencies are actively upgraded to latest version by Google led team: apache#30067 (some of the libraries are already updated) with target to update all dependencies by mid-May * Apache-Beam has already merged protobuf4 support apache/beam#25874 with the target of releasing it in 2.47.0 mid-May * The mysql-connector-python in MySQL provider is already turned into optional dependency: apache#30487 The only remaining dependency limiting us to protobuf 3 (<3.21) is yandexcloud. We've opened an issue to yandexcloud yandex-cloud/python-sdk#71 3 weeks ago and while there was an initial interest, there is no progress on the issue, therefore - in order to prepare for running all the tests and final migration to protobuf4 we need to suspend Yandex provider - following the suspension process we agreed and got a LAZY CONSENSUS on in the https://lists.apache.org/thread/g8b3k028qhzgw6c3yz4jvmlc67kcr9hj mailing list discussion. The yandex provider can be removed from suspension by a PR reverting this change once yandexcloud dependency removes the protobuf limitation in their release and PR reverting this change (and fixing all tests and static check that will be needed) is the way it can be done.
potiuk
force-pushed
the
suspend-yandex-provider
branch
from
bbebd70 to
34ef673
Compare
38 tasks
ephraimbuddy
added
the
changelog:skip
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Yandex provider brings protobuf dependency down to <4 and we are gearing up to updating it everywhere else. Protobuf3 support ends in Q2 2023 for Python https://protobuf.dev/support/version-support/#python
Yandex is the last provider that we do not closely collaborate with on fixing it
The only remaining dependency limiting us to protobuf 3 (<3.21) is yandexcloud. We've opened an issue to yandexcloud
yandex-cloud/python-sdk#71 3 weeks ago and while there was an initial interest, there is no progress on the issue, therefore - in order to prepare for running all the tests and final migration to protobuf4 we need to suspend Yandex provider - following the suspension process we agreed and got a LAZY CONSENSUS on in
the https://lists.apache.org/thread/g8b3k028qhzgw6c3yz4jvmlc67kcr9hj mailing list discussion.
The yandex provider can be removed from suspension by a PR reverting this change once yandexcloud dependency removes the protobuf limitation in their release and PR reverting this change (and fixing all tests and static check that will be needed) is the way it can be done.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.