-
Notifications
You must be signed in to change notification settings - Fork 14.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add deferrable GCSObjectExistenceSensorAsync
#28763
Conversation
80e5717
to
a98cf06
Compare
GCSObjectExistenceSensorAsync
When i re-requested for review. It automatically removed reviewers and now i am not able to add them back. |
a335094
to
573025a
Compare
Hi @rajaths010494 @kaxil - I wanted to point out a possible bug that is introduced here - it seems like using this approach: async def get_storage_client(self, session: ClientSession) -> Storage:
"""Returns a Google Cloud Storage service object."""
with await self.service_file_as_context() as file:
return Storage(service_file=file, session=cast(Session, session)) does not honor the impersonation_chain when application default credentials (ADC) are being used. We've tried a similar approach from astronomer-providers and encountered issues on our end. I believe this is because in the case that
TalkIQ's async io storage client has no idea of the impersonation chain on instantiation here --> https://github.com/talkiq/gcloud-aio/blob/4e0006240913ecadde68722b8986f184a9b1adea/storage/gcloud/aio/storage/storage.py#L149-L168 And as a result, the token gets instantiated with ADC without regard for impersonation chain. Edit: On second look, it seems like provide_gcp_credential_file_as_context does not honor the impersonation chain at all? Compare this with get_credentials_and_project_id which handles the impersonation chain properly |
Thanks @vchiapaikeo i will look at this and check get_credentials_and_project_id which handles the impersonation chain properly. |
This PR donates the following developed GCSObjectExistenceSensorAsync` in astronomer-providers repo to apache airflow.
GCSObjectExistenceSensorAsync