Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AIP-72: Move Secrets Masker to task SDK #46375

Merged
merged 15 commits into from
Feb 5, 2025

Conversation

amoghrajesh
Copy link
Contributor

@amoghrajesh amoghrajesh commented Feb 3, 2025

closes: #45438

Moving the secrets masker class to task sdk so that it can be used to write dags and tasks, in form of custom operators

TODO:

  • Move the tests over to sdk
  • Add a newsfragment as its a breaking change (?)

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@amoghrajesh
Copy link
Contributor Author

@eladkal / @Lee-W would this require a newsfragment?

SecretsMasker was always a public interface and some users would have some custom operators would probably have used it for their redaction reasons / anything or that sort. That would mean that they might have to rewrite these operators atleast for the imports.

@Lee-W
Copy link
Member

Lee-W commented Feb 4, 2025

@eladkal / @Lee-W would this require a newsfragment?

SecretsMasker was always a public interface and some users would have some custom operators would probably have used it for their redaction reasons / anything or that sort. That would mean that they might have to rewrite these operators atleast for the imports.

Yes, it would be categorized as Behavior change based on previous discussion

Copy link
Member

@ashb ashb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does put a hard dep on Task SDK for various things in the scheduler/code that we might want to split out later, but that exists for now anyway and not duplicating the code is better.

@amoghrajesh amoghrajesh force-pushed the AIP72-move-secrets-masker branch from 8735fed to dde6603 Compare February 4, 2025 12:36
@amoghrajesh amoghrajesh self-assigned this Feb 5, 2025
@amoghrajesh amoghrajesh added the airflow3.0:breaking Candidates for Airflow 3.0 that contain breaking changes label Feb 5, 2025
@amoghrajesh
Copy link
Contributor Author

@Lee-W I added a newsfragment for this, does it look OK only with respect to that?

Copy link
Member

@Lee-W Lee-W left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the newsfragment looks good to me 👍

@amoghrajesh
Copy link
Contributor Author

Thanks for the reviews, merging it

@amoghrajesh amoghrajesh merged commit e78b5b4 into apache:main Feb 5, 2025
91 checks passed
@amoghrajesh amoghrajesh deleted the AIP72-move-secrets-masker branch February 5, 2025 07:34
jedcunningham added a commit to astronomer/airflow that referenced this pull request Feb 5, 2025
insomnes pushed a commit to insomnes/airflow that referenced this pull request Feb 6, 2025
niklasr22 pushed a commit to niklasr22/airflow that referenced this pull request Feb 8, 2025
ambika-garg pushed a commit to ambika-garg/airflow that referenced this pull request Feb 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
airflow3.0:breaking Candidates for Airflow 3.0 that contain breaking changes area:API Airflow's REST/HTTP API area:CLI area:dev-tools area:providers area:secrets area:serialization area:task-sdk area:webserver Webserver related Issues kind:documentation legacy api Whether legacy API changes should be allowed in PR provider:amazon AWS/Amazon - related issues provider:cncf-kubernetes Kubernetes provider related issues provider:openlineage AIP-53
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Port SecretsMasker log filter to Task SDK
3 participants