-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor(airflow): remove verbose log from airflow plugin #6516
refactor(airflow): remove verbose log from airflow plugin #6516
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
had some minor questions, but thanks for taking this on
@@ -275,7 +275,6 @@ def custom_on_success_callback(context): | |||
|
|||
|
|||
def task_policy(task: BaseOperator) -> None: | |||
print(f"Setting task policy for Dag: {task.dag_id} Task: {task.task_id}") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
maybe we can change it to logger.debug
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@@ -302,8 +301,6 @@ def _patch_policy(settings): | |||
|
|||
|
|||
def _patch_datahub_policy(): | |||
print("Patching datahub policy") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
imo this one should still be logger.info - I believe this one only gets printed once (as opposed to per dag per task), so a higher log level is ok
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Lgtm, thanks!
…oject#6516) Co-authored-by: John Joyce <[email protected]>
This PR removes the verbose log from the airflow plugin.
These logs are pretty verbose since
task_policy
are executed for all tasks in dags whenever the airflow CLI command is executed.Checklist