Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CLI support for bulk pause and resume of DAGs #36956

Closed
2 tasks done
shahar1 opened this issue Jan 22, 2024 · 2 comments · Fixed by #38265
Closed
2 tasks done

Add CLI support for bulk pause and resume of DAGs #36956

shahar1 opened this issue Jan 22, 2024 · 2 comments · Fixed by #38265
Assignees
Labels
area:CLI kind:feature Feature Requests

Comments

@shahar1
Copy link
Contributor

shahar1 commented Jan 22, 2024

Description

Allow Airflow CLI's dag subcommands , pause and unpause, to support multiple DAG IDs instead of only one.

Use case/motivation

There are cases where we would like to pause or resume multiple DAGs at once. As a Cloud Composer user - I'd like to do both, for example, before and after upgrading an environment, in correspondence. The current methods of doing it, at least in Cloud Composer, are manually via the UI (not scalable), via Airflow API using dag_id_pattern (not easily accessible, at least in CC), or one-by-one via the CLI - most straightforward to perform, but only supports one DAG id per run (considering the extra overhead for accessing the CC's GKE - it could take up to a minute to pause or unpause each DAG!).

Related issues

Possible Caveats

#32360 - as dag_id_pattern might be changed in Airflow's API - the method for passing multiple DAG ids should be considered.

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@shahar1 shahar1 added kind:feature Feature Requests needs-triage label for new issues that we didn't triage yet labels Jan 22, 2024
@Taragolis Taragolis added area:CLI and removed needs-triage label for new issues that we didn't triage yet labels Jan 23, 2024
@enchant3dmango
Copy link
Contributor

@shahar1 @potiuk I think this feature request could be developed along with the PR for this feature.

@potiuk
Copy link
Member

potiuk commented Jan 26, 2024

@shahar1 @potiuk I think this feature request could be developed along with the PR for this feature.

Or separately. There is no particular reasons to bundle such things together, the smaller the scope of the PR, the higher chance someone will complete it and the faster it goes through review.

@shahar1 shahar1 changed the title Add CLI support for bulk pause and resume of DAGs Add CLI support for bulk pause and unpause of DAGs Mar 18, 2024
@shahar1 shahar1 changed the title Add CLI support for bulk pause and unpause of DAGs Add CLI support for bulk pause and resume of DAGs Mar 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:CLI kind:feature Feature Requests
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants