Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consolidate to one schedule param #25410

Merged
merged 38 commits into from
Aug 10, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
c30cb15
Rename DAG `schedule_on` parameter to `schedule`
dstandish Jul 29, 2022
dba73a5
add deprecation warning for old params
dstandish Jul 29, 2022
1b38a3f
harvest schedule interval args and timetables from `schedule`
dstandish Jul 29, 2022
2696733
fixup! harvest schedule interval args and timetables from `schedule`
dstandish Jul 29, 2022
f24a55a
fix runtime checkable import
dstandish Jul 29, 2022
4d0ef73
rename most usages of schedule_interval= to schedule=
dstandish Jul 29, 2022
59fd25c
stacklevel
dstandish Jul 29, 2022
0c96ac0
fix param handling
dstandish Jul 29, 2022
8d90bc4
update some docs
dstandish Aug 7, 2022
f80e3a6
update docs
dstandish Aug 7, 2022
68cd2ee
fix tests
dstandish Aug 7, 2022
d08c5d5
fix test dag endpoint
dstandish Aug 7, 2022
ae3b1d0
fix dag run endpoint test
dstandish Aug 7, 2022
cf11b53
fix param namee
dstandish Aug 7, 2022
d8e06fa
fix type
dstandish Aug 7, 2022
a4b99cc
fix tests
dstandish Aug 7, 2022
244e496
fix tests
dstandish Aug 8, 2022
759c10c
provider examples should stick with schedule_interval for now
dstandish Aug 8, 2022
b32a0aa
remove schedule_interval param from example dags
dstandish Aug 8, 2022
c1d0c34
fixup! remove schedule_interval param from example dags
dstandish Aug 8, 2022
7bc7f76
fix test
dstandish Aug 8, 2022
721881e
Update scripts/in_container/verify_providers.py
dstandish Aug 9, 2022
6d00cef
Update docs/apache-airflow/faq.rst
dstandish Aug 9, 2022
fd808bd
Update docs/apache-airflow/dag-run.rst
dstandish Aug 9, 2022
fd3d9e9
Update docs/apache-airflow/dag-run.rst
dstandish Aug 9, 2022
b49c39b
add example
dstandish Aug 9, 2022
2509e8e
simplify example
dstandish Aug 9, 2022
17531f8
Update newsfragments/25410.significant.rst
dstandish Aug 9, 2022
8e77857
add test
dstandish Aug 9, 2022
46ee72c
Simplify and fix test
uranusjr Aug 9, 2022
babf5c5
uppercase
dstandish Aug 9, 2022
7858b38
fix log testing
dstandish Aug 9, 2022
e5e50a3
Unify comment format
uranusjr Aug 9, 2022
eaca727
Fix casing an wording
uranusjr Aug 9, 2022
b168c2f
More casing and wording fixes
uranusjr Aug 9, 2022
d0559ac
A few more formatting fixes
uranusjr Aug 9, 2022
86238c6
Some more TaskDag conversions
uranusjr Aug 9, 2022
6f0003b
remove redundant param
dstandish Aug 9, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
2 changes: 1 addition & 1 deletion airflow/example_dags/example_bash_operator.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@

with DAG(
dag_id='example_bash_operator',
schedule_interval='0 0 * * *',
schedule='0 0 * * *',
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
catchup=False,
dagrun_timeout=datetime.timedelta(minutes=60),
Expand Down
4 changes: 2 additions & 2 deletions airflow/example_dags/example_branch_datetime_operator.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
catchup=False,
tags=["example"],
schedule_interval="@daily",
schedule="@daily",
)

# [START howto_branch_datetime_operator]
Expand All @@ -57,7 +57,7 @@
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
catchup=False,
tags=["example"],
schedule_interval="@daily",
schedule="@daily",
)
# [START howto_branch_datetime_operator_next_day]
empty_task_12 = EmptyOperator(task_id='date_in_range', dag=dag2)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
catchup=False,
tags=["example"],
schedule_interval="@daily",
schedule="@daily",
) as dag:
# [START howto_operator_day_of_week_branch]
empty_task_1 = EmptyOperator(task_id='branch_true')
Expand Down
2 changes: 1 addition & 1 deletion airflow/example_dags/example_branch_labels.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@

with DAG(
"example_branch_labels",
schedule_interval="@daily",
schedule="@daily",
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
catchup=False,
) as dag:
Expand Down
2 changes: 1 addition & 1 deletion airflow/example_dags/example_branch_operator.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
dag_id='example_branch_operator',
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
catchup=False,
schedule_interval="@daily",
schedule="@daily",
tags=['example', 'example2'],
) as dag:
run_this_first = EmptyOperator(
Expand Down
2 changes: 1 addition & 1 deletion airflow/example_dags/example_branch_operator_decorator.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
dag_id='example_branch_python_operator_decorator',
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
catchup=False,
schedule_interval="@daily",
schedule="@daily",
tags=['example', 'example2'],
) as dag:
run_this_first = EmptyOperator(task_id='run_this_first')
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ def should_run(**kwargs):

with DAG(
dag_id='example_branch_dop_operator_v3',
schedule_interval='*/1 * * * *',
schedule='*/1 * * * *',
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
catchup=False,
default_args={'depends_on_past': True},
Expand Down
2 changes: 1 addition & 1 deletion airflow/example_dags/example_complex.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@

with models.DAG(
dag_id="example_complex",
schedule_interval=None,
schedule=None,
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
catchup=False,
tags=['example', 'example2', 'example3'],
Expand Down
2 changes: 1 addition & 1 deletion airflow/example_dags/example_dag_decorator.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ def execute(self, context: Context):

# [START dag_decorator_usage]
@dag(
schedule_interval=None,
schedule=None,
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
catchup=False,
tags=['example'],
Expand Down
12 changes: 6 additions & 6 deletions airflow/example_dags/example_datasets.py
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@
dag_id='example_dataset_dag1',
catchup=False,
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
schedule_interval='@daily',
schedule='@daily',
tags=['upstream'],
) as dag1:
# [START task_outlet]
Expand All @@ -61,7 +61,7 @@
dag_id='example_dataset_dag2',
catchup=False,
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
schedule_interval=None,
schedule=None,
tags=['upstream'],
) as dag2:
BashOperator(outlets=[dag2_dataset], task_id='upstream_task_2', bash_command="sleep 5")
Expand All @@ -71,7 +71,7 @@
dag_id='example_dataset_dag3_req_dag1',
catchup=False,
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
schedule_on=[dag1_dataset],
schedule=[dag1_dataset],
tags=['downstream'],
) as dag3:
# [END dag_dep]
Expand All @@ -85,7 +85,7 @@
dag_id='example_dataset_dag4_req_dag1_dag2',
catchup=False,
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
schedule_on=[dag1_dataset, dag2_dataset],
schedule=[dag1_dataset, dag2_dataset],
tags=['downstream'],
) as dag4:
BashOperator(
Expand All @@ -98,7 +98,7 @@
dag_id='example_dataset_dag5_req_dag1_D',
catchup=False,
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
schedule_on=[
schedule=[
dag1_dataset,
Dataset('s3://this-dataset-doesnt-get-triggered'),
],
Expand All @@ -114,7 +114,7 @@
dag_id='example_dataset_dag6_req_DD',
catchup=False,
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
schedule_on=[
schedule=[
Dataset('s3://unrelated/dataset3.txt'),
Dataset('s3://unrelated/dataset_other_unknown.txt'),
],
Expand Down
4 changes: 2 additions & 2 deletions airflow/example_dags/example_external_task_marker_dag.py
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@
dag_id="example_external_task_marker_parent",
start_date=start_date,
catchup=False,
schedule_interval=None,
schedule=None,
tags=['example2'],
) as parent_dag:
# [START howto_operator_external_task_marker]
Expand All @@ -65,7 +65,7 @@
with DAG(
dag_id="example_external_task_marker_child",
start_date=start_date,
schedule_interval=None,
schedule=None,
catchup=False,
tags=['example2'],
) as child_dag:
Expand Down
2 changes: 1 addition & 1 deletion airflow/example_dags/example_kubernetes_executor.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@
if k8s:
with DAG(
dag_id='example_kubernetes_executor',
schedule_interval=None,
schedule=None,
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
catchup=False,
tags=['example3'],
Expand Down
2 changes: 1 addition & 1 deletion airflow/example_dags/example_latest_only.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@

with DAG(
dag_id='latest_only',
schedule_interval=dt.timedelta(hours=4),
schedule=dt.timedelta(hours=4),
start_date=dt.datetime(2021, 1, 1),
catchup=False,
tags=['example2', 'example3'],
Expand Down
2 changes: 1 addition & 1 deletion airflow/example_dags/example_latest_only_with_trigger.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@

with DAG(
dag_id='latest_only_with_trigger',
schedule_interval=datetime.timedelta(hours=4),
schedule=datetime.timedelta(hours=4),
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
catchup=False,
tags=['example3'],
Expand Down
2 changes: 1 addition & 1 deletion airflow/example_dags/example_local_kubernetes_executor.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@
if k8s:
with DAG(
dag_id='example_local_kubernetes_executor',
schedule_interval=None,
schedule=None,
start_date=datetime(2021, 1, 1),
catchup=False,
tags=['example3'],
Expand Down
2 changes: 1 addition & 1 deletion airflow/example_dags/example_nested_branch_dag.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
dag_id="example_nested_branch_dag",
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
catchup=False,
schedule_interval="@daily",
schedule="@daily",
tags=["example"],
) as dag:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ def print_env_vars(test_mode=None):

with DAG(
"example_passing_params_via_test_command",
schedule_interval='*/1 * * * *',
schedule='*/1 * * * *',
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
catchup=False,
dagrun_timeout=datetime.timedelta(minutes=4),
Expand Down
2 changes: 1 addition & 1 deletion airflow/example_dags/example_python_operator.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@

with DAG(
dag_id='example_python_operator',
schedule_interval=None,
schedule=None,
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
catchup=False,
tags=['example'],
Expand Down
2 changes: 1 addition & 1 deletion airflow/example_dags/example_sla_dag.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ def sla_callback(dag, task_list, blocking_task_list, slas, blocking_tis):


@dag(
schedule_interval="*/2 * * * *",
schedule="*/2 * * * *",
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
catchup=False,
sla_miss_callback=sla_callback,
Expand Down
2 changes: 1 addition & 1 deletion airflow/example_dags/example_subdag_operator.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
dag_id=DAG_NAME,
default_args={"retries": 2},
start_date=datetime.datetime(2022, 1, 1),
schedule_interval="@once",
schedule="@once",
tags=['example'],
) as dag:

Expand Down
2 changes: 1 addition & 1 deletion airflow/example_dags/example_time_delta_sensor_async.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@

with DAG(
dag_id="example_time_delta_sensor_async",
schedule_interval=None,
schedule=None,
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
catchup=False,
tags=["example"],
Expand Down
2 changes: 1 addition & 1 deletion airflow/example_dags/example_trigger_controller_dag.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@
dag_id="example_trigger_controller_dag",
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
catchup=False,
schedule_interval="@once",
schedule="@once",
tags=['example'],
) as dag:
trigger = TriggerDagRunOperator(
Expand Down
2 changes: 1 addition & 1 deletion airflow/example_dags/example_trigger_target_dag.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ def run_this_func(dag_run=None):
dag_id="example_trigger_target_dag",
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
catchup=False,
schedule_interval=None,
schedule=None,
tags=['example'],
) as dag:
run_this = run_this_func()
Expand Down
2 changes: 1 addition & 1 deletion airflow/example_dags/example_xcom.py
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ def pull_value_from_bash_push(ti=None):

with DAG(
'example_xcom',
schedule_interval="@once",
schedule="@once",
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
catchup=False,
tags=['example'],
Expand Down
4 changes: 2 additions & 2 deletions airflow/example_dags/example_xcomargs.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ def print_value(value, ts=None):
dag_id='example_xcom_args',
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
catchup=False,
schedule_interval=None,
schedule=None,
tags=['example'],
) as dag:
print_value(generate_value())
Expand All @@ -53,7 +53,7 @@ def print_value(value, ts=None):
"example_xcom_args_with_operators",
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
catchup=False,
schedule_interval=None,
schedule=None,
tags=['example'],
) as dag2:
bash_op1 = BashOperator(task_id="c", bash_command="echo c")
Expand Down
2 changes: 1 addition & 1 deletion airflow/example_dags/subdags/subdag.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ def subdag(parent_dag_name, child_dag_name, args):
default_args=args,
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
catchup=False,
schedule_interval="@daily",
schedule="@daily",
)

for i in range(5):
Expand Down
2 changes: 1 addition & 1 deletion airflow/example_dags/tutorial.py
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@
},
# [END default_args]
description='A simple tutorial DAG',
schedule_interval=timedelta(days=1),
schedule=timedelta(days=1),
start_date=datetime(2021, 1, 1),
catchup=False,
tags=['example'],
Expand Down
2 changes: 1 addition & 1 deletion airflow/example_dags/tutorial_etl_dag.py
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@
default_args={'retries': 2},
# [END default_args]
description='ETL DAG tutorial',
schedule_interval=None,
schedule=None,
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
catchup=False,
tags=['example'],
Expand Down
2 changes: 1 addition & 1 deletion airflow/example_dags/tutorial_taskflow_api_etl.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@

# [START instantiate_dag]
@dag(
schedule_interval=None,
schedule=None,
start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
catchup=False,
tags=['example'],
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@
)
else:

@dag(schedule_interval=None, start_date=datetime(2021, 1, 1), catchup=False, tags=['example'])
@dag(schedule=None, start_date=datetime(2021, 1, 1), catchup=False, tags=['example'])
def tutorial_taskflow_api_etl_virtualenv():
"""
### TaskFlow API example using virtualenv
Expand Down
Loading