Skip to content

Commit

Permalink
Migrate HTTP example DAGs to new design AIP-47 (#23991)
Browse files Browse the repository at this point in the history
closes: #22448 , #22431
  • Loading branch information
chenglongyan authored Jun 1, 2022
1 parent 3dd7b1d commit 9398586
Show file tree
Hide file tree
Showing 4 changed files with 18 additions and 57 deletions.
17 changes: 0 additions & 17 deletions airflow/providers/http/example_dags/__init__.py

This file was deleted.

14 changes: 7 additions & 7 deletions docs/apache-airflow-providers-http/operators.rst
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ to ``true``.

Here we are poking until httpbin gives us a response text containing ``httpbin``.

.. exampleinclude:: /../../airflow/providers/http/example_dags/example_http.py
.. exampleinclude:: /../../tests/system/providers/http/example_http.py
:language: python
:start-after: [START howto_operator_http_http_sensor_check]
:end-before: [END howto_operator_http_http_sensor_check]
Expand Down Expand Up @@ -69,14 +69,14 @@ the response text back.
In the first example we are calling a ``POST`` with json data and succeed when we get the same json data back
otherwise the task will fail.

.. exampleinclude:: /../../airflow/providers/http/example_dags/example_http.py
.. exampleinclude:: /../../tests/system/providers/http/example_http.py
:language: python
:start-after: [START howto_operator_http_task_post_op]
:end-before: [END howto_operator_http_task_post_op]

Here we are calling a ``GET`` request and pass params to it. The task will succeed regardless of the response text.

.. exampleinclude:: /../../airflow/providers/http/example_dags/example_http.py
.. exampleinclude:: /../../tests/system/providers/http/example_http.py
:language: python
:start-after: [START howto_operator_http_task_get_op]
:end-before: [END howto_operator_http_task_get_op]
Expand All @@ -91,30 +91,30 @@ it on the next task downstream use ``response_filter``. This is useful if:
Below is an example of retrieving data from a REST API and only returning a nested property instead of the full
response body.

.. exampleinclude:: /../../airflow/providers/http/example_dags/example_http.py
.. exampleinclude:: /../../tests/system/providers/http/example_http.py
:language: python
:start-after: [START howto_operator_http_task_get_op_response_filter]
:end-before: [END howto_operator_http_task_get_op_response_filter]

In the third example we are performing a ``PUT`` operation to put / set data according to the data that is being
provided to the request.

.. exampleinclude:: /../../airflow/providers/http/example_dags/example_http.py
.. exampleinclude:: /../../tests/system/providers/http/example_http.py
:language: python
:start-after: [START howto_operator_http_task_put_op]
:end-before: [END howto_operator_http_task_put_op]

In this example we call a ``DELETE`` operation to the ``delete`` endpoint. This time we are passing form data to the
request.

.. exampleinclude:: /../../airflow/providers/http/example_dags/example_http.py
.. exampleinclude:: /../../tests/system/providers/http/example_http.py
:language: python
:start-after: [START howto_operator_http_task_del_op]
:end-before: [END howto_operator_http_task_del_op]

Here we pass form data to a ``POST`` operation which is equal to a usual form submit.

.. exampleinclude:: /../../airflow/providers/http/example_dags/example_http.py
.. exampleinclude:: /../../tests/system/providers/http/example_http.py
:language: python
:start-after: [START howto_operator_http_task_post_op_formenc]
:end-before: [END howto_operator_http_task_post_op_formenc]
32 changes: 0 additions & 32 deletions tests/providers/http/operators/test_http_system.py

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -19,14 +19,19 @@
"""Example HTTP operator and sensor"""

import json
import os
from datetime import datetime

from airflow import DAG
from airflow.providers.http.operators.http import SimpleHttpOperator
from airflow.providers.http.sensors.http import HttpSensor

ENV_ID = os.environ.get("SYSTEM_TESTS_ENV_ID")
DAG_ID = 'example_http_operator'


dag = DAG(
'example_http_operator',
DAG_ID,
default_args={'retries': 1},
tags=['example'],
start_date=datetime(2021, 1, 1),
Expand Down Expand Up @@ -107,3 +112,8 @@
# [END howto_operator_http_http_sensor_check]
task_http_sensor_check >> task_post_op >> task_get_op >> task_get_op_response_filter
task_get_op_response_filter >> task_put_op >> task_del_op >> task_post_op_formenc

from tests.system.utils import get_test_run # noqa: E402

# Needed to run the example DAG with pytest (see: tests/system/README.md#run_via_pytest)
test_run = get_test_run(dag)

0 comments on commit 9398586

Please sign in to comment.