Skip to content

Commit

Permalink
Add docs about Celery monitoring (#14533)
Browse files Browse the repository at this point in the history
Part of: #11161

To have a full description of the monitoring of all Airflow components, I add information about HTTP and CLI checks for Celery. Thanks to this, we will not have to search for information one by one, but everything will be in one place.

Documentation for Celery does not describe the inspect ping command, but hopefully, this will be added soon.

(cherry picked from commit 8cabae7)
  • Loading branch information
mik-laj authored and potiuk committed Mar 3, 2021
1 parent 2418915 commit cf1a66f
Showing 1 changed file with 27 additions and 0 deletions.
27 changes: 27 additions & 0 deletions docs/apache-airflow/logging-monitoring/check-health.rst
Original file line number Diff line number Diff line change
Expand Up @@ -89,3 +89,30 @@ CLI Check for Database

To verify that the database is working correctly, you can use the ``airflow db check`` command. On failure, the command will exit
with a non-zero error code.

HTTP monitoring for Celery Cluster
----------------------------------

You can use Flower to monitor the health of the Celery cluster. It also provides an HTTP API that you can use to build a health check for your environment.

For details about installation, see: :ref:`executor:CeleryExecutor`. For details about usage, see: `The Flower project documentation <https://flower.readthedocs.io/en/stable/>`__.

CLI Check for Celery Workers
----------------------------

To verify that the database is working correctly, you can use the ``celery inspect ping`` command. On failure, the command will exit
with a non-zero error code.

To check if the worker running on the local host is working correctly, run:

.. code-block:: bash
celery --app airflow.executors.celery_executor.app inspect ping -d celery@${HOSTNAME}
To check if the all workers in the cluster running is working correctly, run:

.. code-block:: bash
celery --app airflow.executors.celery_executor.app inspect ping
For more information, see: `Management Command-line Utilities (inspect/control) <https://docs.celeryproject.org/en/stable/userguide/monitoring.html#monitoring-control>`__ and `Workers Guide <https://docs.celeryproject.org/en/stable/userguide/workers.html>`__ in the Celery documentation.

0 comments on commit cf1a66f

Please sign in to comment.