STEPS:
-
Create "airflow" directory and download docker-compose.yaml into it. (make sure internal and external ports are mapped for Postgres container, otherwise you won't be able to connect dbeaver or another database IDE for that matter to you PostgreSQL database)
-
Create sub-directories such as "./logs", "./dags", "./config" and "./plugins"
-
Run command
docker compose up -d
inside "airflow" folder and wait until run is success. -
Download and put ETL_RUN.py to your "./dags" folder
-
Use Dbeaver to create connection (username and password are "airflow")
-
Use DDL's from SQL folder to create necessary entities.
-
Go to http://localhost:8080/ ,find your dag and run it.
-
Check result in the tables below: