Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Helm GitSync dag volume mount #15331

Merged
merged 1 commit into from
Apr 12, 2021
Merged

Conversation

matttattoli
Copy link
Contributor

This fixes an issue where the workers cannot find the dag that is meant to run.

With git-sync enabled, the airflow_dag location is being set to where the dags are being cloned to, ex $AIRFLOWHOME/dags/repo/dags, but the subpath is remapping it to $AIRFLOWHOME/dags, thus the workers are not able to find the actually dag.

Also related: https://github.com/apache/airflow/blob/master/chart/templates/_helpers.yaml#L333

Removing the subpath resolves this issue. I locally ran the helm unit tests, and it succeeded.

@boring-cyborg boring-cyborg bot added the area:helm-chart Airflow Helm Chart label Apr 12, 2021
@boring-cyborg
Copy link

boring-cyborg bot commented Apr 12, 2021

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst)
Here are some useful points:

  • Pay attention to the quality of your code (flake8, pylint and type annotations). Our pre-commits will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
  • Consider using Breeze environment for testing locally, it’s a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: [email protected]
    Slack: https://s.apache.org/airflow-slack

@KevinDukelow5
Copy link

👍

@kaxil kaxil merged commit e4c0689 into apache:master Apr 12, 2021
@boring-cyborg
Copy link

boring-cyborg bot commented Apr 12, 2021

Awesome work, congrats on your first merged pull request!

matttattoli added a commit to colpal/airfloss that referenced this pull request Apr 15, 2021
@matttattoli
Copy link
Contributor Author

matttattoli commented Apr 15, 2021

@kaxil

I did some testing in master, and this is actually an invalid fix. It seems this is broken in branch v2-0-stable, but may have been fixed in master from some other commit. Please advise if you want to revert this PR.

@kaxil
Copy link
Member

kaxil commented Apr 15, 2021

@matttattoli Sure if it is an invalid fix let's revert this and add a test so next time this test fails on such a change :)

We are planning to release Helm chart -- early May so only use it from Master instead of v2-0-stable

@matttattoli
Copy link
Contributor Author

Revert PR here: #15390

Good to know, excited to hear the helm chart will be hosted soon. Will use master until then. Sorry for any inconvenience.

kaxil pushed a commit that referenced this pull request Apr 15, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:helm-chart Airflow Helm Chart
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants