Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bitnami/airflow] Add dag volume mounts to web deployment for configmap-based dags #31620

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

TheRobotCarlson
Copy link
Contributor

@TheRobotCarlson TheRobotCarlson commented Jan 28, 2025

Description of the change

When using configmap-based DAGs, the DAGs are not linked to the webapp correctly, leading to them not being properly loaded in the UI. This PR adds them and the init containers that sync them to the necessary volume and volume locations.

Benefits

Possible drawbacks

Applicable issues

  • fixes #

Additional information

Checklist

  • Chart version bumped in Chart.yaml according to semver. This is not necessary when the changes only affect README.md files.
  • Variables are documented in the values.yaml and added to the README.md using readme-generator-for-helm
  • Title of the pull request follows this pattern [bitnami/<name_of_the_chart>] Descriptive title
  • All commits signed off and in agreement of Developer Certificate of Origin (DCO)

@github-actions github-actions bot added airflow triage Triage is needed labels Jan 28, 2025
@github-actions github-actions bot requested a review from carrodher January 28, 2025 15:56
Signed-off-by: Brian Carlson <[email protected]>
Signed-off-by: Bitnami Containers <[email protected]>
@TheRobotCarlson TheRobotCarlson changed the title Add dag volume mounts to web deployment for configmap-based dags [bitnami/airflow] Add dag volume mounts to web deployment for configmap-based dags Jan 28, 2025
@carrodher carrodher added verify Execute verification workflow for these changes in-progress labels Jan 29, 2025
@github-actions github-actions bot removed the triage Triage is needed label Jan 29, 2025
@github-actions github-actions bot removed the request for review from carrodher January 29, 2025 07:48
@github-actions github-actions bot requested a review from dgomezleon January 29, 2025 07:48
Signed-off-by: Bitnami Containers <[email protected]>
dgomezleon
dgomezleon previously approved these changes Jan 29, 2025
Copy link
Member

@dgomezleon dgomezleon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for contributing

@dgomezleon
Copy link
Member

Hi @TheRobotCarlson ,

We needed to apply this change to make it work with our tests. Could you please rebase it? Also, you would need to add the syncDags sidecar container.

@TheRobotCarlson
Copy link
Contributor Author

Hi @dgomezleon, I've rebased. The sync-dags init container isn't needed for configmap-based deployments, is it? Since it is responsible for syncing git repos and not configmaps?

@juan131
Copy link
Contributor

juan131 commented Jan 30, 2025

Hi @dgomezleon, I've rebased. The sync-dags init container isn't needed for configmap-based deployments, is it? Since it is responsible for syncing git repos and not configmaps?

Correct, but you're adding a generic solution that should be valid for both obtaining DAGs from an existing ConfigMaps OR cloning a repository with them.

@TheRobotCarlson
Copy link
Contributor Author

@juan131 Does this mean the git-based DAG loading flow is not being tested at all? This would mean neither configmap-based or git-based DAG flows are working from the default deployment. I have been testing configmap-based deployments and don't have a git-based deployment flow to test from currently.

Signed-off-by: Bitnami Containers <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
airflow in-progress verify Execute verification workflow for these changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants