Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FlowETL doesn't pick up custom QA checks #3484

Closed
jc-harrison opened this issue Jan 13, 2021 · 2 comments · Fixed by #3489
Closed

FlowETL doesn't pick up custom QA checks #3484

jc-harrison opened this issue Jan 13, 2021 · 2 comments · Fixed by #3489
Labels
bug Something isn't working FlowETL

Comments

@jc-harrison
Copy link
Member

I have tried adding a custom QA check template in '<dag_folder>/qa_checks/', as indicated in the get_qa_checks docstring, but it doesn't get added as a QA check task.

As a workaround, it would be useful to add a template_searchpath parameter to create_dag, so that the path to QA check templates could at least be manually added.

@greenape greenape added the bug Something isn't working label Jan 14, 2021
@greenape
Copy link
Member

Hmm. Looks like we're trying to include these on the assumption that airflow rolls in the dag automatically to the template env, which it does but at the point of there being a DagRun, not at the point of there being a dag - so when we call dag.template_searchpath as part of our dag creation, the dag folder isn't included.

@greenape
Copy link
Member

Looks like we should explicitly add airflow.settings.DAGS_FOLDER to the list we search, but would also make sense to allow adding additional search paths.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working FlowETL
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants