Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scheduled ingestions does not include env variables #10739

Closed
am2222 opened this issue Jun 18, 2024 · 6 comments
Closed

Scheduled ingestions does not include env variables #10739

am2222 opened this issue Jun 18, 2024 · 6 comments
Labels
bug Bug report

Comments

@am2222
Copy link

am2222 commented Jun 18, 2024

Describe the bug
Once we create a new ingestion from UI we have options to set env variables for the task. If we set some env variables and run that ingestion manually it applies them on the venv but datahub keeps forgetting the env variables for the rest of the scheduled tasks.
To Reproduce
Steps to reproduce the behavior:

  1. create an ingestion using ui. lets say create an mlflow ingestion
  2. set an env variable. I will set {"MLFLOW_TRACKING_INSECURE_TLS":"true"} this allows mlflow to communicate with invalid ssl servers
  3. Run it manually and IF your server has self signed ssl it will run with no issues.
  4. Schedule this ingestion for daily task
  5. See the scheduled tasks are failing due to ssl error. It seems {"MLFLOW_TRACKING_INSECURE_TLS":"true"} is not passed to venv correctly

Expected behavior
I above scenario it should finish successfuly
Screenshots
image

@am2222
Copy link
Author

am2222 commented Jun 18, 2024

related issue #10739

@hsheth2
Copy link
Collaborator

hsheth2 commented Jun 25, 2024

Which version are you on? I believe this was fixed by #10195

@am2222
Copy link
Author

am2222 commented Jun 26, 2024

Thanks @hsheth2 so I am using DATAHUB_VERSION=v0.13.2 How can I check which version includes that fix?

@hsheth2
Copy link
Collaborator

hsheth2 commented Jun 27, 2024

@am2222 it was part of 0.13.3

@am2222
Copy link
Author

am2222 commented Jun 27, 2024

@hsheth2 oh so I am one version low! Thanks!

@hsheth2
Copy link
Collaborator

hsheth2 commented Jun 27, 2024

Going to close this for now - let me know if you have issues after upgrading.

@hsheth2 hsheth2 closed this as completed Jun 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Bug report
Projects
None yet
Development

No branches or pull requests

2 participants